OldGlobeMOOC is about to start it’s 4th week (following a week’s break for July 4th celebrations in the US), and the Week 3 assignment peer reviews are in. For me this assessment process is one of the most interesting aspects of this xMOOC. I have thought since the first MOOC in 2008 (CCK08 Connectivism and Connective Knowledge), designed and run by Stephen Downes and George Siemens, that assessment may be the sticking point for MOOCs.
In my last post , I outlined some of the difficulties that OldGlobeMOOC is experiencing with the assessment and peer review process. It seems to me, once again, but this time for an xMOOC, that if MOOCs are going to be sustainable and successful, then the assessment process has to be ‘cracked’ and meaningful.
Some MOOCs have taken the approach of restricting the number of participants who can be assessed. CCK08 did this. I think the number was 25, and FSLT12 and 13 have done this with a similar number – the idea being that a small number of participants can be assessed by a tutor. FSLT13 offers credit for this:
The course has been recently accredited (10 transferrable academic credits at level 7, postgraduate). FSLT is recognised towards the Oxford Brookes Postgraduate Certificate in Teaching in Higher Education (PCTHE) and Associate Teachers (AT) courses. (http://openbrookes.net/firststeps13/)
But these are cMOOCs.
OldGlobeMOOC has taken a different approach as I described in my last post and I understand from other participants that this is similar to a number of other Coursera MOOCs. For me this my first xMOOC, but it is not for quite a few OldGlobeMOOC participants, who have taken numerous Coursera courses and in the forums have shared their experience of the peer review process.
I will add my experience to the mix, and just so you know what we are talking about here are links to my assignments with their peer reviews.
If you read these, you will see that the assignments are not very different in their style and level to my blog posts, i.e. they are not academic pieces of work – rather discussion pieces or personal reflection. And judging by the assignments I have reviewed, other participants’ assignments are of a similar level.
Which brings me to the review process, which I reflected on in my last post, but will add a few things here.
- The idea is that each participant submits an assignment and peer reviews five assignments for each week, which I have done. If this is not done, i.e. the peer review, then a 20% penalty is incurred.
All students wishing to obtain a Statement of Accomplishment must achieve 7 out of 12 points and submit 5 peer reviews each week. If a student fails to complete the 5 peer reviews, that week’s assignment will incur a 20% penalty.
Despite the fact that I definitely submitted five peer reviews for Assignment 3, I received a 20% penalty and therefore scored 1.6 instead of 2. It’s very easy to know that you have completed the 5 peer reviews, by the way the Coursera system takes you through the 5 assignments allocated for review; and the system confirms for you at the end of the process that you have submitted 5 – so I know that I did. So there’s been a blip in the system somewhere. It’s not a big deal for me, as I’m only doing this to experience the process and because I like the assignments and find the discussions interesting. I am not doing the course for the Certificate – but I do wonder how a blip in the system affects people who are really keen to receive a Statement of Accomplishment.
- There is no guarantee that you will receive 5 peer reviews. I received five in Week 1, three in Week 2 and four in Week 3. There has been some discussion in the forums about how this might affect the overall system and whether or not you have to review more than 5 assignments to receive 5 reviews.
- I have no complaints about the quality of most of the peer reviews and so far no one has given me a score of less than 2 – but this peer review for Assignment 3 is indicative of how the game can be played to ensure that you get a Certificate. It made me smile🙂
peer 2 → I’m headed for an airplane so don’t have time to review, and I won’t be back until after evaluation time ends so I’m just giving everyone a 2.
Aside from this here are some further reflections. The OldGlobeMOOC is a great experience in terms of the diversity of participants. Unfortunately the younger participants, in their teens, who signed up, seem to have fallen out of the discussion forums. This does not mean that they are no longer participating through observation and reading – it’s difficult to know. But I have wondered how an 11 year old might review the assignment of an academic Professor, or how an academic Professor might respond to a learner with special needs, or a very young participant, or someone whose first language is not English, and so on. The assignment submission is anonymous. Do these differences have implications for the equity of the peer review process?
Despite all this I am finding OldGlobeMOOC a fascinating and enjoyable experience and am looking forward to the start of Week 4.