Assessment of discussion forum posts in ModPo

Today I am disappointed in ModPo for the first time. Why? Because, I realize that it has fallen into the trap of believing that requiring posts to a discussion forum can in some way measure the success of learning.

On checking I see that it does say this on the Announcements page in the very first post ‘a thought on plagiarism’, but I failed to notice it until it was mentioned in the audio discussion between Al Filreis and Julia Bloch that was posted today.

To be considered a student who has “completed” the course, you need to have written and submitted the four short essays, commented on others’, submitted (and minimally ‘passed’) the quizzes, and participated in the discussion forum.

Evidently to get a certificate of completion, a ModPo participant must make a post in each week of the course, in one of the staff initiated weekly forums.

I completely fail to see the point of this. It is not as if ModPo is short of discussion in the forums. It is completely swamped with discussion. In addition it is the kind of assessment requirement that tempts me to simply ‘play the game’ (if I was that keen to get a certificate, which I am not). I could put any meaningless post about any meaningless thing in each week’s forum and theoretically I have fulfilled the requirement.

I have already accepted that ModPo is not completely open, simply because it is tied to the Coursera platform and therefore does not have ‘open’ resources in the original cMOOC sense of participants being able to aggregate, remix, repurpose and feed forward resources at will.

But I have otherwise been very impressed by the pedagogy – the standard of teaching is very high, the level of support from and engagement by the tutors is beyond the call of duty for a MOOC, and the content is so stimulating.  All credit to the tutors and TAs.

But this requirement to post to the forums is a definite blip, in my book. Why? – because it puts (in this context) an unnecessary constraint on the autonomy of those learners who would like to achieve a certificate of completion, and won’t necessarily add anything to the learner experience. It definitely wouldn’t to mine.

I have listened to all the videos, read all the poems, completed the quizzes for Weeks 1 and 2, written and submitted my first assignment, but this requirement to post to the forums is one hoop that I will not be jumping through. If there had been a meaningful activity around being engaged in the discussion forums, then I would have been happy to comply. As it is, I don’t feel that I need to learn how to post to forums, I have done lots of this in the past, nor do I need to learn the value of social learning. I have been practicing and promoting it for years. If I feel that I can genuinely make a contribution in a forum, then I will.

Despite this disappointment, ModPo remains a highly stimulating experience, on a number of levels, and one that I would recommend to anyone interested in open learning, pedagogy and poetry.

 

 

12 thoughts on “Assessment of discussion forum posts in ModPo

  1. Scott Johnson September 22, 2013 / 12:04 am

    Hi Jenny, wonder if the posting requirement is an artifact of learning being something that has to be confirmed by others? Learning for yourself without proof of participation undoes some of the justification for taking the course through an institution. Reading through many of the forms our college submitted to accreditation authorities we found all sorts of odd little confirmations of accomplishment that were never struck from the forms. This rule may have replaced the requirement to bring a small lump of digital coal each morning to heat the school-room:-)

    Seriously, in the background of the openness discussion is the question of who decides and how they indicate that learning has occurred. Especially if the learning is necessary to qualifications to practice a trade or profession What each individual in a community knows is, in a sense, a public “good” in the common wealth of the group.

    I think some people see self-directed learning as almost a declaration of disloyalty to the group by the abandonment of structured pathways to membership. Or…as an adult learner you decide what matters to you:-)

  2. Ian Chowcat September 25, 2013 / 5:54 pm

    Hi Jenny, I’m one of the ‘community TAs’ on ModPo having been a student on it last year – I’m also in the UK with lengthy experience in online learning, past involvement with ALT etc. We didn’t have the requirement to post in the forums for certification last year and I don’t know the story behind it being introduced now. I certainly share your doubts about its pedagogical effectiveness. However when you say it is “an unnecessary constraint on the autonomy of those learners who would like to achieve a certificate of completion” I think you are conflating two distinct things. Any certification, even the minimal certification of a MOOC like ModPo, is public currency and hence what has to be done to earn it can’t be entirely within the control of the learner – no more than a learner in conventional education can control the criteria against which they are accredited. And while a minimum of one weekly forum posting isn’t doing much for learning, although it might be a spur for some, it is a way of demonstrating a basic level of participation (and I wonder whether in fact Coursera required it to be brought in for just some such reason). The learner in ModPo retains their autonomy by being able to decide how they want to participate in the course, and there are many ways of doing so. If you choose to go for the certification then I think you are choosing to be bound by the criteria against which the certificate is awarded. But this is no more and no less a restriction on individual autonomy than is engaging in any activity bound by public rules.

  3. scottx5 September 25, 2013 / 8:33 pm

    Don’t have the answer for this but we seem to be trapped in replicating the habits of recognition developed by institutionalized education when MOOCs are expected to explore other possibilities. If these habits are the only proof of learning a person can demonstrate then how do we claim to be moving forward?

  4. jennymackness September 26, 2013 / 7:43 am

    Hi Scott, thanks for your two comments.

    >I think some people see self-directed learning as almost a declaration of disloyalty to the group by the abandonment of structured pathways to membership.

    This is an interesting thought and relates to Stephen Downes’ thoughts on groups and networks, cooperation and collaboration, which he referred to again at the ALT conference that I attended earlier this month.

    And I do agree that we ‘we seem to be trapped in replicating the habits of recognition developed by institutionalized education’, but I don’t think xMOOCs are necessarily exploring other possibilities in these terms. My limited experience of them is that they are focussed on how best to manage large numbers of learners. In relation to assessment, their response so far seems to have been to use, in the main, peer assessment of short assignments, with limited success, quizzes and now, in ModPo, the requirement to post to discussion forums.

    It’ll be interesting to follow developments in assessment in xMOOCs, particularly if they start running xMOOCs for credit, as I think it’s a nut that has not been cracked yet.

  5. jennymackness September 26, 2013 / 8:13 am

    Hi Ian – thanks for your visit and your comment. I had already recognized your name in the ModPo forums.

    >If you choose to go for the certification then I think you are choosing to be bound by the criteria against which the certificate is awarded. But this is no more and no less a restriction on individual autonomy than is engaging in any activity bound by public rules.

    Yes of course, I completely agree, and I have heard Stephen Downes, who has written a lot about autonomy, say something similar.

    I can see that I didn’t express myself very well. A key principle of the original cMOOCs is autonomy and for me that means thinking through what that means and how it might be promoted or constrained at every level of the MOOC. ModPo (like other MOOCs) promotes autonomy in many ways. As you say, learners can choose to participate in the forums or not; they can choose to do the assessment or not; they can choose how much or how little of the content they want to cover and so on.

    I have chosen, for this MOOC, as I did for CCK08, not to participate in the forums. Poetry is new to me and I need space to get my head round the poems. I find the ModPo forums excessively ‘noisy’. However, I can see that if I can get my head around the poetry this time, I might be more able to participate in the forums the next time. I am not anti forums, per se.

    So for someone like me, who wants to learn about poetry, who is willing to take on the challenge of the assignment and have them subjected to peer review (although as I write in the next blog post, I would have preferred to have been asked whether I wanted my assignment posted to 34 000 people), posting to the discussion forums is an ‘unnecessary’ constraint on my autonomy – and it’s the word ‘unnecessary’ that is important.

    Whilst I agree that assessment will bring with it some constraints and that MOOC learners have the autonomy to choose whether or not to do the assessment, I think MOOC convenors, if they value learner autonomy, can think this through even within the assessment requirements. As you will know, there has been plenty of work done on this, e.g. when learners are asked to write their own assessment criteria etc.

    Finally, ‘Just to say’ ☺ – that I would have been happy to comply with the requirement to post to discussion forums, if I had been asked to do anything meaningful, e.g. (off the top of my head), Find x number of forum posts that provide interpretations of Ezra Pound’s black bough. Write a response to each of these saying how your interpretation is similar or different. For your assignment write x words, which summarise what you have learned from this interaction (or words to that effect).

    I can straight away see what the problems of this particular assessment task might be, but hopefully it illustrates what I mean by a meaningful assessment requirement related to discussion forum posts might be.

    Thanks for making me think this through further! Jenny

  6. quillfyre September 26, 2013 / 7:15 pm

    The discussion forums that link to the specific poems each week are meant for students to supplement what they are learning through the videos, the readings and the recordings. I agree that they are noisy. And that some of them do not add anything to the learning. However, if you choose just one of them based on a poem that resonates for you and look for a thread that addresses an aspect of that poem you would like to explore, how can that not add to your learning about poetry?

    There are also some excellent discussions on the nature of poetry and some about the techniques and structure of it. These also might be among the threads under a poem-specific forum And if they are, those ones count too.

    Given the size of the enrollment, I think the teaching staff have supplemented the ways they can be sure they are effectively reaching the students. It also might work to show things that are not as effective as they might wish, and that can lead to improvement.

    I would have thought that the assignment requirement or the quiz requirement might be more off-putting than the requirement of one post in the forums.

    And of course, there is still the fact that the course is under the auspices of an accredited university, which provides the certificate of completion. So I would expect they would have a minimum requirement to be met in order to affix their name to it.

    And anyone is free to take advantage of as little or as much of the course as they want, free of charge. No-one is really required to do any quiz, forum or assignment.

    Most free courses have no requirements but also provide no certificate either.
    In fact, many courses that charge you provide no certificate.

    Carol A. Stephen

  7. Scott Johnson September 27, 2013 / 2:00 am

    Hi Carol, as institutions change is it wise to defend policy by referring to the value of a certificate with an unknown future? As a free and open course the ModPo MOOC will not will not carry the currency of the of the actual credit version anyway so why not free it of restrictions and see what happens without rules? It might be that course quality and the discussions are reward enough for the participants and the certificate has little or no motivating power.

    What the teaching staff have done to make this a great course is important and aids to future design. Of equal value is how participants manage themselves within the course and provide direct evidence that feeds into future design. A different way of presenting education may need different tools for assessment.

  8. quillfyre September 27, 2013 / 3:28 am

    Scott, I was not defending, merely speculating on why an institution might be careful about where it puts its name. And the point was that having to participate in the discussions apparently was the issue.

    As far as providing evidence, how does the staff monitor that evidence if there is no way to know whether a student participated or not?

    One purpose of the discussion requirement is that it illustrates the importance of discussion and shared close readings. One of the main aspects of the course is based on just such close readings of poems.

    Besides, one of the greatest outcomes of this course for me, and for everyone I know taking it, is learning through discussing with so many other students from so many countries, and gaining new perspectives on the poems, on, poetics, and on the poets as well. That is why I am taking it a second time.

    I am not just sitting in front of a computer screen waiting to be fed information, or having canned lectures pass by my eyes, passively taking it in. I am using the platform interactively, and learning much more than I would have in a traditional classroom setting. A bonus is that in some of those very discussion forums we will be able to ask questions directly of the poet whose poem we are studying, as there will be visiting poets taking part in the forums.

    And having said all that, any student enrolled is quite free to just read the material, watch the videos, listen to the audios, and not take the quizzes, nor do the assignments, nor actively participate. For those not motivated by the certificate, they are certainly free to take this route through the course. None of those things are required to access freely all the materials of the course.

  9. Scott Johnson September 27, 2013 / 4:51 am

    Hi Carol, no argument with the course being a valuable and exciting experience for the participants. There is also value for the sponsoring institution as online courses count every click and can be analyzed for some very useful information such as participants more often seeking help from peers rather than from tutors. Or all those discussions creating new understandings of old material that you yourself are contributing to.

    In my case the past experience of open learning causes me to engage with a course, or any other web material, in a manner that is sensitive to determining every action I take. This includes an action that would or wouldn’t get me a certificate. The problem for me is the assumption that conditions placed on my participation are considered reasonable while my tiny bit of independence disqualifies me from the certificate. Yes, it is my choice but the certificate is valued by someone, or it wouldn’t be there and withdrawing it illustrates I am in a transaction with a system separate from my learning. A system that doesn’t seem to value my presence, which is voluntary and I think of as valuable is not uncommon, though I wonder if we need more of them?

    My hope is we are moving away from meaning being made for us to something more open and determined by each as we need.

  10. quillfyre September 27, 2013 / 2:51 pm

    Of course, I have not tried very many of the online courses that Coursera offers. Only about 5 so far. ModPo asks for 4 assignments and one or two quizzes per wk over the 10 weeks. There are no grades attached. The quizzes consist of only 1 or 2 questions, mutliple choice, 10 tries at each quiz. To “pass” you need a score of anything above zero. Highest score is the one that “counts”. Assignments count as long as you submit the four, and comment on 4 others for each assignment. No marks. Just participation. And one post per week about the material we study.

    Other courses have more assignments, one per week. Longer quizzes, fewer tries. One course, the most difficult one for me, gave you one try at the quizzes, and a minimum number of them had to have a grade of 60% or more. (Don’t remember exactly) Not as engaging a course structure either.

    So, on the scale of restrictions, I think ModPo doesn’t really impose much.

    I have tried a few other open, online courses and while there are no requirements, they’ve not measured up from a learning perspective and no interaction with anyone, no feedback either.

    I can only comment based on my own experience. For me ModPo is an excellent one, and I don’t think it deserves a down vote for the minimal requirements it has.

  11. jennymackness September 27, 2013 / 4:16 pm

    Hi Carol and Scott – thank you both for your thought-provoking comments – and first, Carol, I want to thank you for helping me to find my way to the Week 3 webcast on Wednesday. I much prefer being able to see the video than only have the radio/audio, and I was really struggling to find it. So thanks!

    I don’t think we are going to end up agreeing Carol, as underlying all this is a difference in our philosophical positions as to what MOOCs and open education are about. Like you, I think ModPo is doing an excellent job and I have said as much a few times in the various blog posts I have made about ModPo. It had never occurred to me that in making a post about what I perceive to be a valueless assessment requirement (posting to the discussion forums), would be regarded as a ‘down vote’ for ModPo. I don’t think in terms of up votes and down votes (and that is one of the features of Coursera that I personally don’t like), but I do think in terms of authentic student learning experiences and what constitutes a worthwhile assessment activity – and for me, for reasons I explained, the requirement to post to discussion forums didn’t measure up. So I have exercised my autonomy and will as a result forfeit the possibility of receiving a certificate. I would hope that ModPo would be sufficiently open to be able to listen to alternative perspectives, not just about poetry, but also about the course.

    The only other thing I would like to say is that there are many different types of interaction. Not participating in the forums doesn’t necessarily equate with ‘passivity’ and neither does lurking. I remember in the early days of MOOCs when ‘lurking’ was discussed a lot, there was a suggestion that we shouldn’t underestimate the influence of the silent majority, or words to that effect. It’s an interesting thought.

    Many thanks to you both. For now I think I will mostly be interacting with ModPo from here (once described by one of my colleagues as my ‘front porch’ as opposed to the bustle of the market place – the forums). That’s not to say that I won’t visit the market place from time to time.

    So onward to Week 4 and Gertrude Stein. Whether I will be able to make any sense of her poetry remains to be seen ☺

  12. quillfyre September 27, 2013 / 11:05 pm

    You are welcome Jenny. Any time. I suppose I said down vote as that came to mind, probably from spending most of my waking hours on ModPo right now. I guess my concern was that it could be perceived as such by people who might really enjoy the course but not bother with it as feeling it wasn’t worthwhile. Sort of that idea that online you get 15 seconds to make your case idea.
    And I guess because I am so caught up in poetry in its many facets, I get enthusiastic! I missed out on 30 years of it by setting it aside without going deeper into the study of it. Big mistake for me. But I find the discussion forums and the friends I have made there just such a wonderful extension of learning that I don’t mind whether it’s a requirement or not. (For me not, this year. I already have the certificate anyway.)

    So I am rather sad that we will not see you in the discussions, but of course, I respect your decision. And I agree about Gertrude. I am hoping to understand her better this time.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s