Problems with MOOC research

Like Frances Bell and Roy Williams, I too have listened to Stephen Downes’ recent presentation to a German audience in Tubingen, Germany. Thanks to Matthias Melcher for sharing the link.

Digital Research Methodologies Redux 

May 26, 2014: Keynote presentation delivered to, Tübingen, Germany

Screen Shot 2014-05-28 at 15.02.37

The image above is a screenshot. You can access the presentation on OLDaily  – or at 

Also like Frances and Roy, I found the presentation very thought-provoking and relevant to the research I am doing with Frances and Mariana Funes on rhizomatic learning – and the research I am doing with Roy on emergent learning. Both these areas of research are trying to discover more about how people learn in open learning environments, such as MOOCs.

But Stephen is skeptical about the possibility of doing any worthwhile research into MOOCs if we continue to take a traditional approach to research, which he describes in this slide:

Screen Shot 2014-05-24 at 11.01.22

(This is a screenshot – not a video!)

I would expect most people who work in education to recognize this model (the hypothetico-deductive model), particularly those with a science background and those who read research papers, many of which follow this model.

Stephen’s argument is that much of the research that has been done into MOOCs or is being done into MOOCs, using this model, presupposes its own conclusions, i.e. you find what you are looking for because that is what you expect to see. In addition there can be, in this type of research, the implication or assumption that it is possible to find some sort of ‘truth’ or a ‘universal theory’, but Stephen believes that observation and experience are the foundation of knowledge and that there is no truth that can be known before experience just by thinking about it; we can also question whether there is any ‘truth’.

So a key question for this presentation was ‘Which methods and theoretical conceptions are appropriate for MOOC research?’

The suggestion was that traditional approaches to research do not account for the horribly messy, complex, always changing world in which we are now living and conducting research. There are no universal theories. Whilst we may walk daily on the earth assuming and believing that the ground will not open up and swallow us, this can in fact happen (see YouTube videos on ‘sink holes’). Our generalizations about the world come about as a result of our experience of the world and are not based on any underlying principle.

Whilst I come from a science background and have published research which would be recognized as using the traditional approach Stephen describes, what he says resonates with me, in particular the discussion about research being like learning a language.

Having just spent three days speaking only Portuguese with some Brazilian friends, and having lived in Brazil in the past, I know from experience that learning a language requires immersion – that this is extremely messy, that at the start we only understand a fraction of what is being said, that there are all sorts of cultural nuances that take years to assimilate, that there are words and even ways of thinking that simply do not translate, that understandings are context-dependent and often out of our control, and that communication can be an illusion. It is only through continuous and/or continual long-term immersion that recognizable patterns of understanding eventually emerge.

Research conducted from this perspective, i.e. a perspective of immersion, communication and collaboration is a process of exploration and discovery. It does not set out to ‘prove’ anything or necessarily to draw conclusions and reify knowledge. It recognizes that there is no one right way of describing the world. If there is a theory, it will emerge from the totality of the work. Developments around MOOCs have happened serendipitously through design-led research, rather than through research-led design. (See Liz Sander’s paper – An evolving map of design practice and design research).

All this makes sense to me, but that doesn’t mean to say that I have done or do research like this. Nor does it mean to say that I can easily put these ideas into practice. It is difficult not to presuppose some conclusions, particularly if I have been involved in the context which I am researching. It is difficult to remain objective if I am fully immersed in the context I am researching. It is difficult to prevent bias and subjectivity creeping in. And it is difficult to be credible if I appear to be ‘tinkering to see what happens?’

But I do agree that MOOC-related research necessitates the description of emergent phenomena rather than the identification of something that is true, and hence my interest in emergent and rhizomatic learning and in how Stephen’s ideas and presentation can inform this.

14 thoughts on “Problems with MOOC research

  1. Glenyan May 29, 2014 / 8:33 pm

    Interesting post, Jenny, thanks. The connection to language is a useful one I feel, it may be worthwhile to remember some points though – learning a second language is in many ways a different process than learning a first. As well, as great as experience and immersion is, it is very easy for learners to have highly demotivating experiences, and even experiences that are harmful to skill progression which may develop into harmful habits – structure and guidance is often needed, especially in some non-western cultures. Maybe a message of a balanced approach is better than one perspective over another?

  2. jennymackness May 30, 2014 / 7:37 am

    Glen – thanks for your comment. I agree that we need a balanced approach (to pretty much everything?) and can see that in my post I failed to report that Stephen Downes said that he was going to present his point of view, but that did not mean that he was saying that everyone else was wrong. I also failed to report that he said that grammar and spelling etc. are needed in the learning of a language but for him they are more useful after the immersion process. I think his point was that learning the grammar and the rules on their own doesn’t help us to learn the language, just as learning the rules doesn’t help us to do good research, although we still need both.

    So I can see that I didn’t report on his talk in balanced way – but maybe that’s the point. We all select what we want to represent and how we want to represent it. Balance and lack of bias is so difficult to maintain.

    It’s not that we don’t need the grammar, spelling etc. I certainly need more for my spoken Portuguese which I know is incorrect in terms of grammar and no doubt I have an appalling accent too – but I am not afraid to open my mouth and just speak. I just get in there – and through the mess and asking lots of questions and for words and correction, I do finally get to improved vocabulary and grammar, although I know that I still make lots of errors. But are these harmful? I’ll have to think about that. As you point out in your most recent post, so much depends on the context.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s