Promoting discussion in FutureLearn MOOCs

                

MOOCs: Back to the Future. This was the title of a lunchtime seminar I attended last week in the Educational Research Department of Lancaster University, UK.

In this seminar Phil Tubman  (PhD student with Educational Research and School of Computing and Communications, and also Senior Learning Technologist in the ISS eLearning team at Lancaster), shared the progress he has been making on his PhD into whether MOOCs are living up to their promises. The title of his talk was chosen to suggest that with respect to MOOCs we currently have our backs to the future. As such we need to turn round and look for ways forward.

Given that he works for Lancaster University Phil is well placed to explore social learning in FutureLearn MOOCs. Lancaster University is a FutureLearn Partner.  Phil is interested in the problems of sustaining discussion in online forums and the role of the FutureLearn platform in this; FutureLearn claims to be a social learning platform.

Phil and his colleagues have looked at discussion threads in forums and found that they don’t include many members and that conversations often do not extend beyond a first reply. These findings are supported by other researchers. Phil and his colleagues have therefore designed an intervention which they hope will increase and improve interaction and social learning in these  forums. This is an interactive word cloud which they have called a Comment Discovery Tool (CDT). This tool displays the top 200 words from a forum thread as a word cloud. MOOC participants can then click on a given word to display all the comments which include that word. The example shown in the seminar was from FutureLearn’s MOOC on Wordsworth. Phil clicked on the word ‘daffodils’ in the word cloud which took us to all the relevant forum posts. Having done this, it would then be possible for the MOOC participant to follow the discussion from there without having to wade through potentially hundreds of forum comments and threads. Phil is continuing to develop this tool for his PhD and is interested in whether and how the comment discovery tool will affect interaction in the FutureLearn forums.

I look forward to seeing how this develops.  Phil and his colleagues, Phil Benachour and Murat Oztok have submitted a paper on this to the Learning Sciences Conference 2018, and a recording of Phil’s seminar will ultimately be posted here – http://www.lancaster.ac.uk/educational-research/news-and-events/seminar-series/

#OpenEdMOOC – Final thoughts

(Source of Image – https://youtu.be/iHmHWVHmyfQ –  0.56)

Like Rebecca Heiser, I have been reflecting on the EdX Introduction to Open Education MOOC.

The title of the MOOC led me to expect a broad discussion about Open Education and in my first post about the MOOC, I wrote:

My hope is that it will offer a fresh perspective and rekindle the enthusiasm I had for open education in the early days of the first MOOC in 2008, but which I have found increasingly difficult to sustain in the last couple of years.

It turned out that my hope was not realised. The MOOC focused on Open Education Resources, which I think is just one aspect of open education, and I have to say the aspect that is of least interest to me since I work independently of an institution and no longer teach, so although I come up against some of the issues presented in the course materials, they do not have the impact on me that they might have on others. This was the syllabus for the course.

Week 1: Why Open Matters
Week 2: Copyright, The Public Domain, and The Commons
Week 3: The 5R Activities and the Creative Commons Licenses
Week 4: Creating, Finding, and Using OER
Week 5: Research on the Impact of OER Adoption
Week 6: The Next Battles for Openness: Data, Algorithms, and Competency Mapping

Disappointingly for me there was little discussion of open pedagogy or open teaching. But I did stick with the MOOC. I found the videos of discussions between George Siemens and David Wiley interesting, as were the videos by Norman Bier and other resources provided, and I was particularly grateful for the alternative thought-provoking perspective of Stephen Downes. Did I learn a lot? I don’t think so. I think this was because OER isn’t of huge interest to me, but also because there was minimal discussion between participants. But I don’t feel negative about the MOOC, just that it didn’t rekindle my early enthusiasm for open education.

Rebecca Heiser  has thrown down the gauntlet twice during the course and been largely ignored. First, at the end of the course she tweeted:

So will we get to see Open Analytics in #OpenEdMOOC? What were we clicking? What were our activity-levels? Hashtags generated? Most importantly what were our most discussed topics, was knowledge transferred? #OpenPL Did we form our own network? Will we continue the conversation?

I saw that Stephen Downes picked this up (I think in OLDaily), but otherwise no-one responded to these good and legitimate questions.

And then Rebecca tried again with a post reflecting on her experience of this MOOC.

I can’t help but be critical of the #OpenEdMOOC, Introduction to Open Education, hosted by edX. With as much excitement that was generated by the Open Community prior to the start of the MOOC, I feel that we were let down by Week 3 of the course. Here’s a recap of my critique by week…

This time she received a response (notably not from George or David) and a few comments.

Geoff Cain commented:

This was pretty ironic for me. MOOCs are the only educational space where instructors and institutions are comfortable with the “Set It and Forget It” model of teaching. Despite everything that we have learned about what makes for a successful online class, all the rules change when it comes to MOOCs. As an instructor who has worked in developmental education and education support my whole life, that model seems like a huge step backwards.

This, and Rebecca and Lena Patterson’s comments, seems to me what we should have been discussing, i.e. some of the more difficult consequences of open education and the questions that remain unanswered.

Having participated in and completed a number of MOOCs, I was not surprised by the “Set It and Forget It” model of teaching that Geoff mentions. From the very first MOOC (CCK08) the idea has been that participants teach each other and take responsibility for doing this, because in a massive networked environment it is impossible for one person (the tutor) to interact with everything that is going on. But the complexity of this as a model for teaching and learning, and whether it is an effective model, has yet to be adequately researched.

Thank you to Rebecca for bravely raising these questions in an environment which is known to reward consensus and punish dissent!

#openedMOOC Week 1: The value of ‘open’

Link to source of image

Why Does Open Matter? This is the key question for Week 1 of the Introduction to Open Education MOOC, being offered on edX by David Wiley and George Siemens.

This question has been asked of participants, but since David Wiley and George Siemens have both fully answered the question in a two-part video, I suppose the question for participants should really be, Why does open matter to you?  And the assumption seems to be that we are talking about openness online as opposed to offline.

David Wiley believes open to be a value, like diversity and that openness is imperative for increasing access to, affordability and effectiveness of, engagement and vibrancy in education. He writes that “To be true to the deeper ethic of open we must be generous and open-hearted, feeling a sense of love, care, and responsibility for all humanity.”

George Siemens talks of the benefits of quick, frenzied, open knowledge generation.

Neither of these responses work particularly well for me. I have personally experienced the opposite of ‘love, care and responsibility for all humanity’ in the open environment. Openness online can encourage an ‘anything can be said’ attitude, presumably because the recipient of the comments cannot be seen. As Lisa Lane has written “…. we now have an appalling acceptance of unacceptable behavior and uncivil conduct, which in my country has now reached the highest levels of power.”

And quick, frenzied knowledge generation doesn’t work for me in terms of learning. I can understand the excitement generated which I acknowledge can be motivating, but for learning and knowledge production I personally need slow, quieter interaction, where everyone has an opportunity to be heard, not just the loudest voices.

But like Lisa, I am an advocate of open education and I am grateful to all those like David, George and Stephen Downes, who have done so much to promote it. As David said in one of the videos, open is beyond free. Even in countries, such as Germany, where education is free, open can unlock new pedagogies. David also said open matters because if we learn by ‘doing’ then anything that constrains that ‘doing’, e.g. copyright restrictions, prevents learning. For me that is a powerful argument in support of open education, but I would add, as mentioned above, that some online behaviours can be equally restrictive. This is the aspect of open education in which I am most interested, i.e. I am interested in both the rhetoric and the reality for individual learners, although I suspect, as Andy Lane (2016) has argued, that currently the reality does not measure up to the rhetoric.

However, in the meantime, I continue to benefit from open education; in fact my work as an independent researcher depends on it. In return, as I have written about before,  I try to be an open practitioner, within the constraints of my own capabilities and personality!

Reference

Lane, A. (2016). Emancipation through Open Education: Rhetoric or Reality? In T. eds. Blessinger, Patrick and Bliss (Ed.), Open Education: International perspectives in higher education (pp. 31–50). Open Book Publishers ,. http://doi.org/10.11647/OBP.0103.02 Retrieved from: https://www.openbookpublishers.com/product/531/open-education–international-perspectives-in-higher-education

#openedMOOC begins October 1st

David Wiley and George Siemens are offering a new 6 week MOOC – Introduction to Open Education – on EdX at the beginning of October.

There is already a Twitter hashtag – #openedMOOC –  and you can enrol on the EdX website where you can also find the course syllabus:

Week 1: Why Open Matters
Week 2: Copyright, The Public Domain, and The Commons
Week 3: The 5R Activities and the Creative Commons Licenses
Week 4: Creating, Finding, and Using OER
Week 5: Research on the Impact of OER Adoption
Week 6: The Next Battles for Openness: Data, Algorithms, and Competency Mapping

I have signed up for the MOOC, mainly out of curiosity. My hope is that it will offer a fresh perspective and rekindle the enthusiasm I had for open education in the early days of the first MOOC in 2008, but which I have found increasingly difficult to sustain in the last couple of years.

Despite this, I remain an advocate of open education in the terms in which it was first offered. It would be difficult not to wish for a global democratic education system which offers free open access to all no matter what their circumstances – or is that an erroneous assumption? I am hoping this course will take a critical approach, encourage diverse perspectives and be willing to surface and challenge assumptions, such as the assumption that ‘open is good’, as implied by the header on the EdX course site.

Source of Image: EdX website 

In 2012 Stephen Brookfield wrote that “critical thinking involves three inter-related phases:

  1. Discovering the assumptions that guide our decisions, actions and choices
  2. Checking the accuracy of these assumptions by exploring as many different perspectives, viewpoints and sources as possible
  3. Taking informed decisions that are based on these researched assumptions 

(Informed decisions are based on evidence we can trust, can be explained to others and have a good chance of achieving the effects we want).”

It is getting increasingly difficult to recognise evidence we can trust. We know that over time ‘open’ has led to as many problems as solutions, not least the pursuit of ‘fame and glory beyond your wildest dreams. Or, at least, a few thousand views’ that David Wiley writes about in his blog post. Is this what we really want from open education? I have recently wondered whether one of the problems of ‘open’ in relation to networks is that it is so often discussed out of context, i.e. out of the context of the principles of networks expounded by Stephen Downes, who believes these to be autonomy, diversity, openness and connectivity. He has written about this many, many times over the years, but here is one reference.

Downes, S. (2010, Oct 26th). What is Democracy in Education?http://halfanhour.blogspot.co.uk/2010/10/what-is-democracy-in-education.html 

I see these four principles as being interdependent, i.e. they should be thought about in relation to each other and the absence of one will have consequences for the others. For example, openness without diversity simply leads to echo chambers. In addition, autonomy is a key principle. An open network must respect personal autonomy. My perspective is that loss of diversity and lack of respect for autonomy is an increasing problem in open networks. Hopefully we will get to discuss some of these issues in the MOOC.

David and George  on their blogs, have asked that we create a 3-5 minute video sharing our perspectives and experiences regarding one or more of the weekly topics. I have exercised my autonomy by deciding not to do that but to begin my thinking here on this blog. But I will point you to the videos Stephen Downes has created in response to this request. He is always a hard act to follow!

Here is the first one:

And here are the links to the others, one for each week

Week 1: https://www.youtube.com/watch?v=FPHYAFcUziA
Week 2: https://www.youtube.com/watch?v=UVVULztlp1s
Week 3: https://www.youtube.com/watch?v=CKaJNTgwHWc
Week 4: https://www.youtube.com/watch?v=m3S3xOK6-GA
Week 5: https://www.youtube.com/watch?v=ic1sRq46hys
Week 6: https://www.youtube.com/watch?v=wT_IaZG797

Reference

Brookfield, S. (2012). Developing Critical Thinkers. Teachers College, April 20th & 21st. p.14 http://www.stephenbrookfield.com/s/Developing_Critical_Thinkers.pdf

Critical Examination of MOOCs by Jeremy Knox

Jeremy Knox’s book – Posthumanism and the Massive Open Online Course. Contaminating the Subject of Global Education was published early this year.  I bought the book soon after it was published but have only this summer got round to reading it.
cover

It’s a pity that this is not an open access e-book, which might have received more immediate attention and discussion. I think it does deserve to be discussed since Knox questions whether MOOCs really have been revolutionary and disruptive saying in the introduction,

‘MOOCs have emerged simply as the latest in a long and established line of educational endeavours premised on the nurturing and refinement of a particular kind of human being: one that thinks in a reasoned way; has a natural capacity for independence; and which shares these exclusive traits with all others assumed to be of the same species’(p.2).

He argues that despite the differences between xMOOCs and cMOOCs, ultimately they both promote humanist assumptions of universalism, essentialism, autonomy and transcendental subjectivity.

The problems with these assumptions are explored through Chapters 2, 3, 4 and 5 of the book, in which he develops the following arguments.

The assumption of a universal humanist subject:

  • has been at the heart of the design of MOOCs which emanate from the West, thus resulting in a new form of colonialism, where instead of acquiring geographical territory MOOCs acquire data. Knox calls this ‘data colonialism’ and uses visualisations of the globe and global barriers, with visualisations of global enrolment numbers in MOOCs to support this view.
  • homogenises MOOC participation and ‘[…] forbids internal difference as well as societal difference, and acts to continually close down the possibilities for alternative, immanent relations with the richness and diversity of the world.’ (p.212)

Knox argues that participation in MOOCs is measured through visible activity, retention and completion rates and ‘lurking’ or associated non-visible activity (i.e. difference) is seen as problematic. This view is supported by the number of research outputs that focus on completion and retention rates. ‘[…] ‘lurking’ is made visible only in the form of a negative response to the specific data capture and quantification strategy’ (p.101). Rather than embrace the diversity of MOOC participants a lot of research has focused on categorising participants. Knox sees the attempt to quantify participation as another colonisation practice.

He also sees the promotion of personal learning networks (PLNs) as a promotion of a focus on the individual humanist subject, which seems to be at odds with the open, sharing, networked learning that MOOCs, particularly cMOOCs, aspire to.

‘[…] the PLN seems to reinforce the idea of MOOC education as a self-determining and self-centred endeavour.‘ (p.115)

  • privileges bounded and located place and face-to-face teaching and learning, maintaining institutional elitism and inequality and promoting in/out boundaries and campus envy. Knox uses the very successful MOOC, Modern and Contemporary American Poetry as an example of a MOOC which uses the campus–based location to promote a sense of place.
  • fails to take account of ‘the complex relations between human action and algorithmic execution, resulting in an impoverished grasp of the way MOOC spaces are enacted’ (p.213) and the influence they can have on each other, how they ‘contaminate’ each other. To support this argument he uses examples from the E-Learning and Digital Cultures MOOC (EDCMOOC) which with colleagues from Edinburgh University he helped to design and deliver in partnership with Coursera. He writes of how learning spaces in this MOOC were not stable but produced through movement and transition and ‘the entanglement of human users and non-human algorithms which create contaminated spatial orderings’ (p. 178).

Given my own involvement in MOOCs and MOOC research since 2008 I can see lots of parallels between Knox’s work and my own research. The notion of MOOCs promoting a new form of Western colonialism makes sense to me, as does an ethos of ‘tyranny of participation’ which I first started to think about in 2007 after a discussion with Vivien Hodgson about the paper she was to present with Debra Ferreday at the 2008 Networked Learning Conference.

My research has also highlighted concerns with the homogenising tendency of MOOCs (Tschofen & Mackness, 2012; Mackness & Bell, 2015)

And from recent research with Frances Bell and Mariana Funes (Bell, Mackness & Funes, 2016) I know that social media algorithms can contaminate spatial orderings and that technology is not neutral.

Even the discussion that the ModPo MOOC’s promotion of a sense of place might result in a form of elitism seems a reasonable argument, but it was this argument that made me realise where I stand in relation to Jeremy Knox’s points of view.

I have been a participant in the ModPo MOOC twice and it stands out for me as one of the best and most stimulating MOOCs I have enrolled in. Having had this experience and looking back through Chapter 4 of the book – Housing the MOOC – I find I have 10 different notes in the margins stating that ‘I don’t agree’ or words to that effect. Whilst, when participating in the MOOC, I was aware that the Kelly Writer’s House (the physical space and place from which the ModPo MOOC was filmed) was inaccessible to me in terms of location, not for one minute did I experience this as exclusion. In fact it had the exact opposite effect. I thought that creating such a unique and ‘real’ but virtual sense of place greatly increased my involvement in and positive experience of the course. It was one of the elements of the MOOC that impressed me.

This means of course that in Jeremy Knox’s terms I must be invested in the humanist subject in relation to education. On thinking about this I realise that that is exactly what I am. I believe that first and foremost learning is a human endeavour, one that relates to issues of identity (Wenger, 1998) and a transformation of ‘being’ (Barnett, 2007; Freire, 1970). Currently I am learning ‘to be’ a researcher. This is turning out to be a very long on-going protracted process. I network, collaborate and engage with a wide range of people and technologies and am at least somewhat aware of the effects of algorithms; I know I am not an island. I am influenced by whatever is in my environment, just as whatever is in my environment is influenced by me.

But for me, learning is ultimately about me. I am unique, not in an arrogant sense, but because my experience of learning, the community, the environment, the technology is unique to me. It can be similar to someone else’s experience but not exactly the same. I think this is what Stephen Downes recognises in his work on personal learning networks and in his talk The MOOC of One.

There are paradoxes in the delivery of MOOCs which I think Jeremy Knox has been successful in uncovering. His book is a thought-provoking critique of humanist assumptions surrounding the design and delivery of MOOCs, which I think are well worth engaging with. His concerns related to homogenisation, the tyranny of participation and the influence of social media algorithms on social interaction and learning in MOOCs are very similar to my own.

If you are interested in MOOCs then I can recommend reading this book.

References

Barnett, R. (2007). A will to learn: Being a student in an age of uncertainty. Open University Press.

Bell, F., Mackness, J. & Funes, M. (2016). Participant association and emergent curriculum in a MOOC: Can the community be the curriculum? Research in Learning Technology.

Freire, P. (1970). Pedagogy of the Oppressed. New York: Continuum.

Knox, J. (2016). Posthumanism and the Massive Open Online Course. Contaminating the Subject of Global Education. Routledge

Mackness, J. & Bell, F. (2015). Rhizo14: A Rhizomatic Learning cMOOC in Sunlight and in Shade. Open Praxis. 7(1), p. 25-38

Tschofen, C. & Mackness, J. (2011) Connectivism and Dimensions of Individual Experience. International Review of Research in Open and Distance Learning.

Wenger E. (1998) Communities of Practice. Learning, Meaning and Identity. Cambridge University Press, Cambridge, UK.

Third Research Paper on Rhizomatic Learning

Slide 3

Source of image: Making sense of the Rhizome Metaphor for Teaching and Learning

Today our third paper about learning in the Rhizomatic Learning: The Community is the Curriculum MOOC (commonly known as Rhizo14) has been published. Here are links to the three papers.

Third paper: Bell, F., Mackness, J. & Funes, M. (2016). Participant association and emergent curriculum in a MOOC: Can the community be the curriculum? Research in Learning Technology.

Second paper: Mackness, J., Bell, F. & Funes, M. (2016). The Rhizome: a problematic metaphor for teaching and learning in a MOOC. 32(1), p.78-91 Australasian Journal of Educational Technology.

First paper: Mackness, J. & Bell, F. (2015). Rhizo14: A Rhizomatic Learning cMOOC in Sunlight and in Shade. Open Praxis. 7(1), p. 25-38

At the end of our first paper, in which we explored alternative perspectives of learners’ experiences in this MOOC, we wrote:

In future writing, we will explore:

  • Interrelated processes of community and curriculum formation in Rhizo14
  • The positive and negative effects of emotion and alienation
  • Moderation and leadership roles in the design and conduct of de-centred courses
  • Distributed spaces, technologies and services in a multi-platform MOOC
  • The rhizome as a metaphor for teaching and learning

I think we have written about all these points, although not as separate points and some have been covered more implicitly than explicitly.

All these papers have been published in open journals and have been openly discussed by a group of Rhizo14 participants. I think most researchers would be gratified that their papers are noticed and discussed. At the beginning of the year Veletsianos and Shepherdson (2016) published a systematic analysis and synthesis of the empirical MOOC literature published in 2013-2015 in which they commented ‘that a select few papers are widely cited while nearly half of the papers are cited zero times’. In other words a lot of research goes unnoticed.

It is too early for this research on Rhizomatic Learning to have received a lot of citations. I know from an early paper that colleagues and I wrote about CCK08 (the first MOOC) that it took two years for the paper to be noticed, but since then it has been cited a number of times.

In the meantime these three papers on rhizomatic learning have not gone unnoticed. Currently they are being discussed in the Rhizo15 Facebook group. This is rather ironic, since this third paper raises the problems, based on evidence, associated with using Facebook for discussion. For this reason we have asked for comment and discussion of the papers, which we welcome, to take place on our blogs. Here is the link to Frances’ blog post – http://francesbell.com/research-in-learning-technology/participant-association-and-emergent-curriculum-in-a-mooc-can-the-community-be-the-curriculum/

I have learned a lot, on so many levels, from these two years of research, which has all been voluntary, unfunded and collaborative and which will inform my future work.

Reference

Veletsianos, G., & Shepherdson, P. (2016). A Systematic Analysis And Synthesis of the Empirical MOOC Literature Published in 2013-2015. The International Review of Research in Open and Distributed Learning. 17(2), Retrieved from: http://www.irrodl.org/index.php/irrodl/article/view/2448/3629  

Further thoughts about our presentation: 10th Networked Learning Conference 2016

This week Jutta Pauschenwein and I presented a paper Visualising Structure and Agency in a MOOC using the Footprints of Emergence Framework at the 10th Networked Learning Conference in Lancaster, UK. Here are the slides for our presentation.

On their own these slides do not make a lot of sense, so we have added notes, which you can see by going to the Slideshare and clicking on Notes. Or here is a PDF with the notes: NLC2016-slides-notes-mackness-pauschenwein

We were pleased that our session generated a few comments and questions. Helen Crump commented that she had used the Footprints Framework in the past and found it interesting. You can see Helen’s footprint here.

I don’t remember the exact questions we were asked at the end of our presentation, but there was some discussion about whether it is realistic to think you can get an alignment between the teacher’s design intentions for a course and the learners’ experiences. Jutta responded to this saying that she wasn’t trying to get an alignment, more to know where there is misalignment, so that she can adapt her course.

We were also asked whether it would be possible to use the Framework with large numbers as in MOOCs. In Jutta’s MOOC, 49 learners  (out of 460) voluntarily drew footprints, without any input other than a video. Normally we would recommend a one hour workshop as a minimum for introducing the Framework to new users. For a long time we have wanted to create an electronic version of the Footprint drawing tool, which would not only allow learners to draw and share them more easily, but would also collect written comments and reflections on the factors and automatically score the footprints for further analysis. For this paper we did this manually. So there is still plenty of scope for development, which is needed if we want to use the Framework with large numbers of users.

I don’t know if other paper authors do this, but I usually try and anticipate the questions we might get. This time, before the conference, I gave the paper to a friend asking for potential questions that might arise from the paper. Although these questions were not asked at the conference, I will share them here (and my answers) as I found them useful in preparation for our session. So here they are:

  1. What does structure and agency mean?

For our work ‘structure’ means how a learning design balances openness and structure and how this balance is implemented in relation to the interactivity it enables.

‘Agency’ means the extent to which learners are enabled to develop their capability for effective action on their own terms; how the environment enables them to explore, establish, network and present themselves, their ideas, aspirations and values through writing and presence.

  1. Why are structure and agency appropriate lenses to study learner experience and teacher/designer role?

Structure and agency are particularly appropriate lenses to study the learner experience and teacher/designer role in the context of MOOCs, because MOOCs are changing the shape of structure and agency in open online learning environments and these changes are beginning to affect more traditional learning environments – as we see in practices such as the flipped classroom. MOOCs are complex learning environments, in which the structure may be very loose/open and in which learners may have considerable autonomy. Ashwin (2012) has argued that research into teaching and learning interactions in Higher Education has consistently looked the interrelationship between structure and agency. He also argues that there has been a tendency in research to separate the learner experience from the teaching experience. Like him, we believe that a holistic view is needed to understand the learning experience in MOOCs.

  1. What is the relationship of structure and agency to the clusters in the footprints model?

In this paper we consider structure and agency in terms of the Footprints of Emergence Framework. The Framework takes a holistic view of learning in prescribed and open learning environments recognising the interrelationship between structure and agency and how they influence each other. In this framework 25 learning characteristics are organised into four clusters. Two of these clusters – open structure and interactive environment, relate to the design or structure of the learning environment, and two to learner agency.

  1. Did the MOOC participants really understand the footprints tool?

It is difficult to know this without interviewing them. Experience with the Footprints has shown that ideally users are introduced to the Footprints through a workshop and that when this is done, it doesn’t take long for people to understand how to use them and draw a Footprint. But the value of the footprints is in the depth of reflection that they can invoke if discussed. Interviews were not part of this research but will be for future research. One of the difficulties has been in translating the Footprint characteristics explanations into German. What we do know is that participants voluntarily drew and shared their footprints, learning how to do this by watching a video.

  1. How can you be sure that the cluster elements are as influential in shaping the learning experience as you claim given the importance of autonomy and self-determination of the learner?

The cluster elements are intended to reflect the learning experience, rather than shape them. They are based on our own experience of learning in open and prescribed environments and of our knowledge of educational research and educational theorists. When we designed the Footprints of Emergence Framework and worked on the factors, we considered them to be a palette. In other words, we did not say that they were the definitive list of characteristics that describe the learner experience. We offered them as a palette of characteristics which could be selected from or which could be replaced by other characteristics. Our experience has shown that these 25 factors provide a rich picture of the learning experience, but a designer using the footprints drawing tool is free to add or take out factors.

  1. Emergent learning is not predictable as you say and the participants were asked not to be too precise about positioning their responses when drawing footprints – doesn’t this make the footprints tool a very crude and approximate way of measuring the efficacy and effectiveness of the learner experience?

The Footprints of Emergence Framework is not intended to measure the efficacy and effectiveness of the learner experience. Principally it is intended as a tool for promoting deep reflection on the learning, design or teaching experience. But in our research we realised that if we scored the Footprints retrospectively and objectively, then it is possible to compare footprints and begin to explore the balance between structure and agency, prescription and emergence. Reflection is never a precise measure. 

  1. How much credence can we give to the scores given the observation in Q6?

As much credence as any evaluation tool. The factors were scored against a given range independently by each of us and then compared. We used a scale of 1-30 for the spectrum of prescribed learning to the edge of chaos. Any disparities (which were minimal) were discussed and then the final score agreed. The scores in themselves are not important. The patterns that the scores might reveal are of more interest. We are confident that the patterns that emerge through this process, can inform us about the balance between structure and agency, but do not see this as a precise measure. All we can say here is that the majority of MOOC participants had an emergent learning experience and that the MOOC design was successful in assuring a balance between structure and agency.

  1. Isn’t there a risk that given the sample who drew the footprints – largely students on a course run by one of the researchers – that the data was biased to how the students perceived the intentions of the researcher?

Yes and that is acknowledged in the research. What is needed is to interview the participants who drew footprints, and ideally these interviews would be carried out by a researcher not known to the participants. Past experience in earlier stages of the research when the drawing tool was used with individuals has shown that interviews are needed to ensure that the final footprint drawing is an accurate representation of the users’ experience. This research using the Footprints of Emergence Framework at scale is in its early stages. Future research will include interviews of participants.