This is the last in a series of five posts written in preparation for an e-learning conference keynote that Roy Williams and I will be giving on September 17 in Graz, Austria.
Previous posts relating to this presentation are:
- Evaluation of Open Learning Scenarios
- Characteristics of Open Learning Environments
- Emergent Learning in Open Environments
- Theoretical influences on the characteristics of open learning environments
Our research   focuses on how learners experience complex, unpredictable, uncertain environments, such as MOOCs, where their learning is likely to be emergent.
Over the last two or three years the amount of research into learning in MOOCs has grown. See for example MOOC Research Initiative Reports from the Gates Foundation funded projects and the proceedings from the European MOOCs Stakeholders Summit 2014 .
Some researchers, like George Veletsianos , have questioned whether there is enough emphasis in recent research on the ‘learner voice’. This is a question that also concerns us. We believe that it is essential to encourage and listen to the ‘learner voice’ (whoever that learner might be), if we are to understand the epistemic and ontological shifts and transformational learning that can happen in open learning environments.
The Footprints of Emergence framework is a tool  [ See also previous posts in this series], which can be used by learners to surface the deep, tacit knowledge and understanding that is associated with these transformational shifts in open learning environments such as MOOCs. We are interested in learning more about the impact of open learning environments on these shifts by encouraging learners to be researchers of their own experience.
The Footprints of Emergence framework   can be regarded as a probe for evaluating learning in open environments. It engages learners in deep reflection, supports them in taking control of their own reflection and evaluation, can be used to encourage discussion and collaboration between learners, teachers and designers, and can be used to visualise the dynamic changes that occur in learning over time.
A difficulty that we have encountered with the framework and drawing tool is that they require explanation and practise in use, i.e. they require time and effort to engage with, sometimes more time than people have and more effort than people want to make. Our aim is to try and simplify the process, without losing the depth of reflection that the current process leads to. To this end we are, through a colleague, hoping to develop some software, which will make the drawing process more straightforward. This would leave the user freer to concentrate on the meaning and use of the factors (see the second post in this series for more information about the factors) and the interpretation of the final footprint visualisation. The development of some software would also potentially make it easier to work with larger groups of learners.
Such a development would enable us to focus on the meaning of evaluation of learning in open learning environments. This has been challenging us for some time. If learning in these environments is emergent, surprising and unpredictable, how can we ‘capture’ it and value it. The common response in current MOOC research has been to try and scale up traditional assessment methods through the use of big data, automated assessment or peer review. Our current thinking is that new paradigms such as open learning may require new ways of thinking about assessment. The Footprints of Emergence framework enables a move away from traditional approaches and puts the emphasis on reflection and self-assessment. This aligns with the view expressed recently by Stephen Downes  that we need to move beyond assessment  as we know it and put it in the hands of learners.
To summarise the directions in which we are moving: We are interested in –
- Exploring further the characteristics of open learning environments that result in transformative learning
- Increasing our understanding of how learners learn in open learning environments
- Finding new approaches which go beyond assessment and put learning and assessment in the control of the learner
- Exploring the notion of probes for assessment and learning design
- Developing the footprints of emergence drawing tool so that it can be used more easily with larger groups of learners.
- Williams, R., Karousou, R. & Mackness, J. (2011) Emergent Learning and Learning Ecologies in Web 2.0. International Review of Research in Open and Distance Learning. Retrieved from: http://www.irrodl.org/index.php/irrodl/article/view/883
- Williams, R., Mackness, J. & Gumtau, S. (2012) Footprints of Emergence. Vol. 13, No. 4. International Review of Research in Open and Distance Learning. Retrieved from: http://www.irrodl.org/index.php/irrodl/article/view/1267
- Veletsianos, G. (2014) ELI 2014, learner experiences, MOOC research, and the MOOC phenomenon – Retrieved from: http://www.veletsianos.com/2014/02/10/mooc-research-mooc-phenomenon/
- Footprints of Emergence open wiki – http://footprintsofemergence.pbworks.com/
- Downes, S. (2014) Beyond Assessment – Recognizing Achievement in a Networked World Jul 11, 2014. 12th ePortfolio, Open Badges and Identity Conference , University of Greenwich, Greenwich, UK (Keynote). Retrieved from: http://www.downes.ca/presentation/344
- Mackness, J. (2014). Blog post – Beyond Assessment – Recognizing Achievement in a Networked World. Retrieved from: https://jennymackness.wordpress.com/2014/07/13/beyond-assessment-recognizing-achievement-in-a-networked-world/