Further thoughts about our presentation: 10th Networked Learning Conference 2016

This week Jutta Pauschenwein and I presented a paper Visualising Structure and Agency in a MOOC using the Footprints of Emergence Framework at the 10th Networked Learning Conference in Lancaster, UK. Here are the slides for our presentation.

On their own these slides do not make a lot of sense, so we have added notes, which you can see by going to the Slideshare and clicking on Notes. Or here is a PDF with the notes: NLC2016-slides-notes-mackness-pauschenwein

We were pleased that our session generated a few comments and questions. Helen Crump commented that she had used the Footprints Framework in the past and found it interesting. You can see Helen’s footprint here.

I don’t remember the exact questions we were asked at the end of our presentation, but there was some discussion about whether it is realistic to think you can get an alignment between the teacher’s design intentions for a course and the learners’ experiences. Jutta responded to this saying that she wasn’t trying to get an alignment, more to know where there is misalignment, so that she can adapt her course.

We were also asked whether it would be possible to use the Framework with large numbers as in MOOCs. In Jutta’s MOOC, 49 learners  (out of 460) voluntarily drew footprints, without any input other than a video. Normally we would recommend a one hour workshop as a minimum for introducing the Framework to new users. For a long time we have wanted to create an electronic version of the Footprint drawing tool, which would not only allow learners to draw and share them more easily, but would also collect written comments and reflections on the factors and automatically score the footprints for further analysis. For this paper we did this manually. So there is still plenty of scope for development, which is needed if we want to use the Framework with large numbers of users.

I don’t know if other paper authors do this, but I usually try and anticipate the questions we might get. This time, before the conference, I gave the paper to a friend asking for potential questions that might arise from the paper. Although these questions were not asked at the conference, I will share them here (and my answers) as I found them useful in preparation for our session. So here they are:

  1. What does structure and agency mean?

For our work ‘structure’ means how a learning design balances openness and structure and how this balance is implemented in relation to the interactivity it enables.

‘Agency’ means the extent to which learners are enabled to develop their capability for effective action on their own terms; how the environment enables them to explore, establish, network and present themselves, their ideas, aspirations and values through writing and presence.

  1. Why are structure and agency appropriate lenses to study learner experience and teacher/designer role?

Structure and agency are particularly appropriate lenses to study the learner experience and teacher/designer role in the context of MOOCs, because MOOCs are changing the shape of structure and agency in open online learning environments and these changes are beginning to affect more traditional learning environments – as we see in practices such as the flipped classroom. MOOCs are complex learning environments, in which the structure may be very loose/open and in which learners may have considerable autonomy. Ashwin (2012) has argued that research into teaching and learning interactions in Higher Education has consistently looked the interrelationship between structure and agency. He also argues that there has been a tendency in research to separate the learner experience from the teaching experience. Like him, we believe that a holistic view is needed to understand the learning experience in MOOCs.

  1. What is the relationship of structure and agency to the clusters in the footprints model?

In this paper we consider structure and agency in terms of the Footprints of Emergence Framework. The Framework takes a holistic view of learning in prescribed and open learning environments recognising the interrelationship between structure and agency and how they influence each other. In this framework 25 learning characteristics are organised into four clusters. Two of these clusters – open structure and interactive environment, relate to the design or structure of the learning environment, and two to learner agency.

  1. Did the MOOC participants really understand the footprints tool?

It is difficult to know this without interviewing them. Experience with the Footprints has shown that ideally users are introduced to the Footprints through a workshop and that when this is done, it doesn’t take long for people to understand how to use them and draw a Footprint. But the value of the footprints is in the depth of reflection that they can invoke if discussed. Interviews were not part of this research but will be for future research. One of the difficulties has been in translating the Footprint characteristics explanations into German. What we do know is that participants voluntarily drew and shared their footprints, learning how to do this by watching a video.

  1. How can you be sure that the cluster elements are as influential in shaping the learning experience as you claim given the importance of autonomy and self-determination of the learner?

The cluster elements are intended to reflect the learning experience, rather than shape them. They are based on our own experience of learning in open and prescribed environments and of our knowledge of educational research and educational theorists. When we designed the Footprints of Emergence Framework and worked on the factors, we considered them to be a palette. In other words, we did not say that they were the definitive list of characteristics that describe the learner experience. We offered them as a palette of characteristics which could be selected from or which could be replaced by other characteristics. Our experience has shown that these 25 factors provide a rich picture of the learning experience, but a designer using the footprints drawing tool is free to add or take out factors.

  1. Emergent learning is not predictable as you say and the participants were asked not to be too precise about positioning their responses when drawing footprints – doesn’t this make the footprints tool a very crude and approximate way of measuring the efficacy and effectiveness of the learner experience?

The Footprints of Emergence Framework is not intended to measure the efficacy and effectiveness of the learner experience. Principally it is intended as a tool for promoting deep reflection on the learning, design or teaching experience. But in our research we realised that if we scored the Footprints retrospectively and objectively, then it is possible to compare footprints and begin to explore the balance between structure and agency, prescription and emergence. Reflection is never a precise measure. 

  1. How much credence can we give to the scores given the observation in Q6?

As much credence as any evaluation tool. The factors were scored against a given range independently by each of us and then compared. We used a scale of 1-30 for the spectrum of prescribed learning to the edge of chaos. Any disparities (which were minimal) were discussed and then the final score agreed. The scores in themselves are not important. The patterns that the scores might reveal are of more interest. We are confident that the patterns that emerge through this process, can inform us about the balance between structure and agency, but do not see this as a precise measure. All we can say here is that the majority of MOOC participants had an emergent learning experience and that the MOOC design was successful in assuring a balance between structure and agency.

  1. Isn’t there a risk that given the sample who drew the footprints – largely students on a course run by one of the researchers – that the data was biased to how the students perceived the intentions of the researcher?

Yes and that is acknowledged in the research. What is needed is to interview the participants who drew footprints, and ideally these interviews would be carried out by a researcher not known to the participants. Past experience in earlier stages of the research when the drawing tool was used with individuals has shown that interviews are needed to ensure that the final footprint drawing is an accurate representation of the users’ experience. This research using the Footprints of Emergence Framework at scale is in its early stages. Future research will include interviews of participants.

2 thoughts on “Further thoughts about our presentation: 10th Networked Learning Conference 2016

  1. Heli May 16, 2016 / 4:40 pm

    Hi Jenny, I’ve followed #NLC2016 and noticed how much appreciation you and Jutta have received there. You felt lonely in 2010 when the other participants were not enough interested in CCK08 – but this time they were. (this is how i remember, don’t know if you agree with my memories).

    I want to continue my pondering around the Footprints of Emergence. For me, the critical point is how the cluster elements are intended to reflect the learning experience. You say that they are based on our own experience of learning in open and prescribed environments and of our knowledge of research. I am very curious to understand this process more, but I guess it is not possible to declare.
    You say
    “We offered them as a palette of characteristics which could be selected from or which could be replaced by other characteristics. Our experience has shown that these 25 factors provide a rich picture of the learning experience”. How could we make the students more aware of the choice they do? If the first task were to pull the factors from a palette? Is it very difficult to implement?

    After reading Jutta’s blog posts about her travels I got some new ideas to adapt the Footprints model to my whole life span. I am 70 yrs now and in Finnish I’ve written stories about our life spans with my old schoolmates. We met when we were 18+50 yrs old and four of us have been blogging.

    Another idea is to spread information about the Footprint of Emergence model in Finland and try if some institution/ teacher/researcher becomes interested in it. That would be so nice. You have done so great work until now that it is worth continuing.
    I hope you can follow my thoughts and I trust you can

  2. jennymackness May 16, 2016 / 5:39 pm

    Hi Heli – thanks for these great questions.

    You are right about my 2010 Networked Learning Conference. Although I enjoyed going to Denmark, I found it difficult to network. This time I knew lots of people, or knew the names of lots of people, and I think it was a smaller conference. I found it very friendly – but maybe I was just more confident this time 🙂

    The question you ask about where the factors have come from is one we have been asked before. I blogged about it here – https://jennymackness.wordpress.com/2014/09/04/theoretical-influences-on-the-characteristics-of-open-learning-environments/ If this post doesn’t answer your question come back to me about it. I know that you have a long history in teaching and learning, so I would be interested to know if you would have chosen different factors.

    Re thinking about the factors as a palette to be selected from, the questions you ask are interesting. When I started using the footprints I would sometimes leave a factor out if I felt I didn’t have the evidence to place a point on the drawing template. But I think users need to really understand the factors to be able to use them like a palette. And interestingly when Roy and I have run workshops, and we have offered participants a choice of factors, without exception everyone has always used all the factors. Our experience is that using them all does lead to a very rich reflection on the learning experience. By offering them as a palette, we wanted to make it clear that we are not claiming the factors to be a definitive list of characteristics of open learning environments. There will be different perspectives.

    I’ll be really interested to hear how you and your friends get on using the footprints. As you noted, Jutta has used them extensively with her work colleagues, her students and personally, so she would be a good person to talk to as well. And do let us know if you get any institution, teacher, researcher interest. If there’s anything we can do to help, just ask 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s