Automating teaching and assessment

George Veletsianos gave an interesting and thought provoking talk to the University of Edinburgh yesterday. This was live streamed and hopefully a recording will soon be posted here.  A good set of rough notes has been posted by Peter Evans on Twitter

Peter Evans@eksploratore

My live and rough notes on #edindice seminar from@veletsianos on #moocs, automation & artificial intelligence at…

As he points out, there were three main topics covered by George’s talk:

  • MOOCs as sociocultural phenomenon;
  • automation of teaching and
  • pedagogical agents and the automation of teaching.

George’s involvement with MOOCs started in 2011 when he gave a presentation to the Change11 MOOC, which I blogged about at the time .

I found myself wondering during his talk to the University of Edinburgh, whether we would be discussing automating teaching, if he had started his MOOC involvement in 2008, as this presentation seemed to come from a background of xMOOC interest and involvement. Those first cMOOCs, with their totally different approach to pedagogy, were not mentioned.

I feel uncomfortable with the idea of automating teaching and having robotic pedagogical agents to interact with learners. The thinking is that this would be more efficient, particularly when teachers are working with large numbers as in MOOCs, and would ‘free up’ teachers’ time so that they can focus on more important aspects of their work. I can see that automating some of the administration processes associated with teaching would be welcome, but I am having difficulty seeing what could be more important, as a teacher, than interacting with students.

George pointed out that many of us already use a number of automating services, such as Google Scholar alerts, RSS feeds, IFTTT and so on, so why not extend this to automating teaching, or teaching assistants, through the use of pedagogical agents such as avatars.

What was interesting is that the audience for this talk seemed very taken with the idea of pedagogical agents, what gender they should be, what appearance they should have, what culture they should represent etc. For me the more interesting question is what do we stand to lose and/or gain by going down this route of replacing teachers with machines.

For some of my colleagues, Karen Guldberg and her team of researchers at Birmingham University, robots have become central to their research on autism and their work with children on the autism spectrum. These children respond in previously unimaginable ways to robots. For some there will be gains from interacting with robots.

But I was reminded, during George’s talk, of Sherry Turkle’s concerns about what we stand to lose by relying on robots for interaction.

And coincidentally I was very recently pointed, by Matthias Melcher, to this fascinating article – Biology’s Shameful Refusal to Disown the Machine-Organism – which whilst not about automating teaching through the use of avatars/robots, does consider the relationship between machines and living things from a different perspective and concludes:

The processes of life are narratives. The functional ideas manifested in the organism belong to the intrinsic inwardness of its life, and are not imposed from without by the mind of an engineer. (Stephen L. Talbott, 2014).

Finally, George Veletsianos’ talk was timely as I am currently discussing with Roy Williams, not how teaching and assessment should be automated, but rather whether and if so how, it can be put in the hands of learners.

This topic will be the focus of a presentation we will give to the University of Applied Sciences, ZML – Innovative Learning Scenarios, FH JOANNEUM in Graz, Austria on September 17th 2014.


8 thoughts on “Automating teaching and assessment

  1. francesbell June 19, 2014 / 1:13 pm

    I was sorry to miss this so thanks for the write-up. My Masters dissertation was on Machine Learning and I can look back on that as a pivotal event in the development of my thinking in the last 25 years – away from a focus on machine learning.
    My small contribution here is to share my response to the automation of teaching/ learning and the transfer of assessment from teachers to learners. In both cases, I think it’s about the effective re-distribution of work between human and non-human agents.
    So as technology emerges, we can devolve to it the things it can do well in teaching, eg monitor likely sources via RSS and as humans do the things we can do well – respond individually to learners, etc.
    Similarly, yours and Roy’s concerns could be reframed as ‘sensible sharing’ of assessment between learners and teachers.
    Unfortunately, the machine metaphor can be a Trojan Horse for eliminating valid human contributions in the name of efficiency, increased profit and often impoverishes discussion of future possibilities.

  2. francesbell June 19, 2014 / 1:15 pm

    Oh sorry – just read one crazy thing I said there. – correction
    In both cases, I think it’s about the effective re-distribution of work between agents – in one case human and non-human agents, in the other teachers and learners.

  3. George Veletsianos June 19, 2014 / 1:56 pm

    Thank you for your thoughtful comments, Jenny! I am typing this response on my mobile device, so please excuse any typos below.

    Perhaps it wasn’t clear in the presentation and q&a, but the conversation/talk did not arise out of a desire or need to replace teachers or to introduce efficiencies. Quite the opposite, actually: there’s no question in my mind that the efficiency narrative is harmful to education. As I’ve noted in my presentation the field of learning/instructional design is enamored with the notion of efficiency and effectiveness and it needs to move beyond that, it needs to aim higher, it needs to try harder to design empowering, socially just, and transformative learning experiences (more details on this can be found in this paper:

    You are correct however in pointing out that there is a fascination with artificial intelligence and machine learning and the efficiencies ideas associated with those fields could contribute to teaching. I find that these ideas are mostly associated with those who view education as content delivery and support standardized assessments.

    Your question about xMOOCs vs cMOOCs is really interesting. As a side note let me point out that we were discussing pedagogical agents and appearance/intelligence/language because my early research was focusing on sociocultural aspects of agent-learner interactions. However, if you consider “bots” as automated tools that could support teaching and learning, it’s possible to find value in them in the context of cMOOCs as well. Consider gRSShopper, for example. I am inclined to see gRSShopper as a tool that automates various processes with the intent to support the development of communities and networks. I see that as being worthwhile in pedagogical terms, allowing us to better support progressive pedagogies.

    Thank you again for the food for thought, Jenny – all these ideas need to be critiqued and questioned and I really appreciate you taking the time to contribute to this conversation!

  4. jennymackness June 19, 2014 / 4:28 pm

    Frances – thanks for your helpful comments. I think you will be interested, as I am in George’s response, as I think they help to fill in the gaps in my post and clarify the key messages from his presentation.

  5. jennymackness June 19, 2014 / 4:45 pm

    Thank you George for taking the time to comment and for the link to your paper. I’m glad to have your comments as I recognised at the time that the way I interpreted your presentation (which I can see now wasn’t exactly what you intended) didn’t seem to fit with the other work you have done on eliciting students reflections on their learning experiences.

    Despite this I have found the discussion about automating teaching and learning and the opportunity to think about this further useful, given that I am currently discussing the problems of assessing learners in open learning environments with Roy Williams.

    I think the whole area of how technologies can better support progressive pedagogies is one that educators will need to engage with, if only to clarify in their own minds how far down the automation route they are willing to go in their teaching.

    Thanks again


  6. scottx5 June 19, 2014 / 6:18 pm

    Thanks for the posting Jenny, I’ll definitely read Stephen Talbott’s paper. If we could build machines that were able to tell us about ourselves, the way our children often do, we might get something useful out of mechanizing ourselves.

    The idea of improving our performance with machines I would think comes from a notion that can be “fixed” by building artificial efficiencies that act in our place. To accept this first we’d need to delude ourselves into thinking them somehow rightfully smarter than us and then allow them status as being our superiors. That’s just the reinvention of systems of self-oppression and we’ve that for a long time.

  7. francesbell June 19, 2014 / 6:56 pm

    Thanks Jenny and George – very enlightening. I much prefer the tool approach to some sort of holy grail of automation. And I think that the field of learning design needs to (as I am sure you do George) include learners as designers of their own learning, including choice and adaptation of tools.

  8. jennymackness June 19, 2014 / 7:40 pm

    Hi Scott – interesting comments as ever 🙂

    > If we could build machines that were able to tell us about ourselves, the way our children often do, we might get something useful out of mechanizing ourselves.

    The problem here is, as you say, that we would have to believe that the machine was right and even then we probably wouldn’t accept it 🙂 It takes a lot for me to accept that someone else might know me better than I do – and only very close trusted and valued friends can get away with it 🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s