Beyond Assessment – Recognizing Achievement in a Networked World

 

This was the third in a series of 3 talks that Stephen Downes gave in London this week.

Jul 11, 2014
Keynote presentation delivered to 12th ePortfolio, Open Badges and Identity Conference , University of Greenwich, Greenwich, UK.

ePortfolios and Open Badges are only the first wave in what will emerge as a wider network-based form of assessment that makes tests and reviews unnecessary. In this talk I discuss work being done in network-based automated competency development and recognition, the challenges it presents to traditional institutions, and the opportunities created for genuinely autonomous open learning.

For recordings of all three talks see OLDaily

Beyond Assessment – Recognizing Achievement in a Networked World
Jul 11, 2014. 12th ePortfolio, Open Badges and Identity Conference , University of Greenwich, Greenwich, UK (Keynote).

Beyond Institutions – Personal Learning in a Networked World
Jul 09, 2014. Network EDFE Seminar Series, London School of Economics (Keynote).

Beyond Free – Open Learning in a Networked World
Jul 08, 2014. 12th Annual Academic Practise & Technology Conference, University of Greenwich, Greenwich, UK (Keynote).

This was perhaps the most forward thinking and challenging of the three talks. I wasn’t at the talk, but listened to the recording. What follows is my interpretation of what Stephen had to say, but it was a long talk and I would expect others to take different things from it and interpret the ideas presented differently.

Educators have been wrestling with the issue of assessment, how to do it well, how to make it authentic, fair and meaningful, how to engage learners in the process and so on for many, many years.

Assessment has become even more of a concern since the advent of MOOCs and MOOC are symptomatic of the changes that are happening in learning. How do you assess thousands of learners in a MOOC?  The answer is that you don’t – or not in the way that we are all accustomed to – which is testing and measurement to award credentials such as degrees and other qualifications. This has resulted in many institutions experimenting with offering a host of alternative credentials in the form of open badges and certificates.

Stephen’s vision is that in the future assessment will be based not on what you ‘know’ but on what you ‘do’ – what you do on the public internet. The technology now exists to map a more precise assessment of people through their online interactions. Whilst this raises concerns around issues of privacy and ethical use of data, it also means that people will be more in control of their own assessment. In the future we will have our own personal servers and will personally manage our multiple identities through public and private social networks. Prospective employers seeking a match for the jobs they want filled can then view the details of these identities. There is some evidence that learners are already managing their own online spaces. See for example Jim Groom’s work on A Domain of One’s Own.

Why might new approaches to assessment such as this be necessary? Here are some of the thoughts that Stephen shared with us.

It is harder and harder these days to get a job, despite the fact that employers have job vacancies.  There is a skills gap.  The unemployed don’t have the skills that employers need. We might think that the solution would be to educate people in the needed skills and then employers could hire them, but employers don’t seem to know what skills are needed and although learning skills inventories help people to recognise what they don’t know, these inventories don’t help them to get to what they do know.

Education is crucial for personal and skills development and more education leads to happier people and a more developed society. The problem is that we confuse the outcomes of education with the process of education. We think that we can determine/control learning outcomes and what people learn. See Slide 14

But useful outcomes are undefinable (e.g. understand that …..) and we need an understanding of understanding. Definable outcomes such as ‘recite’ and ‘display’ are simpler but behaviourist (Slide 18).   There is more to knowing than a set of facts that you need to pass the test.  Knowing something is to recognise it, in the sense that you can’t unknow it.  Stephen used ‘Where’s Wally’ as an example of this:

Knowing, according to Stephen, is a physical state – it is the organisation of connections in our brain. Our brain is a pattern recogniser. Knowing is about ‘doing’ rather that some mental state.

My understanding of what Stephen is saying is that if we believe that knowing is about pattern recognition, then achievement will be recognized in how good learners are at pattern recognition as evidenced by what they ‘do’ in their online interactions. ‘Assessors’ will also need to be good at pattern recognition.

Learners are increasingly more sensitive to the patterns they see in the huge amount of data that they interact with on the internet, and machines are getting closer to being able to grade assignments through pattern recognition.  As they interact online learners leave digital traces. Big data is being used to analyse these internet interactions.  This can be used for assessment purposes. But this has, of course, raised concerns about the ethics of big data analysis and the concern for privacy is spreading – as we have recently seen with respect to Facebook’s use of our data. (Slide 55)

A move to personally managed social networks rather than centrally managed social networks will enable learners to control what they want prospective employers to know about them and human networks will act as quality filters.

Stephen’s final word was that assessment of the future will redefine ‘body of work’.

All these are very interesting ideas. I do wonder though whether it’s a massive assumption that all learners will be able to manage their own online identities such that they become employable. What are the skills needed for this? How will people get these skills? Will this be a more equitable process than currently exists, or will it lead to another set of hierarchies and marginalisation of a different group.

Lots to think about – but I really like the move to putting assessment more in the control of learners.

26-09-2014 Postscript

See also this post by Stephen Downes – http://halfanhour.blogspot.co.uk/2014/09/beyond-assessment-recognizing.html – which provides all the details of this talk

Beyond Institutions – Personal Learning in a Networked World

This was Stephen Downes’ second talk in a series of 3, which he is giving in London this week. This is how he introduced it on his blog Stephen’s Web 

In this presentation I look at the needs and demands of people seeking learning with the models and designs offered by traditional institutions, and in the spirit of reclaiming learning describe a new network-based system of education with the learner managing his or her education.

Although I have only listened to the recording of this talk, I found it more interesting than the first talk, which I listened to live, having been a delegate at the conference, although there was plenty of interest in that one too. What I like about Stephen’s talks is that he doesn’t pull any punches. He always challenges my thinking.

The thrust of this talk, from my perception, is, as the title suggests, that learning is no longer in the control of institutions, but increasingly personal and in the control of learners as they occupy a networked world. There is a distinction between personal learning and personalized learning. Institutions don’t understand personal learning because personal learning has to be in the control of the learner. It is made to order. Learning is built not from a kit but from scratch. Institutions think they are catering for personal learning, but in fact are offering personalized learning – which is ‘off the shelf’ learning; one package with a bunch of options.

There is evidence that today’s students are demanding change and want more control. Learning is no longer about remembering. The content, nature and means of learning are changing on a daily basis. Learning today is more about play and socializing. Lecturing is also changing. Lecturing today is not so much about content as creating the potential for dialogue.

A particularly challenging point that Stephen made was ‘Do away with models’ – learning models and design models.  The right model is no model. New versions of old models don’t produce results. It is obvious that people learn differently, have different objectives, priorities, goals and times when they want to learn, but if you use a learning model you are attempting to predefine the outcome, whereas learning should be about discovery and exploration. I would also say from the work I have done with Roy Williams, that we need to recognize that  learning will often be unpredictable and emergent. (See Emergent learning and learning ecologies in Web 2.0)

Autonomy rather than control is the essential in education. Autonomy does not mean no structure, it means choice of structure. Personal learning is based on self-organization and self-organizing networks. Learners need to reclaim management and organization of learning. The way forward will be for students/learners to have their own personal web server and run their own web services from their own home networks.  The University will be a box in your living room. Learning should be cooperative and networked. It is not content that is important, but the making of connections. Learners need networking skills.

What do we need from institutions?

We do not need

• more models, more designs
• more learning theories
• more standards, measurement and centralization
• more control
• more of making the same mistakes

We do need mechanisms to support people in learning and bettering their lives. Institutions need to think in terms of serving many different people in many different ways and supporting personal learning, rather than attempting to control and personalize learning.

*************************************************************************************************

And here is an interesting blog post about this talk by Sonja Grussendorf – Beyond institutions: Stephen Downes at NetworkEDGE

See also Arun Karnad’s post:

[View the story “LSE NetworkEdge: Stephen Downes” on Storify]

Connected Learning in an Open World

At the beginning of this week I was in Greenwich, London for the first time in my life. On Monday I travelled up the Thames from Embankment to Greenwich Pier by Clipper (another first) and stood on the decks of the Cutty Sark.

On Tuesday I spent the day at the University of Greenwich’s APT2014 Conference, the reason for the trip.

On Wednesday I stood on the Meridian Line at the Royal Observatory.

A key question asked in the main exhibition room of Flamsteed House  at the Observatory is ‘Where am I? This related to how you can work out your exact location on the open seas, by knowing how to fix your latitude and longitude positions. But ‘Where am I?’ seems such an important and relevant question for an educator and although I didn’t visit Flamsteed House until the day after the Greenwich conference, I found myself constantly wondering where I am in relation to the discussions that were held during the conference.

One of the main reasons for attending the conference was to hear Stephen Downes speak. Where am I in my understanding of what he had to say and the implications of what he had to say? Here is the link to a recording of his full talk, Beyond Free – Open Learning in a Networked World  and this is the Abstract for the presentation:

This was the first in a series of 3 talks that Stephen is giving in London this week. He started his second talk, Beyond Institutions: Personal Learning in a Networked World – given to the NetworkEDGE conference at the London School of Economics on Wed 10th July – with the words: If you feel unfulfilled at the end of this talk, it’s because it doesn’t really have a beginning and doesn’t really have an end, i.e. it’s the middle talk in a series of three. I have only listened to the recording of this second talk.

I did feel somewhat unfulfilled after the first talk. It’s not that I didn’t enjoy the conference – I thoroughly enjoyed it, attended some interesting sessions and talked to some great people, but at the end of the day I felt that a lot of questions had been raised but not many answers had been found. These questions were around what we mean by ‘open’, what we mean by ‘connected learning’ and what do universities understand about open, connected learning – not only what do they understand, but what are they doing about it, what are they becoming as a result of open learning in a networked world – and are they becoming what we would hope they become? As Stephen said, ‘Institutions are what we make them’.

This thinking about unanswered questions made me wonder whether the idea of flipped classrooms, which was mentioned in the opening talk by the Vice Chancellor, should be applied to conferences. Should we engage with the ideas to be presented by the keynote speaker before the conference, and present a discussion paper/workshop as a result of that – so that the key questions can be discussed.

The points I took from Stephen’s talk were that

‘Open’ means open in all senses, particularly in the sense of open sharing of thought processes, and should be the default position in Universities. Free and open access is not enough.

But Universities are resistant to openness in the sense of open sharing, and content providers do not want people to have free and open access. The promise of open resources has not materialized.

Open access makes a massive economic difference to users, but cost IS the problem for universities because universities see online learning in terms of money making.

The issue is not finding innovative ways of teaching, but innovative ways of learning.

The bulk of MOOCs are created in the image of traditional courses, but this was never the intention of the original cMOOCs.

Change in Universities is slow – too slow.

None of these points came as a surprise. None of them is unfamiliar, but challenging Universities to become more ‘open’ can be a risky business for employees and those that do can land themselves in trouble, as Stephen pointed out in his presentation. (See slide 29 for an example).

In general people seem to be more aware of the risks than the benefits. A new lecturer at the conference said that ‘openness’ is a risk for someone like her who is new in the job and trying to establish a reputation. Sheila McNeill, who was a panel member at the end of the day, urged this lecturer to be brave and just go for it. I wonder whether being strategic about openness is more important than being brave. Sharing openly doesn’t mean that you have to ‘bare your soul’ – there are other ways of sharing. A more impersonal and less risky approach is reporting. If open sharing doesn’t come easily then share what you have discovered to be useful, rather than your own work or personal thoughts. As Stephen said in his second talk to the London School of Economics, every learner is different and reacts to each learning scenario differently.

The Deputy Pro-Vice Chancellor, also on the panel, seemed to recognize the difficulties when she said that open sharing in the form of lecturers recording their lectures and openly sharing them, is a risk to the University’s reputation – but she also acknowledged that a major issue for an institution is the need for cultural change. As she put it – universities will have to wait for some staff to shift or die before this culture change can be achieved.

Stephen asked for a show of hands for who was tweeting the conference proceedings and a show of hands for who had recorded their conference presentation.  Some were tweeting, but only one person had recorded their presentation. The person sitting next to me during the final panel session was inspired and enthusiastic about what she had heard during the day, but said that she had never taught online and had never taken an online course. It was all new for her.

For me, the concerns raised about openness should not be brushed aside. Questions of whether an academic’s or an institution’s reputation can be damaged by openness need to be discussed. The benefits or otherwise of openness need to be articulated. For me, it is not about whether you tweet at a conference or record your presentation and upload your Slideshare; all these can help to model a spirit of openness, but it’s more about trying to understand why openness is necessary and how we can all be supported in understanding and doing this. Ultimately, isn’t it about personal values and educational philosophies?

So I came away from Greenwich feeling that many questions had been raised, but that they were left hanging. I would have been interested in more discussion about whether there is agreement about the changes that Stephen suggested Universities need to make and if so how they will make these changes. But I have now listened to Stephen’s second talk to the London School of Economics, which helped me to understand the context of the first talk. Inge de Waard has blogged about it here: Fabulous ideas: economics, innovation, #education  and I hope to return with another blog post.

A big thank you to Simon Walker, Gillian Keyms and colleagues for organizing a thought-provoking event, and to all at Greenwich, particularly the students, who were so helpful, friendly and welcoming.