Finding different voices

In an OLDaily post this week, Stephen Downes has encouraged the authors of a recent publication ‘Open at the Margins’ to redouble their efforts to find the voices not being heard. I interpret this as a call by Stephen for greater diversity in the people who are being recognised as speaking for open education. I think all groups could and should aspire to this.

I’m not sure how I would go about finding voices not being heard. Perhaps it’s a question of being aware of the direction of our attention, and that what we choose to attend to determines not only what we see, but also what we don’t see. Rather than trying to find voices not being heard it might be easier to find different voices; this might require a cross or multi-disciplinary approach to seek different perspectives, which may or may not include minority voices.

On reflection I realise that I have spent most of this year seeking and listening to different voices. A positive outcome of the Covid-19 pandemic for me (and I realise how privileged I am to be in this position) has been to discover a wide range of different people who have made their work and thoughts available online (either freely or at minimum cost). Many organisations have supported this opening up of access to different voices. These are some that I have found interesting and enjoyed over the past few months, offering me new avenues for thought and/or exploration.

Channel McGilchrist

This is a new platform, which I joined in June. Membership requires paying a fee, but there is also access to some materials for non-paying members. Since Iain McGilchrist is a polymath, which is very evident from his book, The Master and His Emissary. The Divided Brain and the Making of the Western World, this channel has attracted a wide diversity of thinkers from different disciplines, which has generated very varied discussion. Lots of different voices here.

The London School of Economics and Political Science

• 04-06-20 Do Morals Matter: presidents and foreign policy from FDR to Trump
• 22-06-20 Brexit and the Post-Covid-19 options for the economy

Being Human Festival of Humanities

• 12-05-20 Being Human in Conversation. On Politics and Plagues
• 24-06-20 Being Human in Conversation with David Lammy. My Education

How to Academy

• 30-06-20 Peter Singer in Conversation with Matthew Stadlen
• 15-07-20 Iain McGilchrist. The Divided Brain and the Search for Meaning
• 06-08-20 An evening with Slavoy Zizek. In conversation with Rowland Smith

I’m looking forward to:

• 02-09-20 The Anarchy – The Relentless Rise of the East India Company with William Dalrymple
• 13-09-20 John Cleese’s Guide to Creativity

The Weekend University

• 26-07-20 A Day on Psychology and Culture

Philosophy of Education Reading Network on Twitter @PhilofEd

• 18-08-20 Discussion of Iris Murdoch’s The Sovereignty of Good

I’m looking forward to next month’s meeting of the network:

• 15-09-20 Discussion of Gert Biesta’s The Beautiful Risk of Education

I have only just discovered (by chance, and most of these events/groups were discovered by chance) the Philosophy of Education Reading Network. Last night’s discussion was the first for this newly established group and was open to anyone. A few people in the zoom call clearly knew each other, but many, like me, were new to the network, and some were new to the philosophy of education. 17 people attended, a good mix of men and women, and of different ages, and the atmosphere was very inclusive without putting pressure on people to contribute. All contributions were welcomed and considered.

Returning to the point made at the beginning of this post about the need to encourage different voices to contribute to a group or collaborative endeavour, and in the light of my experience last night of a newly formed group, I wonder at what point does it become difficult for a group to recognise that some voices are not heard, or that critical perspectives are being narrowed and limited through group think and a lack of diversity? Is it inevitable that this happens in groups that share and enjoy a common interest?

This has reminded me of Stephen Downes work in 2007 on the difference between groups and networks, and his post Groups Vs Networks:The Class Struggle Continues, which I think speaks for itself and speaks to this topic of finding different voices.

Source of image:   https://www.flickr.com/photos/stephen_downes/252157734/

Challenges of 21st century education: Past and present reforms

The last two weeks of Exeter University’s FutureLearn open course: Testing Times in the Classroom: Challenges of 21^st Century Education were devoted to key changes that have taken place in the field of education over the last 20 or so years. These changes were discussed mostly in the context of the UK and Europe, but participants were encouraged to add their knowledge and perspectives from their own cultures and countries.

The 20^th century in the UK saw the creation of universal education, through the growth of state funded education and the raising of the school leaving age from 12 to 16. Following the Education Act in 1944 state-funded secondary education was organised into three type of schools; grammar, technical and secondary modern. Allocation to these schools depended on children’s performance in the 11+ exam. Between 1944 and 1965 this tripartite system came to be increasingly criticised for being divisive and leading to educational inequalities. In response to these concerns in 1965 the Labour Government introduced comprehensive schools for secondary aged children, with the aim of providing an entitlement curriculum for all, without selection through financial considerations or attainment. I was at University at this time and remember having long discussions with people of my parents’ generation who were appalled that good grammar schools were being replaced by comprehensive schools. I myself, in my youth, was ‘fired up’ by the thought that comprehensive schools would ensure that any and every child would have an equal opportunity for a good education. Ultimately comprehensive schools were also discredited with comparisons being made between comprehensive and independent schools.

In the FutureLearn course this was illustrated through two YouTube videos – one of Radley College – an independent boys school, and the other of Faraday High School, a state comprehensive.

Radley College

Faraday High School

Personally, I did not think this was a fair comparison to make. My first teaching experience was in an inner city comprehensive and it was nothing like Faraday High School. Faraday High School would be a ‘bad’ school in any circumstances. Evidence from the video suggests that it had incompetent teachers and poor leadership. Nevertheless comprehensives like Faraday High School did exist such that the system failed and led to increasing concern with educational inequalities related to social class and ethnicity, which still exists today, together with additional equality and diversity concerns, such as gender and disability.

Over the past 20 to 30 years, much educational reform in the UK has focussed on a response to these equality and diversity concerns, raising research questions such as:

• Do schools favour girls?
• Do schools make the rich richer?
• Does social class still matter?
• Is the school system failing black children?

Whilst there are many research articles that deal with these questions separately, there is now increasing recognition of the importance of intersectionality, i.e. that the wide range of different inequalities intersect. For example, a student’s educational experience will not be affected by gender alone, but also by social class, ethnicity, sexuality, disability and so on.

Another question that was asked in these last two weeks of the course was:

• • Is the purpose of school reform to improve international economic competitiveness?

Surprisingly, to me, when course participants were asked this question 54% answered ‘Yes’. I myself had no hesitation in answering ‘No’. For me the first concern of education should always be the learners/students. We should ask ‘how can the system support each individual in realising his/her full potential?’ If this could be achieved then perhaps international economic competitiveness would follow or, better still, lead to educated thinking adults who would question whether international economic competitiveness should be the purpose of education. Some in the course considered my view unrealistic and utopian, since they argued that education is simply a means to an end.

So it seems that my view is not the majority view and certainly the UK’s approach to educational reform in recent years has been based on a belief in the importance of education for international economic competitiveness. Thus some recent key reforms, which are easy to recognise, have focussed on:

• Accountability and performance management. This has led to increased testing and school inspections, performance based pay and funding, and increasing focus on management. This system rewards success and punishes failure.
• Competition and markets – league tables, choice for parents, and the marginalisation of collaboration and collective effort. This approach to reform can already be seen to be leading to hierarchies and differences between socially advantaged and disadvantaged students. For example, some middle class parents are prepared to move house to ensure that they are in the catchment area for schools high in the league tables.
• Increased control over schools and universities – inspections, audits, reviews and evaluations to measure educational performance, all supported by increased capacity to collect and store data. This necessarily neglects aspects of education that cannot be measured.

Most of these educational reforms are being adopted worldwide, and led to Pasi Salhlberg coining the term Global Education Reform Movement (GERM). Of this acronym-as-analogy, Fuller and Stevenson (2018) write that it:

worked perfectly to describe a phenomenon that Sahlberg identified as both spreading and destructive, behaving “like an epidemic that spreads and infects education systems through a virus” (Sahlberg, 2012, no page).

and that:

Sahlberg has identified the principal features of the GERM as increased standardisation, a narrowing of the curriculum to focus on core subjects/knowledge, the growth of high stakes accountability and the use of corporate management practices as the key features of the new orthodoxy.

In writing about how Finland views educational reform differently, Pasi Sahlberg questions whether this global education reform movement (GERM) is counter-productive.

In the UK, 30 years of these reforms has led to layer upon layer of change and a degree of complexity that could conceivably take at least another 30 years to unravel, even assuming that the ‘powers that be’ think this necessary. We now have a UK education system which has shifted to decentralisation with over 70 different types of schools, whilst at the same time increasing centralisation through the introduction of the national curriculum and increased testing. Derek Gillard (2018) in the conclusion to his report writes:

This history has focused on the long struggle to create for England’s children an education system which values them all. It has, in many ways, been a sad story.

But he ends on a more optimistic note, writing:

Meanwhile, across the country, tens of thousands of teachers still care deeply about the well-being and prospects of their pupils, and go to work every morning determined, despite the often unhelpful interventions of politicians, to provide them with the best and most humane education they can.

References

Cohen, M. (2004) Knowledge and the gendered curriculum: the problematisation of girls’ achievement – http://www.historyandpolicy.org/policy-papers/papers/knowledge-and-the-gendered-curriculum-the-problematisation-of-girls-achieve

Courtney S. (2015) Mapping school types in England. Oxford Review of Education. 41(6):799-818. https://www.tandfonline.com/doi/pdf/10.1080/03054985.2015.1121141?needAccess=true

Department for Children, Schools and Families (DCSF), corp creator. (2009) Gender and education : mythbusters : addressing gender and achievement : myths and realities https://dera.ioe.ac.uk/9095/

Equality and Human Rights Commission – https://www.equalityhumanrights.com/en Is Britain Fairer? (2018) https://www.equalityhumanrights.com/en/publication-download/britain-fairer-2018

Fuller, K. and Stevenson, H. (2019) Global education reform: understanding the movement, Educational Review, 71:1, 1-4, DOI: 10.1080/00131911.2019.1532718 https://www.tandfonline.com/doi/full/10.1080/00131911.2019.1532718

Gillard, D. (2018) Education in England: the history of our schools http://www.educationengland.org.uk/history/

Gillborn, D. and Mirza, H. S. (2000) Educational inequality: mapping race, class and gender – A synthesis of research evidence. Office for Standards in Education. https://www.researchgate.net/publication/319490152_EDUCATIONAL_INEQUALITY_MAPPING_RACE_CLASS_AND_GENDER_-_A_synthesis_of_research_evidence

Hall D. Grimaldi E, Gunter, H, Moller, J, Serpieri, R and Skedsmo G. (2016) Educational Reform and Modernisation in Europe: The Role of National Contexts in Mediating the New Public Management. European Educational Research Journal. 14(16):487-507. https://journals.sagepub.com/doi/pdf/10.1177/1474904115615357

Hall D. and Gunter H. (2016) England. The Liberal State: Permanent Instability in the European Educational NPM Laboratory. In: Gunter H, Grimaldi, E, Hall D, and Serpieri, R, editors. (2016) New Public Management and the Reform of Education: European Lessons for Policy and Practice. London:Routledge. https://warwick.ac.uk/fac/soc/ces/postgrads/teachfirst/1/november21/hall_and_gunter_-_the_liberal_state.pdf

Sahlberg, P. (2012). How GERM is infecting schools around the world? Retrieved from https://pasisahlberg.com/text-test/

Schleicher, A. (2018) Equity in Education. Breaking down barriers to social mobility http://www.oecd.org/education/equity-in-education-9789264073234-en.htm Organisation for Economic Co-operation and Development (OECD) website

The Fawcett Society https://www.fawcettsociety.org.uk/centenary-resources

The Gender and Education Association – http://www.genderandeducation.com/resources-2/

Ward, H. (2018) More male role models are needed in early years, say heads. TES https://www.tes.com/news/more-male-role-models-are-needed-early-years-say-heads

Weale, S. (2017) Sexual harassment ‘rife’ in schools but largely unreported, study says. The Guardian – https://www.theguardian.com/world/2017/dec/12/sexual-harassment-rife-in-schools-but-largely-unreported-study-says

Women in STEM: how gender inequality could damage Scotland’s economy – https://theconversation.com/women-in-stem-how-gender-inequality-could-damage-scotlands-economy-107627

The power of silent learners

 

This week I attended a webinar which focussed on silent learners. Thank you to Martina Emke for drawing this webinar to my attention. The title immediately resonated as recently I have found myself increasingly troubled by the constant noise of the online environment. This has been a growing awareness and I recognise it as a problem since I rely on the open web and my online network for the work I do as an independent education consultant and researcher. This has made me wonder to what extent open learning environments exclude those who find ‘noisy’ environments difficult.

The webinar was run as part of a Nordic Nordplus project, Silent learners – Is lurking working?,  investigating how silent or peripheral learners can be better understood even in courses where participation is the focus. It was organised by NVL Distans  (Nordic Network for Adult Education), EDEN (European Distance and E-learning Network) and the Swedish Network for IT in Higher Education (ITHU). (Source of information)

The webinar was very ably facilitated by Francisca Frenks, Alastair Creelman and Jan Willem Kemper  – more than ‘ably’ in fact. This was probably one of the best webinars I have attended (and over the years I have attended very many). It was extremely well structured and very interactive. I would like to spend a bit of time at the beginning of this post recording how it was run for future reference and as an example of good practice.  There were 72 people in the webinar – quite a big number to manage. Most participants were from Scandinavia, but some were from further afield.

These are the points that I think made it work so well.

1. There were very good joining instructions and email reminders of the event.
2. We were invited to log in half an hour early to check our connection. Microphones and videos were not enabled until the end of the webinar, but the opening screen invited us to interact by marking our location on the map and using the chat. Clear instructions were written on the screen and reiterated by the presenters.

The presenters were in the room with their microphones open and welcomed us both verbally and in the chat. Changing the layout of the screen by dragging and dropping boxes was either not enabled or is not a feature of Adobe Connect. If it had been I would have moved the chat to my second monitor – but it was not a problem. What I particularly liked about the whole webinar was that the presenters changed the layout of the screen according to the activities they had planned for us.

3. After an introduction to the presenting team and the background to this webinar, interaction started almost immediately and we were asked ‘What brings you here? – a nice open, opening question. The chat box was enlarged to help us to follow it more easily.

4. The focus of the webinar was then explained. Before the webinar we had been asked to watch Susan Cain’s well known video on introverts.

5. We were told that when discussing silent learners in this webinar the focus would be on learners who are silent by nature rather than silent for other reasons.

6. In particular the focus was on how silent leaners work in groups. Three questions were posed:

• How do we recognise learners who are silent by nature?
• What are their learning needs?
• How can we empower the silent learner in the group context?

The focus for discussion was therefore made very clear.

7. We were then asked the questions listed above and asked to respond to them in the chat, but before responding to each question Francisca interviewed Jan Willem, a self-professed ‘silent by nature learner’ about his experience in relation to the question.

8. What was particularly effective was that we were asked to identify ourselves as a silent learner, a noisy learner, or something in between, by typing in a particular colour.

Finally we were asked to respond to some statements made by silent learners about their experience of group work. These had been collected from interviews conducted by Taru Kekkonen. How would we encourage the silent learners who made these statements to participate in groups? These were the statements:

‘It is simply easier to study independently at my own pace’.

‘I have no need to express myself loud among others’.

‘I hate spontaneous discussions because I don’t know what appropriate is to say’.

‘It is easier to join an online discussion, because the structure is more clear’.

All the questions and statements in the webinar promoted fast and furious typing in the chat box and although I have watched the recording it would take a while to go through it all and determine more fully how silent learners are perceived or perceive themselves.

I thoroughly enjoyed this webinar. The time flew by. Not only was it extremely well run, but also thought provoking.

The main thought I have come away with is to question whether it helps silent learners to focus on them in this way. Jan Willem felt it does, because he feels that there is not enough recognition of what silent learners can offer. For me the danger is that in doing this we may reinforce the view that somehow silent learners are a problem and that we need to solve this problem by enabling them, empowering them, to become a bit noisier. Personally, I don’t think that learners can be empowered by others. They empower themselves, although they can be supported in doing this.

In addition, I don’t believe that silent learners can be identified as a separate group, although they can, as we saw in the webinar, self-identify. Learners will be more or less silent/noisy as individuals and according to the context. Thinking of silent versus noisy is probably not helpful. There will be a spectrum of learners and perhaps diversity is best served by keeping groups as heterogeneous as possible and ensuring that multiple perspectives are always considered.

This webinar attracted a cross-section of silent and noisy learners and many in between and we were invited to identify ourselves as silent, noisy or in between learners. This showed that we were a diverse group, but the discussion focused on silent learners as a separate group. It would be interesting to go back through the chat and analyse whether silent learners are considered to be ‘a problem to be solved’ or whether they are considered to be necessary to a diverse community of learners, even though they are in the minority. Evidently only 20-25% of the population identify themselves as silent learners.

Thank you to Francisca, Alastair and Jan Willem for organizing this excellent and thought-provoking open webinar.

Update 18-03-17

For a more detailed account of the webinar see Alastair’s follow up blog post which also provides links to further information – https://silentlearnersblog.wordpress.com/2017/03/15/webinar-the-power-of-silent-learners-in-a-group-2/

Update 20-03-17

See also Lotte Christoff’s post – A Voice of a ‘Silent Learner’ – https://studyelearning.wordpress.com/2017/03/19/a-voice-of-a-silent-learner/  The post is written in Swedish, but translates well enough to follow the text.

Diversity is hard

Source of image

dana boyd has written a post in which she discusses why America is self-segregating and she comes up with a few suggestions such as the role of social media in segregating people into filter bubbles and echo chambers. But a key point she makes is that diversity, which is ‘often touted as highly desirable’ is hard – ‘uncomfortable, emotionally exhausting and downright frustrating’. So instead of using the many online tools we now have at our disposal to become diversely connected, we use them instead to find like-minded people who, as Kirschner wrote in 2015, ‘discuss, confirm, validate and strengthen the group’s position’ (p.622). In doing this we reduce diversity.

(This tendency to try to reduce diversity is not only evident in online networks. It can also be seen in ‘The Big Sort’ and geographical clustering that I mentioned in my last post, i.e. people physically move geographical location to live near those more like themselves.)

More than ten years ago in 2005 in his ‘Introduction to Connective Knowledge’ (revised in 2007) Stephen Downes wrote of diversity as a key principle of ‘knowing’ networks. Downes sees the fostering of diversity as the means to

 ‘counterbalance the tendency toward a cascade phenomenon in the realm of public knowledge’.  

(Information cascades occur when external information obtained from previous participants in an event overrides one’s own private signal, irrespective of the correctness of the former over the latter’ (Wikipedia ). Cascade phenomena can sweep through densely connected networks very rapidly).

Downes writes

‘the excesses made possible by an unrestrained scale-free network need to be counterbalanced through either one of two mechanisms: either a reduction in the number of connections afforded by the very few, or an increase in the density of the local network for individual entities’.

According to Downes, the only way to avoid information cascades is to ensure multiple viewpoints and alternative perspectives from observers with different sets of prior experiences, world views and interpretations.

Related to this, a couple of years later Downes wrote of the different affordances of groups and networks – Groups vs. Networks: The Class Struggle Begins – saying that a group is about what members have in common, whereas ‘a network is like an ecosystem where there is no requirement that all the entities be the same.’ If we accept this it follows that a group tends towards homogeneity, but a network to heterogeneity (see also my post on the hazards of group work). Diversity is therefore essential to a healthy network.

But what is diversity?  Dictionaries, e.g. Cambridge dictionary, define diversity as being many different types of things or people, ideas or opinions, being included in something. I would add that in addition many different resources are needed to inform these ideas or opinions. In a paper that Carmen Tschofen and I published in 2012, Connectivism and Dimensions of Individual Experience, we also suggested that there is a need to recognise the importance of psychological diversity of online learners, the complexity of their human needs and connections, i.e. that diversity is not just an external manifestation of difference, but also internal to individuals. Each individual is unique. We argued that connectivity needs to be viewed not only in terms of the network but also in terms of individual characteristics and biases, further complicating an understanding of diversity.

But why is diversity ‘desirable’? dana boyd points to more diverse teams outperforming homogeneous teams and claims that diversity increases cognitive development. In my own field of research into learning in open online environments, this point of view is endorsed by the call for more interdisciplinary, multidisciplinary and cross global, international working (see for example Haywood, 2016 and Eynon et al., 2016).

However, Cilliers (2010) suggests that there are deeper reasons. These are related to viewing the world in which we live as a complex adaptive system. Complex systems are heterogeneous, asymmetrical and full of non-linear, unpredictable interactions, which means we cannot fully know or control them. Complex environments exhibit the following characteristics (and more!):

• Distributed knowledge
• Disequilibrium
• Adaptive
• Self-organisation
• Unpredictable
• Emergence
• Connectedness
• Diversity
• Openness
• Co-evolution
• Interaction
• Retrospective coherence

Cilliers tells us that diversity is a key characteristic of complex systems and is essential to the richness of the system, because it is difference not sameness that generates meaning.

An abundance of difference is not a convenience, it is a necessity. Complex systems cannot be what they are without it, and we cannot understand them without the making of profuse distinctions. Since the interactions in such systems are non-linear, their complexity cannot be reduced. The removal of relationships, i.e. the reduction of difference in the system, will distort our understanding of such systems. (Cilliers, 2010, p.58)

But this does not mean that ‘anything goes’. To get the most out of diversity and difference, complex systems require boundaries and constraints, negative, enabling constraints, ‘which determine what is not allowed to happen, rather than specifying what does have to happen’ (Williams, Karousou & Mackness, 2011, p.46). There needs to be an effective balance between openness and constraint, structure and agency.

And difference does not mean opposition. Meaningful relationships develop through difference (Cilliers, 2010), but achieving the right amount of difference to support this development, depends on ethical judgement and choice.

To make a responsible judgement—whether it be in law, science or art—would therefore involve at least the following components:

• Respecting otherness and difference as values in themselves.
• Gathering as much information on the issue as possible, notwithstanding the fact that it is impossible to gather all the information.
• Considering as many of the possible consequences of the judgement, notwithstanding the fact that it is impossible to consider all the consequences.
• Making sure that it is possible to revise the judgement as soon as it becomes clear that it has flaws, whether it be under specific circumstances, or in general. (Cilliers, 1998, p.139)

These points seem as relevant today, if not more so, than when they were written in 1998. Respect for differences and an understanding of diversity is a key ethical rule for complex systems and no amount of retreating into homogeneous groups will help us cope with living in an increasingly complex world.

As Stephen Downes wrote in 2005 when proposing connectivism as a new learning theory appropriate for living and learning in a digitally connected world:

‘Connective knowledge is no magic pill, no simple route to reliability and perhaps even more liable to error because it is so much more clearly dependent on interpretation.’

but

‘Freedom begins with living free, in sharing freely, in celebrating each other, and in letting others, too, to live free. Freedom begins when we understand of our own biases and our own prejudices; by embracing autonomy and diversity, interaction and openness….’

I agree with dana boyd – diversity is hard, but if as Cilliers (2010, p.56) says, ‘Difference is a necessary condition for meaning’ in a complex world, in order to learn we will need to embrace diversity and maintain, sustain and increase our global networks and connections.

References

Cilliers, P. (1998). Complexity and postmodernism. Understanding complex systems. London and New York, Routledge

Cilliers, P. (2010). Difference, Identity, and Complexity. Philosophy Today, 54(1), 55–65.

Downes, S. (2007). An Introduction to Connective Knowledge in Hug, Theo (Ed.) (2007): Media, Knowledge & Education – Exploring New Spaces, Relations and Dynamics in Digital Media Ecologies. Proceedings of the International Conference held on June 25-26, 2007. – http://www.downes.ca/post/33034

Eynon, R., Hjoth, I., Yasseri, T., & Gillani, N. (2016). Understanding Communication Patterns in MOOCs: Combining Data Mining and qualitative methods. In S. ElAtia, D. Ipperciel, and O. Zaïane (Eds.), Data Mining and Learning Analytics: Applications in Educational Research, Wiley.

Haywood, J. (2016). Learning from MOOCs: lessons for the future. In E. de Corte, L. Engwall, & U. Teichler (Eds.), From Books to MOOCs? Emerging Models of Learning and Teaching in Higher Education, p. 69-80. Oregon: Portland Press Limited.

Kirschner, P. A. (2015) ‘Facebook as learning platform: Argumentation superhighway or dead-end street?’ Computers in Human Behavior, vol. 53, December, pp. 621–625. Elsevier Ltd. [Online] Available at http://dx.doi.org/10.1016/j.chb.2015.03.011

Tschofen, C., & Mackness, J. (2012). Connectivism and Dimensions of Individual Experience. The International Review of Research in Open and Distance Learning, 13(1). http://www.irrodl.org/index.php/irrodl/article/view/1143

Williams, R., Karousou, R., & Mackness, J. (2011). Emergent Learning and Learning Ecologies in Web 2.0. The International Review of Research in Open and Distance Learning, 12(3). http://www.irrodl.org/index.php/irrodl/article/view/883

Update on OldGlobeMOOC and Peer Assessment

OldGlobeMOOC is about to start it’s 4^th week (following a week’s break for July 4^th celebrations in the US), and the Week 3 assignment peer reviews are in. For me this assessment process is one of the most interesting aspects of this xMOOC. I have thought since the first MOOC in 2008 (CCK08 Connectivism and Connective Knowledge), designed and run by Stephen Downes and George Siemens, that assessment may be the sticking point for MOOCs.

In my last post , I outlined some of the difficulties that OldGlobeMOOC is experiencing with the assessment and peer review process. It seems to me, once again, but this time for an xMOOC, that if MOOCs are going to be sustainable and successful, then the assessment process has to be ‘cracked’ and meaningful.

Some MOOCs have taken the approach of restricting the number of participants who can be assessed. CCK08 did this. I think the number was 25, and FSLT12 and 13 have done this with a similar number – the idea being that  a small number of participants can be assessed by a tutor. FSLT13 offers credit for this:

The course has been recently accredited (10 transferrable academic credits at level 7, postgraduate). FSLT is recognised towards the Oxford Brookes Postgraduate Certificate in Teaching in Higher Education (PCTHE) and Associate Teachers (AT) courses. (http://openbrookes.net/firststeps13/)

But these are cMOOCs.

OldGlobeMOOC has taken a different approach as I described in my last post and I understand from other participants that this is similar to a number of other Coursera MOOCs.  For me this my first xMOOC, but it is not for quite a few OldGlobeMOOC participants, who have taken numerous Coursera courses and in the forums have shared their experience of the peer review process.

I will add my experience to the mix, and just so you know what we are talking about here are links to my assignments with their peer reviews.

Assignment 1 with peer review

Assignment 2 with peer review

Assignment 3 with peer review

If you read these, you will see that the assignments are not very different in their style and level to my blog posts, i.e. they are not academic pieces of work  – rather discussion pieces or personal reflection. And judging by the assignments I have reviewed, other participants’ assignments are of a similar level.

Which brings me to the review process, which I reflected on in my last post, but will add a few things here.

• The idea is that each participant submits an assignment and peer reviews five assignments for each week, which I have done. If this is not done, i.e. the peer review, then a 20% penalty is incurred.

All students wishing to obtain a Statement of Accomplishment must achieve 7 out of 12 points and submit 5 peer reviews each week. If a student fails to complete the 5 peer reviews, that week’s assignment will incur a 20% penalty.

Despite the fact that I definitely submitted five peer reviews for Assignment 3, I received a 20% penalty and therefore scored 1.6 instead of 2. It’s very easy to know that you have completed the 5 peer reviews, by the way the Coursera system takes you through the 5 assignments allocated for review; and the system confirms for you at the end of the process that you have submitted 5 – so I know that I did. So there’s been a blip in the system somewhere. It’s not a big deal for me, as I’m only doing this to experience the process and because I like the assignments and find the discussions interesting. I am not doing the course for the Certificate – but I do wonder how a blip in the system affects people who are really keen to receive a Statement of Accomplishment.

• There is no guarantee that you will receive 5 peer reviews. I received five in Week 1, three in Week 2 and four in Week 3. There has been some discussion in the forums about how this might affect the overall system and whether or not you have to review more than 5 assignments to receive 5 reviews.

• I have no complaints about the quality of most of the peer reviews and so far no one has given me a score of less than 2 – but this peer review for Assignment 3 is indicative of how the game can be played to ensure that you get a Certificate. It made me smile 🙂

peer 2 → I’m headed for an airplane so don’t have time to review, and I won’t be back until after evaluation time ends so I’m just giving everyone a 2. 

Aside from this here are some further reflections. The OldGlobeMOOC is a great experience in terms of the diversity of participants. Unfortunately the younger participants, in their teens, who signed up, seem to have fallen out of the discussion forums. This does not mean that they are no longer participating through observation and reading – it’s difficult to know. But I have wondered how an 11 year old might review the assignment of an academic Professor, or how an academic Professor might respond to a learner with special needs, or a very young participant, or someone whose first language is not English, and so on. The assignment submission is anonymous. Do these differences have implications for the equity of the peer review process?

Despite all this I am finding OldGlobeMOOC a fascinating and enjoyable experience and am looking forward to the start of Week 4.

 

Diversity and Peer Assessment in OldGlobe MOOC

I am finding OldGlobeMOOC a fascinating experience – quite unlike any other MOOC I have participated in, and my first xMOOC. For me one of the best things about OldGlobe is the diversity of the participant group. All the other MOOCs I have participated in have attracted groups in which similarities can easily be seen, i.e. mostly from academic backgrounds or interested in e-learning technologies.

But OldGlobeMOOC is truly diverse. It has attracted a huge age range from 11 to upper 80s, and people from all continents apart from Antarctica, although as you would expect the American participant presence is, I think, dominant – it would be interesting to see the analytics. But more importantly, it has attracted people from all walks of life and from very different education backgrounds. We have people very knowledgeable about the discipline of gerontology and related health and medical issues, but also very many people who have no subject related background other than we are all ageing and therefore all have a point of view.

This diversity is great. The stories being told in OldGlobe are richly diverse and a privilege to read.

But this diversity brings its own problems in relation to peer assessment of the OldGlobe assignments.

I mentioned in a previous post how the assignments are open to a range of approaches and creativity. Originally I hadn’t intended to complete the assignments, but I was drawn in by the energy and enthusiasm of the OldGlobe community and I’m glad I was.  I have seen examples of participants posting videos of themselves speaking about the assignment question, writing about their own personal experiences, drawing on academic literature, posting links to videos, websites and photos, and drawing on literature and poetry to illustrate their response. A requirement of the assignment process is peer review of 5 assignments and I have reviewed some wonderful submissions. The first assignment I reviewed used this song in response to the question ‘What is ageing?”

Fantastic!

But the peer review process is where the wonderful diversity in OldGlobeMOOC creates problems. We have the whole continuum of people from those who have no experience of peer review to people who have worked in Higher Education for years and are very experienced in assessing student assignments and reviewing research articles.

This is a dilemma for OldGlobe, because some participants are getting a bit of a rough deal in terms of their feedback, despite the OldGlobe team urging participants to be generous with their feedback and scoring. For example, one participant has been accused of plagiarism for his/her original submission, another has been accused of plagiarism for using an essay site, even though this was cited as a source, another has received the feedback ‘I don’t get it’ and a mark of zero for an academic piece of work, others have received one line or less in their feedback. It all seems a bit of a lottery. So diversity brings both advantages and disadvantages. How might this dilemma be overcome?

I applaud the OldGlobe team for designing the MOOC to attract such a diversity of participants and for designing the assignment tasks such as they can be completed by anyone from any background. We are all getting older. We all live in a society where we can see people getting older. We all know old people. We all have some thoughts and perspectives about the ageing society. Even an 11 year old can say something about their grandparents and an 89 year old has a wealth of experience to share. We can all read other people’s discussion forum posts and assignments and have a response.

OldGlobe has asked us to answer the following questions when responding

Please type your 100-250 word peer assessment below.

What do you think about this participant’s portfolio item choice to answer this question of the week?
How does this participant’s perspective differ from your point of view?
How is your point of view similar?

I think the whole age range could do this, provided they could understand the submission.

The problem comes with the scoring. Here are the instructions:

Here is the rubric for the assignment. You’ll use this as a guide to complete your own work in the Submission Phase, and as a guide for grading your peers in the Evaluation Phase:

2 points	Assignment is completed with a clear commentary of 250 – 500 words that pertains to the question of the week
1 point	Assignment is completed with some commentary that may or may not pertain to the question of the week
0 points	Assignment is missing an item, a commentary, or both

It seems straightforward, but given the diversity of course participants is so open to misinterpretation or overly subjective interpretation, which some participants seem to have experienced.

For me, the peer review process on the first assignment has been positive. I have really enjoyed reading the submissions I was sent to review. They were not all academic pieces of work, but they had all been thought about and I appreciated the open sharing of experience however articulate or inarticulate that might be.

This MOOC is not for credit. Participants will simply get at Statement of Achievement.

All students wishing to obtain a Statement of Accomplishment must achieve 7 out of 12 points and submit 5 peer reviews each week. If a student fails to complete the 5 peer reviews, that week’s assignment will incur a 20% penalty.

This makes me wonder if we need points at all. I think the feedback is very valuable and I would prefer to call it feedback than peer review, which I think puts the emphasis in a different place.

But perhaps we don’t need the points. Perhaps it’s enough for participants to complete the assignments and 5 peer reviews to receive the Statement of Accomplishment. Of course, using this system, some people will receive the Statement of Accomplishment for exceptional work and some for simply submitting ‘any old thing’. But does that really matter, given that this course is not for Higher Education credit?

If it came to a choice between diversity and peer review – I would go for diversity, and trust that people are participating in the learning environment just as much as they want to and need to for their own purposes.

There is so much of interest in OldGlobeMOOC. As an educator myself I find this tension beween diversity and peer assessment very interesting, quite apart from the fascinating discussions about ageing.

The Case for SmOOCs

On reflection #fslt12 was a SmOOC – a small open online course. I suspect that just as the number of Massive Open Online Courses of the Stanford type will proliferate – at least in the short term – so too will SmOOCs.

SmOOCs have a lot going for them, principally in terms of the relationship between size, diversity and openness.

We had 151 people register for FSLT12 and 168 register for the Moodle site.  Canada, USA, South America, Africa, Europe, India, the Far East and Australia were all represented and at the time of writing 60 people have accessed the Moodle site within the last 3 weeks. We haven’t yet examined the data in any detail, so these are just rough estimates and we don’t know how many people accessed the Moodle site as a Guest. We had 28 people add their blog to the course WordPress site, but again we don’t yet know how many people blogged about the course, without aggregating their blog.  12 people completed the assessment activities.

So in my terms, compared to some of the MOOCs I have been involved with, this was a small MOOC.

As a result of this experience, my perception is that in SmOOCs, ‘openness’ is safer. It was interesting to observe this in FSLT12, which was open enough to ensure diversity, but small enough to ensure that ‘cliques’ didn’t form and that there was a very good mix between novice and experienced participants, different ages, disciplines and cultures. This in itself is interesting, as in the early days of MOOCs it was thought that large numbers were required for diversity. I have thought about and discussed this before – see

Mooc principles and course design

Change 11- massiveness and diversity

For me the question remains as to how massive does a MOOC have to be to hit the ‘sweet spot’ of diversity and openness. In 2012 Roy Williams, Sui Fai John Mak and I published a paper about the Ideals and Reality of Participating in a MOOC, where some of these tensions were discussed.

In FSLT12 I was surprised at how much diversity there can be in a much smaller MOOC – and equally surprised at how this did not lead to sub groups or cliques but to an apparent genuine desire to interact with this diversity.  In past MOOCs I have been involved with it has been the different cultures and resources that have offered the diversity, but in this MOOC, although it was enriched by different cultures, it was the mix of experts and novices that worked so well. This was particularly evident in the microteaching activity where both novices and experts engaged, supported and learned from each other. My feeling is that this was made more possible because of the smaller numbers and also because the smaller numbers made the learning spaces (Moodle and Blackboard Collaborate) feel more intimate, supportive and safe.

So I can see that SmOOCs can offer diversity with relatively ‘safe’ opportunities for connectivity, interaction, autonomy and openness, but do they avoid ‘group think’? This is something that I need to think more about.