Crafting Research

This seminar that I attended last week on crafting research was very interesting. It was organised by the Department of Organisation, Work and Technology at Lancaster University, UK,  and delivered by Professor Hugh Willmott  from City University London. Hugh Willmott has been working with Professor Emma Bell from the Open University. His talk was based on a paper they are working on, in which they are exploring the significance of crafting research in business and management, although having heard this talk the ideas presented seem relevant to social sciences research in general.

The essence of the work lies in an interest in how to produce well-crafted research and avoid Baer and Shaw’s (see reference list) criticism:

As editors, we are often surprised by the lack of “pride and perfection” in submitted work, even when there is a kernel of a good idea somewhere in the manuscript. Submitted manuscripts that report results from research designs in which many shortcuts have been taken are rather commonplace. In addition, many papers seem to have been hastily prepared and submitted, with obvious rough edges in terms of grammar and writing style.

In their article Baer and Shaw quote C.W. Mills as follows:

Scholarship is a choice of how to live as well as a choice of career; whether he knows it or not, the intellectual workman forms his own self as he works toward perfection of his craft; to realize his potentialities, and any opportunities that come his way, he constructs a character which has at its core the qualities of the good workman. —C. W. Mills, 1959

The seminar started with a look at the online etymology dictionary where we can see that the meaning of the word craft has, over time, shifted in meaning from ‘power, physical strength, might’ to ‘skill, dexterity’.

The thrust of the argument made was that researchers should shift towards being craftsmen who are dedicated to the community, have a social conscience and are aware of and acknowledge the ethical and political dimensions of their research. Such an approach would also openly acknowledge uncertainty and bias in research and the role of embodiment and imagination.

The image that ran through this presentation was Simon Starling’s art work ‘Shedboatshed’.

Starling was the winner of the Turner Prize in 2005.  For this work he dismantled a shed and turned it into a boat; loaded with the remains of the shed, the boat was paddled down the Rhine to a museum in Basel, dismantled and re-made into a shed. See:  http://www.tate.org.uk/whats-on/tate-britain/exhibition/turner-prize-2005/turner-prize-2005-artists-simon-starling for further information.

I’m not sure that I fully understand the significance of using Starling’s work in relation to crafting research unless it’s that his work has been described as research-based and clearly involves research and craft. Maybe it’s simply that Starling deconstructs familiar things to recreate them in different forms?

Interestingly, in the questions that followed the seminar, the thorny issue of having to write in a prescriptive way to be accepted in high ranking journals was discussed. Some members of the audience seemed to accept this as a given constraint which cannot be surmounted, i.e. we need to write and present research in a way which will not only be accepted by the given journal, but also will meet the requirements of the University’s REF. For some of the seminar participants there seemed to be no room for embodied, imaginative research which embraces uncertainty. My suggestion that we should perhaps look for alternative publishing outlets, blogs being one example, was met with an outcry of protest from one or two in the audience. ‘No one reads blogs’ they said, and besides ‘Blogging is cowardly’. I neither understood this nor agreed. The conversation seemed to endorse these sentences from Baer and Shaw’s paper:

Our goal was to reaffirm the notion that scholarly pursuit in the management sciences is a form of craftsmanship—we are craftsmen! Some may dismiss our arguments as idealistic or romantic. The realities of life as an academic, the pressures we are under—to publish in order not to perish—offer an all-to-convenient excuse to dismiss our ideas.

What a sorry state of affairs, but I do know from experience that many journals are not prepared to take a chance on non-conventional styles of presentation; Introduction, Literature Review, Method, Results, Conclusion remains the format most likely to get accepted and to suggest that the research endeavour might have failed or that there is a degree of uncertainty around the results is unlikely to lead to a favourable response. It seems there’s a long way to go before the idea of crafting research in the terms presented by Hugh Willmott is widely accepted.

A wide range of Literature was referred to in this seminar, which will be interesting to follow up on. See the references below.

References

Adamson, G. (2013). The Invention of Craft. Bloomsbury Academic.

Alley, M. (2018 4th edition). The Craft of Scientific Writing. Springer

Baer, M. & Shaw, J.D. (2017). Falling in love again with what we do: Academic Craftsmanship in the Management Sciences. Academy of Management Journal. 80(4), 1213-1217.

Bell, E., Kothiyal, N. & Willmott, H. (2017). Methodology-as-Technique and the Meaning of Rigour in Globalized Management Research. British Journal of Management, 28(3), 534–550.

Burrell, G. & Morgan, G. (1979). Sociological paradigms and organisational analysis: elements of the sociology of corporate life. Heinemann

Cunliffe, A. (2010). Crafting Qualitative Research: Morgan and S Smircich 30 Years On. Organizational Research Methods OnlineFirst.

Delamont, S. & Atkinson, P.A. (2001). Doctoring uncertainty: mastering craft knowledge. Social Studies of Science, 31(1), 87-107.

Frayling, C. (2017, reprint edition). On Craftsmanship: Towards a New Bauhaus. Oberon Books

Kvale, S. & Brinkmann, S. (2008 2nd edition). Interviews: Learning the Craft of Qualitative Research. SAGE Publication.

Wright Mills, C. (2000). Sociological Imagination. Oxford University Press.

Promoting discussion in FutureLearn MOOCs

                

MOOCs: Back to the Future. This was the title of a lunchtime seminar I attended last week in the Educational Research Department of Lancaster University, UK.

In this seminar Phil Tubman  (PhD student with Educational Research and School of Computing and Communications, and also Senior Learning Technologist in the ISS eLearning team at Lancaster), shared the progress he has been making on his PhD into whether MOOCs are living up to their promises. The title of his talk was chosen to suggest that with respect to MOOCs we currently have our backs to the future. As such we need to turn round and look for ways forward.

Given that he works for Lancaster University Phil is well placed to explore social learning in FutureLearn MOOCs. Lancaster University is a FutureLearn Partner.  Phil is interested in the problems of sustaining discussion in online forums and the role of the FutureLearn platform in this; FutureLearn claims to be a social learning platform.

Phil and his colleagues have looked at discussion threads in forums and found that they don’t include many members and that conversations often do not extend beyond a first reply. These findings are supported by other researchers. Phil and his colleagues have therefore designed an intervention which they hope will increase and improve interaction and social learning in these  forums. This is an interactive word cloud which they have called a Comment Discovery Tool (CDT). This tool displays the top 200 words from a forum thread as a word cloud. MOOC participants can then click on a given word to display all the comments which include that word. The example shown in the seminar was from FutureLearn’s MOOC on Wordsworth. Phil clicked on the word ‘daffodils’ in the word cloud which took us to all the relevant forum posts. Having done this, it would then be possible for the MOOC participant to follow the discussion from there without having to wade through potentially hundreds of forum comments and threads. Phil is continuing to develop this tool for his PhD and is interested in whether and how the comment discovery tool will affect interaction in the FutureLearn forums.

I look forward to seeing how this develops.  Phil and his colleagues, Phil Benachour and Murat Oztok have submitted a paper on this to the Learning Sciences Conference 2018, and a recording of Phil’s seminar will ultimately be posted here – http://www.lancaster.ac.uk/educational-research/news-and-events/seminar-series/

Teaching and Learning in 2017 and beyond

This is the time of year when many authors/commentators are getting into reflective mode and writing reviews of 2017, not only looking back but also looking forward. One of these is Stephen Downes, who has been prolific in the last few weeks, sharing a number of presentations and his current thinking.

Jim Groom in his blog post ‘Containing the Future of OER‘ draws our attention to Stephen’s presentation – Applications, Algorithms and Data: Open Educational Resources and the Next Generation of Virtual Learning. In this presentation Stephen discusses how The next generation of OERs will take a step beyond traditional media and classroom support and begin to take advantage of the unique properties of virtual learning’.

In writing about this presentation Jim Groom notes:

‘It’s funny how you begin to rely on certain figures in your network for so much that at some point you begin to take it for granted. Downes remains so prolific on so many topics—not to mention the single most important aggregator of the work in edtech—that you begin to just assume. This talk was a good reminder of how fresh and relevant his work remains.’

I fully endorse this statement about Stephen Downes and think this is particularly obvious in his recent publication about the future of teaching and learning. See

Downes, S. (2017). Quantum Leaps we can expect in teaching and learning in the Digital Age. A roadmap. Published by Contact North.  

https://teachonline.ca/sites/default/files/pdfs/quantum_leaps_we_can_expect_in_teaching_and_learning_in_the_digital_age_-_a_roadmap.pdf

This very fluent essay is about ‘the larger changes sweeping through society’. It is full of thought-provoking insights about where we have been, where we are and where we are going with teaching and learning, particularly in relation to the influence of technology. The essay ‘ is addressed to policy makers and pundits, to technology designers and developers, and to those who by virtue of office or inclination have the voice to speak to the future, to inform the weld of what we can do and what we want to do’, but must also be of significance to educators, teachers and learners as we can see from the following quotes:

page 9.  When we teach these students, it’s hard to fight the temptation to teach them for a world that no longer exists. It’s even harder not to teach them for conditions that apply today. The world we are preparing them for, however, is literally a next-generation world. We need to use the technologies of today to teach for the world of tomorrow.

page 14.  In an education system focused on the future, therefore, the core of learning is found not in what is defined in the curriculum, but in how teachers help students discover new possibilities from familiar things, and then from new things. It is, to my mind, transformation from an idea of education defined as acquisition of skills or progression along a learning path to one characterized by exploration, discovery and finally creativity.

page 17.  … despite the conservative nature of the educational system, we have conclusively and irreversibly entered the digital society, and students have permanently changed, as has society. The static structures that used to define education have shifted; we are in an era of changing boundaries between formal, non-formal, informal and post-formal education.

page 21. We can seek in vain to return to that former understanding, or, moving forward, we can seek to identify those core values that underlie the why of pedagogy. What did it matter what a teacher did? What were the outcomes we hoped for? How does our current understanding of instructional design meet those, or fall short? What can we develop in the future to address these issues?

These four quotes resonated with me, as I continue to struggle with what it means to be a teacher and learner in this digital age. Needless to say, I recommend reading the whole document.

 

#OpenEdMOOC – Final thoughts

(Source of Image – https://youtu.be/iHmHWVHmyfQ –  0.56)

Like Rebecca Heiser, I have been reflecting on the EdX Introduction to Open Education MOOC.

The title of the MOOC led me to expect a broad discussion about Open Education and in my first post about the MOOC, I wrote:

My hope is that it will offer a fresh perspective and rekindle the enthusiasm I had for open education in the early days of the first MOOC in 2008, but which I have found increasingly difficult to sustain in the last couple of years.

It turned out that my hope was not realised. The MOOC focused on Open Education Resources, which I think is just one aspect of open education, and I have to say the aspect that is of least interest to me since I work independently of an institution and no longer teach, so although I come up against some of the issues presented in the course materials, they do not have the impact on me that they might have on others. This was the syllabus for the course.

Week 1: Why Open Matters
Week 2: Copyright, The Public Domain, and The Commons
Week 3: The 5R Activities and the Creative Commons Licenses
Week 4: Creating, Finding, and Using OER
Week 5: Research on the Impact of OER Adoption
Week 6: The Next Battles for Openness: Data, Algorithms, and Competency Mapping

Disappointingly for me there was little discussion of open pedagogy or open teaching. But I did stick with the MOOC. I found the videos of discussions between George Siemens and David Wiley interesting, as were the videos by Norman Bier and other resources provided, and I was particularly grateful for the alternative thought-provoking perspective of Stephen Downes. Did I learn a lot? I don’t think so. I think this was because OER isn’t of huge interest to me, but also because there was minimal discussion between participants. But I don’t feel negative about the MOOC, just that it didn’t rekindle my early enthusiasm for open education.

Rebecca Heiser  has thrown down the gauntlet twice during the course and been largely ignored. First, at the end of the course she tweeted:

So will we get to see Open Analytics in #OpenEdMOOC? What were we clicking? What were our activity-levels? Hashtags generated? Most importantly what were our most discussed topics, was knowledge transferred? #OpenPL Did we form our own network? Will we continue the conversation?

I saw that Stephen Downes picked this up (I think in OLDaily), but otherwise no-one responded to these good and legitimate questions.

And then Rebecca tried again with a post reflecting on her experience of this MOOC.

I can’t help but be critical of the #OpenEdMOOC, Introduction to Open Education, hosted by edX. With as much excitement that was generated by the Open Community prior to the start of the MOOC, I feel that we were let down by Week 3 of the course. Here’s a recap of my critique by week…

This time she received a response (notably not from George or David) and a few comments.

Geoff Cain commented:

This was pretty ironic for me. MOOCs are the only educational space where instructors and institutions are comfortable with the “Set It and Forget It” model of teaching. Despite everything that we have learned about what makes for a successful online class, all the rules change when it comes to MOOCs. As an instructor who has worked in developmental education and education support my whole life, that model seems like a huge step backwards.

This, and Rebecca and Lena Patterson’s comments, seems to me what we should have been discussing, i.e. some of the more difficult consequences of open education and the questions that remain unanswered.

Having participated in and completed a number of MOOCs, I was not surprised by the “Set It and Forget It” model of teaching that Geoff mentions. From the very first MOOC (CCK08) the idea has been that participants teach each other and take responsibility for doing this, because in a massive networked environment it is impossible for one person (the tutor) to interact with everything that is going on. But the complexity of this as a model for teaching and learning, and whether it is an effective model, has yet to be adequately researched.

Thank you to Rebecca for bravely raising these questions in an environment which is known to reward consensus and punish dissent!

#SOCRMx Week 7: Objectivity and Subjectivity in Social Research

The past two weeks in the Introduction to Social Research Methods MOOC (SOCRMx) have covered ‘How to Lie with Statistics’ (Week 6 Quantitative data analysis) and ‘Interpretation and Trustworthiness in Qualitative Data Analysis’ (Week 7).

Lots of resources have been provided for the Week 6 topic on quantitative data analysis. I have saved them for future reference, should I ever go down that track. It was interesting to see that more course participants got involved in discussion in the Week 6 forums. I’m not sure if that’s because a positivist approach comes more easily to those new to research, or whether it was because once again the resources were extremely good and the tutor for this week – Rory Ewins – actually engaged in the forums and provided feedback!

The resources for Week 7 on qualitative data analysis are very familiar given that all my research has taken this approach, but once more the resources provided on what we mean by trustworthiness and how to code qualitative data are helpful and thought provoking. Again I have saved these resources for future reference when I can explore them with more time. I think both these topics require and deserve more time than one week to get to grips with.

The topic of qualitative analysis reminded me of an excellent video that was shown in the first week of the course under the title of theoretical considerations in research design. This is a SAGE research methods video and therefore not open access, but if anyone from SAGE visits this post, I would urge them to make this video publicly available. I think both novice and experienced researchers would find it hugely helpful.

The video shows John Scott (Professor of Sociology, Plymouth University), Malcolm  Williams (Professor in the School of Social Sciences, Cardiff University) and Gayle Letherby (Professor of Sociology , Plymouth University) discussing objectivity and subjectivity in social research, having published a book about this in 2014:

Scott, J, Williams, M & Letherby, G. (2014). Objectivity and subjectivity in social research, SAGE Publications Ltd., London.

These are the notes I made from watching the video.

John Scott starts by drawing on the work of Kant and Mannheim to discuss how subjectivity, objectivity, relativity and truth have multi-faceted meanings. We see the world differently according to our social location (our perspective) and we construct knowledge relative to that location. But if this is so, how, as researchers, can we present a truthful representation of the world. One way we can do this (which Iain McGilchrist has also discussed in his work on the Divided Brain) is to try and synthesise different standpoints (alternative perspectives) to achieve a better model for understanding the world.

The three authors each approach research from different perspectives. Malcolm Williams adopts a socially situated objectivity approach, Gayle Letherby a theorised subjectivity approach and John Scott, is somewhere in the middle of these two perspectives, and the point was made that it’s not necessarily a question of either/or, as shown by mixed methods approaches.

Malcolm Williams sees objectivity as a value which is socially constructed. The pursuit of truth is a key research value and he argues that a necessary condition of objectivity is to begin with subjectivity.

Gayle Letherby recognises the personhood of research and the complex relationship between researcher and respondent. From this perspective research is subjective, power laden, emotional and embodied. A theorised subjectivity approach is concerned with how identity plays out in the research process. Gayle Letherby tells us that interrogation of the self in research, with reference to the ‘other’, gets us closer to a position that we might call objective and that autobiography is always relevant. We need to interrogate our biases, but we should avoid demonization of subjectivity. Subjectivity is not a redefinition of objectivity, but starts from a different place. Objectivity and subjectivity are interrelated.

Whether we take a subjective or objective approach, social science research does have validity and needs to be defended. This does not mean that one account is as good as any other. Researchers must be responsible and explain how we got what we got and how what we did affects what we got. Research always involves a view from somewhere and we need to write about our subjective positions. Where did the research question come from? Why was a particular approach adopted? Can we justify why our findings are important? What is our ethical position? Have we acknowledged that ideas about knowledge can’t be separated from ideas about power? (Foucault’s work is relevant here) The social and political role of research means that there is no escape from issues of power.

Scott, Williams and Letherby conclude that the validity of knowledge depends on having:

  • Open societies (Karl Popper)
  • An ideal speech situation (Habermas)
  • Free discussion (Mannheim)
  • Leaving power outside the door as much as possible.

#SOCRMx Week 5: Data Analysis

In this second half of the Introduction to Social Research Methods course the focus shifts from data generation and research methods to data analysis.

 

(Click on image for source)

The main task for this week has been to look at the way in which the authors of two research papers (provided by the course) have analysed their data and presented their subsequent findings. One paper took a qualitative approach to data generation and the other a quantitative approach. Useful prompt questions have been provided to support this task.

For me, more interesting than this task are two points raised in the course text, with the suggestion that these are discussed in our blogs, but to my knowledge no-one has done this.

The first is related to the messiness of research which is drawn to our attention through a quote from Hardy and Bryman’s text – Handbook of Data Analysis – which incidentally is not open access.

This is the quote:

active researchers seldom march through the stages of design, data collection, and data analysis as if they were moving through security checkpoints that allowed mobility in only one direction. Instead, researchers typically move back and forth, as if from room to room, taking what they learn in one room and revisiting what was decided in the previous room, keeping the doors open. (Hardy and Bryman 2004, p.2)

This is very much in keeping with my experience, but I would suggest that it is even more messy than Hardy and Bryman suggest. Richard Feynman talked about living and working with doubt and uncertainty…

… and Paul Feyerabend argued ‘Against Method’. In the Stanford Encyclopedia of Philosophy is written of Feyerabend 

‘…. whereas he had previously been arguing in favour of methodology (a “pluralistic” methodology, that is), he had now become dissatisfied with any methodology. He emphasised that older scientific theories, like Aristotle’s theory of motion, had powerful empirical and argumentative support, and stressed, correlatively, that the heroes of the scientific revolution, such as Galileo, were not as scrupulous as they were sometimes represented to be. He portrayed Galileo as making full use of rhetoric, propaganda, and various epistemological tricks in order to support the heliocentric position.’  

More recently Stephen Downes has also argued against method suggesting that traditional approaches to research do not account for the horribly messy, complex, always changing world in which we are now living and conducting researchSee Digital Research Methodologies Redux for his presentation.

A course like this Introduction to Social Research Methods necessarily presents an orderly sequenced set of resources and activities, but research ‘in the wild’ is far from orderly.

The second point made in the Week 5 course text that stood out for me was this one:

Hardy and Bryman (2004) ……  also discuss data reduction as a core element of analysis (my bold): to analyze or to provide an analysis will always involve a notion of reducing the amount of data we have collected so that capsule statements about the data can be provided. (p.4)

This for me is an enormously significant statement. As McGilchrist says (p. 28 The Master and His Emissary)

The kind of attention we bring to bear on the world changes the nature of the world we attend to ……… Attention changes what kind of a thing comes into being for us: in that way it changes the world.

It follows then that how we choose to reduce the amount of data we have collected will determine what research outcomes ‘come into being’, what we learn from the research and will have implications for how the findings are used.

Both these statements in the Week 5 course materials, concerning the messiness of research and the reduction of data, seem to me to perhaps warrant more attention than they have received in the course.

References

Hardy, M. and Bryman, A. (2004). Introduction: common threads among techniques of data analysis. In Hardy, M. & Bryman, A. Handbook of data analysis (pp. 1-13). : SAGE Publications Ltd. doi: 10.4135/9781848608184

McGilchrist, I. (2010). The Master and his Emissary. The Divided Brain and the Making of the Western World. Yale University Press

#OpenedMOOC Week 6: The Triumph of the Immaterial

This is the final week of the #openedMOOC and my final post. Like Matthias Melcher I have surprised myself by reaching the end of this MOOC. When I first considered signing up, I thought the MOOC would be more about open education more broadly and discussions around open pedagogy, than about open educational resources. Also like Matthias I have not wanted to complete the MOOC tasks or meet the course objectives; instead I have followed my own interests.

Like some others – see the Twitter stream #openedMOOC – I have enjoyed the weekly videos of discussions between George Siemens and David Wiley, which succeeded in being very informative, and also Stephen Downes’ videos, because he always brings an alternative perspective.

Again this week, Merle Hearns has done a really good job of pulling together this week’s content in her blog,  so there is no need for me to repeat it. Merle discusses Norman Bier’s video, which is well worth watching and from which I made these brief notes:

For Norman Bier the future of OER will depend on how technology is used to collect and apply data, and provide better feedback for students and teachers. He tells us that our practice is already data driven and this should raise concerns for open education research, particularly if that research focusses on static content. He warns that there is no visibility on how data is being used and that it’s important for the OER movement to understand and explore the algorithms in analytic systems. Algorithms are not neutral and if we can’t avoid the biases of the developers getting into the systems then we need transparency to mediate this.

So having written all this you may be wondering why this post bears the title – The Triumph of the Immaterial.

The reason is that this is the title of a clay sculpture by ceramic artist Phoebe Cummings, who today won the BBC Woman’s Hour Craft Prize 2017, with this clay fountain:

Screenshot from BBC Radio 4 Website

And this photo by Laura Snoad gives us a close up of the detail of the work

In this video Phoebe talks about how her work is temporary – making the raw clay sculpture as a fountain means that the water will erode the clay over time.

And in this short video clip we can see the effect the fountain of water is having on the clay sculture

For source of video see: http://alreadyshared.com/explore/medias/1632069198719374070_1663972413

In hearing that this sculpture (a piece of work that will no longer exist when the created fountain erodes the raw clay) has won the prize, it occurred to me that this may inform David Wiley’s 5 Rs of open content. Phoebe Cummings has created a piece of work that she expects to be reused, reworked, remixed and redistributed. She created it to be ephemeral, to enact its own performance and to no longer exist after a period of time, in this case the time it takes for the fountain water to erode the clay.

Phoebe Cummings has a unique approach and way of thinking about materiality and ownership and I’m wondering whether she has anything to teach us about open education resources and copyright. Maybe the success of open education will depend more on how we think about issues such as copyright and ownership, rather than what we do about them.

Thanks to David Wiley and George Siemens for the MOOC, and to Stephen Downes and Norman Bier for their videos.