This is a question that was raised near the end of Tom Reeves very interesting presentation to ChangeMooc this week.
The focus of Tom’s presentation was educational research and the lack of impact of educational technology research on educational policy and practice. To address this problem he has worked with colleagues to develop a model – Design-Based Research. in which the focus is on researching a problem.
Tom emphasised the importance of academic rigour, but this led to the question – what is academic rigour? There was no ready answer or consensus in the session. Some answers to this question from Tom and participants were:
- not for the faint-hearted; takes effort and commitment (Tom Reeves)
- unchanging, in the sense that ‘rigorous’ means performing the same (type of) study every time, conforming to the same (set of) principles etc. (Stephen Downes)
- more likely to lead to the truth (but what is truth?) (Stephen Downes)
- disciplined, measurable, stands up to scrutiny by others (brainysmurf)
- can replicate the methods (Tom Reeves)
None of these answers quite satisfies me and this dissatisfaction has led to some further thinking and discussion.
It seems to me now that there is not an absolute concept of academic rigour – but rather there are degrees of it depending of the closeness of research to known theory, whether or not the research is supported by known theory and the credibility of the data from which inferences are drawn. This can be thought of in terms of the following diagram:
Following a given model or a systematic process won’t necessarily lead to academic rigour or even reliable success. I was interested in Stephen Downes’ comment that he is more a follower of Feyerabend in being “Against Method” and that ‘there is great liberation in understanding that ‘method’ is based on sociological desire for conformity rather than scientific desire for truth’.
Finally, there was the question of whether or not research needs to start with a problem – the Design-Based Research Model states that it does …..
… but this wouldn’t account for ‘Eureka’, ‘Ah Ha’ moments or accidental findings . This would suggest that ‘academic rigour’ is not always needed for good research.
Jenny Connected 
Just two quick comments on what sounds like an interesting discussion .
1. Design Research is quite widely used not just in Ed Tech.
2. In my own discipline Information Systems, where many of us would question ‘truths’ about complex human endeavours (as well as simplistic methods), there has been a long-running dialogue about the tensions between rigour and relevance see http://scholar.google.co.uk/scholar?q=rigor+and+relevance&hl=en&as_sdt=0&as_vis=1&oi=scholart&sa=X&ei=l_NAT-2KHIjC8QPxp_TMDQ&ved=0CCIQgQMwAA . I think that people with different ontologies and epistemologies would have their views on rigour. So it’s not that rigour is not needed, more that in can be variably interpreted.
So my view is that we need a good and approoriate combination of rigour and relevance in ‘good research’.
Hello Frances – thanks for your comment. I hadn’t meant to imply that Design-Based Research is just for Ed Tech. In fact the case study that Tom Reeves used in his presentation was not an Ed Tech one.
Thanks for the link to the discussions about the tensions between rigour and relevance. I think Tom Reeves’ emphasis on ‘impact’ is similar to the quest for relevance.
My question was more about what we mean by rigour. It seems unreasonable to expect people to be rigorous in their research, but not to be able to say clearly what we mean by rigour.
Sorry if my reference to ‘accidental’ outcomes was too much of a distraction from the main question. It just struck me as another interesting point.
Jenny
A scientist might mean one thing by rigour (randomised controlled trails, etc.) whereas an interpretive researcher might mean something else (clear methodology, recognition of role of researcher, etc.) and a critical researcher something else entirely. These could all be clear but different – do you we should be able to agree one meaning for rigour? or establish attributes of rigour that can fit different epistemologies?
Hi Frances – you have pinpointed the question I was struggling with.
I was interested that in Tom Reeves’ session – which was about conducting research, neither he nor anyone else was clear about the meaning of academic rigour.
Maybe this is because it means different things to different people according to the research they are doing, as you say – but this doesn’t feel very satisfactory to me – which is why I tried to sort it out in my own head in the ‘What is Academic Rigour?’ diagram I drew in my post.
It seemed to me that all research can be discussed in terms of theory and data and that we can determine how rigorous the research has been by thinking about the extent to which it relates to theory and whether or not there is supporting data and how well that data stands up to scrutiny. By drawing the diagram, I was able to see that four different types of writing can be examined in this way for rigour.