Benefits of e-learning: ROI or R&D?

Here's a nice graph, which I found in a JISC report with the compelling title Tangible benefits of e-learning (p.15). The authors used it to categorise their 37 case studies of e-learning in UK universities, and it's very useful to keep the different kinds of potential benefits clear in one's mind.

Here's how they explain the graph:

X Axis – Nature of issue. The x axis of the graph in Figure 1 shows the type of tangible benefit demonstrated and the sort of metrics that can be used to evaluate such benefits…. A well-defined problem such as how to assess large cohorts of students within a tight time-frame can be measured against a very specific and readily quantifiable set of metrics and that it is relatively easy to put accurate figures on time and cost savings. …Towards the middle of the scale we find activities where the intended benefit is to improve learners‟ understanding of a particular subject – in other words a pedagogically-driven change where the tangible benefits can be measured in terms of course or module pass rates or other direct measures of achievement. At the
far end of the scale, we encounter approaches intended to address far ‘softer’ and more complex issues of student engagement. …

Y Axis – e-Approach. The y axis shows how the ‘e-approaches’ differ in nature from those that seek to automate existing practices through those that add increased value by the application of information to those that ultimately seek to transform the learning process. The term ‘informate’ is taken from Zuboff (1988)…. Schein (1989) makes the further distinction between ‘informating down’ whereby control type information is passed downwards and ‘informating up’ whereby those closest to the issues pass information up the chain (in our case upwards from the student to the lecturer). It can thus be seen that the approaches clustered in the bottom left quadrant are those that represent the clearest return on investment (ROI) and it is easily possible to assess their scalability and the value for money represented by further investment. Those in the top right quadrant however are more research and development (R&D) in nature and in their present form may represent overheads without any immediately obvious return.

When you're talking to people about the use of technology in teaching, it can be helpful to clarify whether you're looking at ROI (saving time, effort and money, eg in development, production, revision, administration), R&D (eg virtual worlds, mobile interactivity, "blue skies" technologies), or most likely somewhere in between: improving learning and teaching by making it more engaging, more penetrating and more effective.