Since my previous post on this subject, there's been a lot of talk about audio feedback at Alt-C (the Association of Learning Technology Conference): at least two papers on it and mentions in many more.
Much of the talk has been wildly enthusiastic, but as Isobel Falconer observed to me, most of the examples presented of good audio feedback were just examples of good feedback pure and simple, which would have been good even if they were given in print. People found it hard to distinguish the benefits which were due to the audio medium and the benefits which were simply due to giving better feedback. For example, in one workshop, we were invited to compare a piece of written feedback which was impersonal, brief and bueaucratic, written to a form template, with a piece of audio feedback which was warm and personal, supportive and encouraging, with suggestions for how to improve. Not exactly comparing like with like.
The Open University since its foundation has made it its business to provide its students with personal, supportive and extended feedback; in fact, it is is this, rather than its TV programmes or even its printed course materials to which its success is chiefly attributable. So what WE want to know is whether there is anything additional which the audio medium adds.
From the sessions I attended - together with the much more sophisticated research paper also presented at Alt-C by Sue Rodway-Dyer et al - here are some of the potential advantages.
It may make it easier for tutors to give better feedback, for example:
- they are less likely to slip into academic language
- it may be easier for non-native speakers of English, the medium being more tolerant of verbal idiosyncracies
- by making their feedback more of an object, from which they can stand back and experience it as students will, it prompts tutors to be more critical.
It may be easier for students to make good use of feedback, for example:
- they may find it easier to take in, if their literacy is poor
- it forces them to pay attention, because it can't be skimmed over the way written feedback can (we know this is a problem) - in fact, students apparently often listen to audio feedback two or three times and sometimes make notes
- BUT where their assignments are long and discursive, it can be harder for them to locate the particular places being referred to in the feedback.
Some lecturers are reporting that audio feedback has encouraged dialogue between students and tutors, and between students themselves. Others have reported additional benefits if students themselves start to submit assignments in audio form also - for example, health studies students started to become more critical of how they related to their clients.
Having just heard Mike Wesch's keynote address on how media are not just tools but mediate relationships, it occured to me that this may be a case where a change in medium opens the possibility, at least of a change to the relationship: making it more personal, more aware, and more critical.
Alt-C had an inspiring conference keynote address yesterday from Michael Wesch, including extracts from his three excellent videos, all available on YouTube: The Machine is Us/ing Us, A Vision of Students Today, and An Anthropological Introduction to YouTube - the last two of which were the results of class projects. How cool must it be to study with him!
The keynote was pretty good too. I loved the idea of starting a conference about educational technology - in which every other person in the room had a netbook open and was madly blogging or tweeting - with an anthropologist talking about his fieldwork in Papua New Guinea, where there was not only no wi-fi or broadband, but no electricity or running water, and the people don't have fixed personal names (making census-taking - which is what Wesch was there to observe - highly problematic, until they invented the concept of the "census name").
His basic message - that media are not just tools but mediate relationships, so that when you change the medium you change the relationship, creating new ways of relating to others and knowing ourselves - was especially welcome to me, since I've been contending for a long time that we need to stop thinking of educational technology in the IT terms of information transfer or the liberal individualist terms of choice and consumption, and more in terms of the network of relationships in which all these activities happen.
Last week, a colleague urgently needed training in Elluminate (a synchronous audiographic conferencing tool) in order to support a live Elluminate session. I arranged for us to meet to set up a live online session together, working on separate machines in adjacent rooms, and on the advice of our local Elluminate "champion" urged her first to read the basic introductory guide so that she would "get the most out of the session", as we learning designers say. I also suggested that she prepare by going into the Elluminate "room" on her own, so that she could practice finding and using all the tools and buttons, even if there was nobody there to talk to.
As things turned out, she wasn't able to do any of the preparation - and I wonder if in fact this wasn't for the best. She was a very unconfident explorer of new software, despite being an experienced computer user, and perhaps the most useful thing I was able to do for her was to sit by her and take her on a tour of Elluminate - which I could do quite briefly, because I could say "and you can follow that up later in the documentation which I sent you".
Many computer users are confident enough within their regular everyday comfort zone but are paralysed when attempting to move outside it for fear of something going wrong which they can't repair. It's mistakes and difficulties which IT training never seems to address (unless it's following John Carroll's "minimalist" approach): trainers only tell you the procedure to follow, not how to recover when things go wrong. For many users, our standard "read first, experience afterwards" will be fine, but for such unconfident users it may be the company of a person (or the support of a personal relationship) which they need first to take them over the threshold. Then, once they have the experience and the confidence, they can explore and read on their own.
This isn't one of those blog posts about how learning could be much better if it was more like a computer game. It's a blog post about how learning DESIGN could be much better if it was more like computer games DESIGN.
I have long seen an analogy between designing games and desgning learning materials. In both cases, your goal is to give your user a good experience - yet you have no direct control over what they actually do with your materials. This analogy is particularly strong with "adventure games", which are based more on puzzles and exploration than on fast reactions and accuracy of hand-eye coordination.
I was reminded of this again through an article by Aaron Connors, the co-author of the highly-respected Tex Avery series of games and a strong proponent of story-centric gaming. Just try substituting "learning" for "game" in these extracts to see how well the analogy applies.
"When you boil game design down to the atomic level ... it’s pretty basic: Create an obstacle for the player. Give them the means to overcome it. Repeat. That’s it."
"[During the 1990s] too much focus was put on the technology. Often, some new innovation was enough to sell an otherwise miserable game."
"The two things casual gamers hate are frustration and confusion. And many of the “classic” adventure games were not only frustrating and confusing, they banked on it to extend gameplay. It may be an oversimplification, but in some respects, casual games could be adventure games with the frustration and confusion removed. In other words, take out the aimless wandering with no idea what (if anything) there is to find; always try to make certain the player knows what the current objective is."
"Better pacing, less navel-gazing, and I also think it’s a great idea to reduce frustration and confusion. We don’t have to get rid of them altogether – that’s what multiple levels of difficulty are for – but we can’t make games that rely on obscurity and repetition to pad game length. Some of you may love games where you’re free to get lost, stuck, annoyed, etc. – but you’re in the vast minority. Most people want games that are accessible, scalable to their skill level, intuitive, and fun. They want to be able to get in and out easily and do something satisfying without making a huge time commitment."
Convinced? I hope so. I don't know how useful this analogy will be in practice though, except to those of us used to thinking about games design, because the sad fact - often overlooked by those who want to make their course materials more like computer games - is that designing a good game is no easier than designing a good course. As Aaron Connors comments: "Have you noticed how seldom you hear well-known game designers criticizing other designers’ work? It’s because we know how tough it is to make a really great game."
A previous post (Audio feedback on written assignments) listed a few examples of tutors giving feedback to students in audio form. (Collective feedback, as a podcast to an entire class, is common; individual audio feedback is rarer and I think more interesting.)I've since found out that in my own faculty (Education and Language Studies) tutors on the language courses give audio feedback quite regularly - which should not be surprising: these being distance learning courses, students make audio recordings to practice their language productive skills, so it's natural for tutors to give feedback in the same medium.But reading about some highly successful podcasts by a philosophy lecturer at Glasgow made me wonder whether this model of spoken-assignment, spoken-feedback could be extended further. Why are these podcasts so popular? Of course, it may just be that she's a brilliant lecturer. But there's another possible reason, to do with the nature of philosophy itself: that it's a discipline which is practiced through language. To do philosophy is to talk. One can do philosophy through writing, of course, but talking is the original technology through which philosophy was created, and through which its business (the persuasive engagement of another soul) is most intimately conducted.If it were not for the traditions of the modern university, which since about 1800 have based their systems of assessment on written examinations, we might place more value on the spoken production of language. Philosophy is just one discipline where we might reasonably expect students to be able not only to think and write persuasively but to speak persuasively. Management is another (eg the team briefing, the strategy presentation, the three-minute business case to the Board). In fact, most professional disciplines have a communicative and persuasive component, which in practical situations is more often conducted in speech than in writing. Perhaps it's time that our assessment methods reflected this?
We seem to have realised, at last, that one cannot simply provide online collaborative technology to students and expect them to collaborate. (One of the most interesting findings from the JISC Great Expectations survey was that although young people may be experienced in social networking technology, on arrival at university they struggle to see how it can be used in learning.) So how should one introduce online collaborative work to students? What kinds of scaffolding and support do beginning students need?
I've just come across a paper (unfortunately it doesn't seem to be available online) by Andy Northedge, author of The Good Study Guide, describing how he and colleagues designed "a fail-safe online group project work environment" for students on an entry level Open University course in health and social care (KZX100). A high degree of structuring to tasks, supported by custom online software, meant that after an initial set-up stage student teams could work largely independently with little teacher support. Satisfaction levels were high, and - most remarkably - students were actually interested in each others' work. (This happens much more rarely than teachers suppose!)
The key features of the learning design were as follows.
(1) A highly structured team set-up stage. This addressed the critical issue for all collaborative work: that students need to be doing their work at roughly the same time - not trivial to arrange with distance learning students, for whom flexibility of study is often critical. Students began by indicating their availability during a six-week period, after which their tutor divided them into teams who could work on the project during the same two-week block. A coordinator in each team then obtained students' commitment to precise start and end dates. Importantly, students received credit for tasks in this stage, even before actually doing any work on their project, "to stimulate them to participate in a timely way and to commit to the team enterprise". (Without the custom software, other courses could do some of these tasks on Doodle.com - a free public web application for event scheduling.)
(2) Visible time targets for project milestones. These were initially generated by the software, from the start and end dates, but could then be adjusted by coordinators, allowing teams to re-plan their time strategy.
(3) An interface which rendered each student's work visible to other members of their team. Each student had first to find three websites relevant to their chosen concern and review them using an online proforma, and then to review two high-rated and two low-rated websites found by other students. The software displayed students' review of the same website in parallel columns, as well as a "hit parade" of websites in rank order.
(4) Discussion to facilitate the transition from exploration to production. After reviewing the websites, the students used an online forum to discuss six evaluation criteria (breadth, accessibility, trustworthiness, etc), drawing on their experience, and also working towards the joint report which they would next be writing.
(5) Individual authorship of discrete parts of the joint product. Each student wrote a separate section of the final report, summarising the team's discussion of one of the six evaluation criteria. The software only allowed them to work on their own section (or sections), but reading and suggesting improvements to other sections was encouraged.
(6) Assessment not of team product ("to reduce strain on team relations"), but of individual reflective essay on what has been learned from the project.
Although the course has now been superceded, the learning design remains valid. Like the InterLoc tool for structuring online discussion, this is an example of how structure and constraints on action can be useful in a teaching environment, as scaffolding to help students attain a level they would have found it difficult to reach on their own - in contradiction to the freedom ethos of Web 2.0.
(The full reference of the paper: Northedge A, 2006. Designing a fail-safe online group project work environment, Proceedings of International Conference on e-Learning: Learning Theories vs. Technologies, 14–16 December, 2006, Ramkhamhaeng University, Thailand, pp 12.1-7)
A colleague has just asked me whether I know of any courses where students get feedback to their written assignments in audio form - either as audio annotations to the document or a separate audio file.
I remembered hearing about people trying this, but I couldn't put my finger on any specific instances. There are none in my own faculty, as far as I can establish. The Open University's Cloudworks website includes some notes form people trying it at Athabasca University (Canada), Marietta College (USA), and Sheffield Hallam (UK) - but not the OU itself!
A quick check turned up some people at Staffordshire University using it in biosciences (August 2007), and a just-finished JISC-funded project at Leeds Metropolitan called "Sounds good", reported in The Guardian with a link to the new project website.
Sounds good indeed! They seem to have found some benefits for students, but I'm wondering what it does to the tutor workload. It ought to make it lighter, of course, but things are never so simple.
