Tag Archives: audio feedback

• 1 Comment

Share this post on:
| | |
Posted in Assessment, Feedback and tagged , on by .

Previously I’ve promoted the use of audio and video feedback on student work. Methods I’ve highlighted include creating audio and video files using a wide range of software tools and distribution methods. (At this point I would normally direct you to my Student Audio Feedback: What, why and how post but recently rediscovered ALT-C 2009 II: Audio and screen visual feedback to support student learning (and research methodologies), which is pretty good)

Recently a member of staff from one of our supported institutions interested in the use of this form of feedback contacted me with concerns over students reposting personal feedback in the public domain i.e. just as a tutor respects a student’s privacy in not publishing a student work without permission, shouldn’t students do the same. In particular they were wondering if any student declaration was needed to prevent this from happening.

My initial response was along the lines of that any feedback produced by the tutor would remain the intellectual property of the institution and any public reposting would automatically need the consent of the institution, therefore all the tutor needs to do is highlight the existing legal position rather than having students make any extra declarations. But as I wasn’t completely sure of my interpretation of IPR I put a query with JISC Legal and here was the response I got (Disclaimer: The following text is provided as information only and does not constitute formal legal advice):

The recording of the feedback given by the lecturer will either belong to the lecturer or the institution.  S.11(2) of the Copyright, Designs and Patents Act 1988 provides that the employer will be the first owner of copyright, unless there has been an agreement otherwise.  It could be that there is sufficient ‘dramatic’ content in giving the feedback too that there is a performer’s right in the recording too, which would stay with the academic, unless there is agreement to transfer those to the institution.

In any case, the student would need to get permission before doing any of the copyright-restricted acts, which would include copying the work, adapting it, and communicating it to the public by internet dissemination in this particular case.  It may be worth reminding the students of this, and I’d suggest including an explanation that the feedback is personal and given within the teaching relationship, and so dissemination of the work would be disrespectful as well as copyright infringement.  Beyond the legal issue, it might also be worthwhile addressing the underlying reasons why the student or students might want to share the feedback – is there a need for more generic feedback that can be shared more widely?

So generally speaking my guidance was along the right lines, but the information from JISC Legal not only identifies particular nuances of the legal implications but also highlights how the risk of getting into problems can be mitigated and addressing some of the fundamental pedagogy. I hard to see how advice like this could get any better.

This isn’t the first time JISC Legal have provided some first-rate guidance and if you haven’t checked out their service it’s well worth an explore. Before you think this level of support is only available to other JISC Advance and JISC related staff it’s not. JISC Legal endeavour to support anyone in the UK tertiary education sector “to ensure that legal issues do not become a barrier to the adoption and use of new information and communications technologies”.

As well as individual guidance JISC Legal have a wealth of support material. Recent goodies include:

JISC Legal = pure quality btw

Share this post on:
| | |
Posted in Assessment, Feedback, How-to and tagged , on by .

Flickr Tag Error: Call to display photo '2469419808' failed.

Error state follows:

  • stat: fail
  • code: 95
  • message: SSL is required

A long, long, long time ago I wrote a post Using Tokbox for Live and Recorded Video Feedback in which I demonstrated how the free ManyCam software could be used to turn your desktop into a virtual webcam to provide feedback on students work in a Russell Stannard styley. Recently my colleague Kenji Lamb was showing me how you could directly record your webcam using YouTube, so I thought I would revisit this idea.

This time instead of focusing on the use of the visual element as a tool to direct students attention to a specific part of a assessment submission (e.g. highlight and talking about parts of a word document), I thought it would be interesting to demonstrate it in a more abstract way using images to reinforcing audio comments (e.g. you did good – happy face; you did bad – sad face).

When previously looking at audio feedback I’ve been very aware that reducing as much of the administrative burden is very important. Online form filling whether it be through the VLE, other systems or in the YouTube example, can be a bit of a chore so in this demonstration I also touch upon using bookmarklets to remove some of the burden. Here is a link to the bookmarklet I created for student feedback on YouTube (YouTube Feedback Template – you should be able drag and drop this to your bookmark toolbar but if you are reading this through an RSS reader it might get stripped out).

Having this link in you toolbar means when you get to the video settings you can click it to populate the form. Bookmarklets are a nice tool to have in the chest so I’ve covered them in more detail in Bookmarklets: Auto form filling and more. This post also shows you how you can create your own custom filling bookmarklet using Benjamin Keen’s Bookmarklet Generator.

So here it his a quick overview of using YouTube for recording student feedback:

[utitle mode=2 comment=yes]lQ0KclIhfl8[/utitle]

• 1 Comment

Share this post on:
| | |
Posted in Assessment, Feedback and tagged , on by .

Flickr Tag Error: Call to display photo '2597104631' failed.

Error state follows:

  • stat: fail
  • code: 95
  • message: SSL is required

For the next post in my ALT-C series I’m going to highlight a session I didn’t actually attend but immediately regretted when comments started filtering in on twitter.

The session was based around the paper by Rodway-Dyer, Dunne and Newcombe from University of Exeter which summaries a study of audio and visual feedback used in two 1st year undergraduate classes. Click here for the paper and abstract.

Comments I picked up on this paper via twitter appeared to show audio feedback was not well received. Issues highlighted were:

  • the finding that “76% of students wanted face-to-face from a tutor in addition to other forms of feedback” [@adamread, @JackieCarter]
  • students found that receiving negative audio comments was harder than when written [@adamread, @ali818, @narcomarco]. Although this is still open to debate as @gillysalmon said that “duckling project at Leicester has found human voice easier to give negative feedback by audio than text”

Obviously there are issues with making assumptions based on a few 140 character tweets and it should be noted that the authors conclude that “overall, it seems that "there is considerable potential in using audio and screen visual feedback to support learning”, although students did express concerns in a number of areas.

Having had a chance to digest the paper the question I’m left with is how much of the negative experiences were a result of the wider assessment design rather than the use of audio feedback in itself. For example, reading the focus group discussions for audio feedback in geography I noted that:

  • students were not notified that they would be receiving audio feedback;
  • that despite the tutors best attempts students hadn’t engaged with assessment criteria; and
  • that this was the first essay students submitted at university level and they were unclear of the expected standards.

Similar issues to these were addressed in the Re-Engineering Assessment Practices (REAP) project, which produced an evolving set of assessment principles. Principles which could be successfully applied to the geography example might be:

Help clarify what good performance is – this could be achieved in a number of ways including creating an opportunity for the tutor to discuss criteria with students, or perhaps providing a exemplar of previous submissions with associated audio feedback.

Providing opportunities to act on feedback – as this was the students first submission providing feedback on a draft version of their essay not only allows students to act on feedback (it’s not surprising when students ignore feedback if they have no opportunity to use it).

Facilitates self-assessment and reflection - One of the redesigns piloted during REAP was the Foundation Pharmacy class, in which students submitted a draft using a pro-forma similar to that used by tutors to grade their final submission. Students were required to reflect on distinct sections of their essay, which again also allowed them to engage with the assessment criteria.

Encourage positive motivational beliefs – using the staged feedback described above would perhaps also address the issue of students becoming disillusioned.

Talking to a friend during the lunch break the research methodology used by the authors was also mentioned, in particular the use of ‘stimulated recall’. For this the authors played back examples of audio feedback to the tutor asking him to explain his thought processes and reflect on how his students would have responded to his comments. This methodology seems particularly appropriate to evaluate the use of audio feedback, and is something I want to take a closer look at.

A moment of serendipity

Whilst searching the twitter feed for comments on the session I noticed a tweet by @newmediac which was promoting a free webinar in which  “Phil Ice shares research on benefits of audio feedback” (here’s the full tweet). The session has already passed  but the recording for this event is here.

Tweets - Moment of serendipity
Moment of serendipity

The presenter, Phil Ice, has been working on audio feedback in the US for a number of years and has a number of interesting findings (and research methodologies) I haven’t seen in the UK.

For example, Ice and his team report:

students used content for which audio feedback was received approximately 3 times more often than content for which text-based feedback [was] received”

and that

students were 5 to 6 times more likely to apply content for which audio feedback was received at the higher levels of Bloom’s Taxonomy then content for which text-based feedback was received”.

These results were from a small scale study of approximately 30 students so aren’t conclusive. Ice has also conducted a larger studies with over 2,000 students which used the Community of Inquiry Framework Survey. Positive differences were found across a number of indicators including excessive use of audio to address feedback at lower levels is perceived as a barrier by students.

Ice has also conducted studies which breaks audio feedback into four types: global – overall quality; mid level – clarity of thought/argument; micro – word choice/grammar/punctuation; and other – scholarly advice. The study indicates that students prefer a combination of audio and text for global and mid-level comments.

Findings from Ice have been submitted for publication in the Journal of Educational Computing Research (which will soon feature a special issue on ‘Technology-Mediated Feedback for Teaching and Learning’).

Screenshot showing inline audio comments
Screenshot showing inline audio comments

Finally, I would like to mention the method Ice uses for audio feedback. He uses the audio comment tool within Acrobat Pro 8 to record comments ‘inline’. This appears to be particularly useful for students to relate comments to particular sections of their submitted work. Click here for a sample PDF document with audio feedback (this isn’t compatible with all PDF readers - I’ve tested on Acrobat Reader and Foxit Reader).

Hopefully this post has not only stimulated some ideas in the use of audio feedback, but also highlight a range of methodologies to effectively evaluate it.

• 3 Comments

Share this post on:
| | |
Posted in Assessment, Feedback, Podcast and tagged on by .

Listen to this:

Download link

Flickr Tag Error: Call to display photo '2597104631' failed.

Error state follows:

  • stat: fail
  • code: 95
  • message: SSL is required
I captured this clip of students from the University of Chester and Sheffield Hallam talking about their experiences of receiving audio feedback at the last Podcasting for Pedagogic Purposes SIG at Glasgow Caledonian University. This event was very fortuitous as at the time I was helping source material for a Queen Margaret University (QMU) staff workshop on this very topic. If you are still unclear as to what audio feedback is here is a nice description from Andrew Middleton:

Audio feedback can be defined as formative messages, recorded and distributed as digital audio to individual students or student groups in response to both ongoing and submitted work, allowing each student to develop their knowledge and the way they learn. (Middleton, A. 2008)

Why do I think audio feedback is worth exploring? It is clear from the National Student Survey (NSS) that feedback is a particular area if dissatisfaction. There is a growing pool of evidence that students perceive audio feedback as a positive to their learning experience (although I’m not aware of research on actual learning gains).

Existing practitioners/projects

In my research for the QMU workshop I came across a number of projects and practitioners exploring this area, which I believe are worth sharing in this post:

[If I’ve missed anyone off please use the comments to highlight them and their work]

In researching audio feedback it was clear while there were distinct benefits but there were some reoccurring themes in terms of limitations. First is scalability. It’s all well and good providing individual feedback to up to 50 students but with figures beyond this it just becomes to onerous.

Audio feedback models

A solution to this problem is to explore other audio feedback models. Andrew Middleton at Sheffield Hallam University has identified a number of alternative models which include:

  • Personal tutor monologue – tutor feedback to individual student
  • Personal feedback conversations – recording tutor/student meetings
  • Broadcast feedback – generic feedback to the class
  • Peer audio feedback – student generated audio feedback
  • Tutor conversations – recording teaching staff conversations

More information on these models is available in a presentation made by Andrew at the Blended Learning Conference or in a forthcoming paper entitled ‘Audio Feedback design: principles and emerging practice’.

Don’t expect to save time

A finding from the Sounds Good 2 project is that providing individual student feedback is unlikely to save any staff time (there is a messy debate about whether high quality feedback offers long term gains in terms of how much additional feedback is required further down the line). Sounds Good have however circumstances where time can be saved (taken from the Sounds Good Final Report):

  • The assessor is comfortable with the technology.
  • The assessor writes or types slowly but records their speech quickly.
  • A substantial amount of feedback is given.
  • A quick and easy method of delivering the audio file to the student is available.

Technology

In terms of the technology there are a number of solutions which various projects propose. These include:

  • using a digital voice recorder which saves the files directly into MP3 format
  • using you Mac/PC using free audio recording software like Audacity (a portable version of Audacity on EduApps) or Wavosaur (this also can be used without installing any software)
  • using a mobile phone – if this feature is available

I also recently posted about Using Google Talk for Audio Feedback. This solution appears to be more troublesome than its worth but was useful as Joe Dale, via Andrew Middleton, highlighted the Vocaroo web service which look incredibly easy to use and worth a look at.

General advice

So what advice would I give to anyone thinking about using audio feedback? There are some very good recommendations from the Sounds Good project (final report) which are worth highlighting grouped under 4 themes: saving time; technical matters; administration; and structure:

Saving time

  • Don’t expect immediate savings in time, if any. Think of the long term returns for you and your students (there is anecdotal evidence that audio feedback reduces the need for follow up face-to-face sessions)
  • except minor mistakes. You are not looking to produce broadcast quality audio

Technical matters

  • optimise your files to minimise download size. Recommended mp3, mono and if possible reduce the bitrate to 32-40kbit/s
  • check files can be played on campus computers
  • make sure the audio is loud enough
  • have clear guidance on how to play files (I wouldn’t bother with guidance on playing files on portable media devices as the majority of students appear to prefer the convenience of playing them from a desktop computer)
  • have a backup of files in a secure location
  • if using shared devices or computers make sure files are deleted once backed up

Administration

  • if audio feedback is particularly being used as part of summative assessment make sure you have a conversation with quality assurance
  • make sure audio files are securely stored and distributed

Structure

  • try to personalise the feedback by introducing yourself, the assignment you are giving feedback on and refer to the student my name
  • keep focused – a 2 minute piece of feedback can be as, if not more, beneficial than a 10 minute ramble (the clip at the beginning of this post is just over a minute and conveys a lot of information)
  • I would recommend using audio feedback as supplemental to written feedback.

In terms of a procedure for creating audio feedback Bob Rotheram from the Sounds Good project recommends this procedure and general structure:

Feedback Procedure

  • Have the assignment details and assessment criteria with me.
  • Read the assignment, making written comments on it as I go along. If it’s on paper, I jot things in the margin. If it’s in an electronic format (e.g. Word), I use the ‘Track changes’ facility to annotate the document.
  • Read it again, more quickly this time, perhaps making a few more comments along the way.
  • Jot down (on scrap paper) the main summary points I wish to make.
  • Start the MP3 recorder.
  • Don’t bother to erase and re-record ‘misspeaks’; just correct them immediately, as in conversation.

General structure

  • Introduce yourself to the student in a friendly manner.
  • Say which assignment you’re giving feedback on. Outline the main elements of the comments which you’ll be giving (see below).
  • Work steadily through the assignment, amplifying and explaining notes you’ve put in the margins and, especially at the end, making more general points.
  • Refer to the assessment criteria.
  • Explain your thought processes as you move towards allocating a mark.
  • Offer a few, reasonably attainable, suggestions for improvement, even if the work is excellent.
  • Invite comments back from the student, including on the method of giving feedback.
  • Round things off in a friendly way.

Bob expands on these in his ‘Practice tips on using digital audio for assessment feedback’

This post is based on material compiled and presented by Jim Sharp and Susi Peacock at Queen Margaret University

• 3 Comments

Share this post on:
| | |
Posted in Assessment, Feedback, Web Apps and tagged on by .

Flickr Tag Error: Call to display photo '2597104631' failed.

Error state follows:

  • stat: fail
  • code: 95
  • message: SSL is required

The use of audio feedback for student assessments is not a new concept. I'm not familiar with all the literature but I've picked up a quote from Rust in 2001 highlighting the advantages of audio feedback:

While reducing the time you spend, this may actually increase rather than reduce the amount of feedback given…Students frequently say that they get far more information from taped comments, including the tone of one’s voice, than they do from written comments, and they also do not have to try to cope with some of our illegible writing. (Rust, 2001: 22)

I picked this quote up from Bob Rotheram, 'Indirector' (Bob explains his title here) of the Sounds Good 2 project.  Bob has been exploring the use of audio feedback for a number of years investigating not only the benefits for students but also finding ways to remove the barriers for teaching staff. There is a very good summary of Bob's work here. While Bob started using the audio recording software Audacity he felt that to achieve greater adoption he needed a simpler solution and moved to digital recorders/dictaphones. 

An alternative 'one button' solution could be to use the new voice features in Google Talk. Google Talk was initially developed as an instant messaging (IM)client, similar to MSN messenger et al., allowing synchronous 'chat' communication between users. As with a lot of other IM clients Google has been adding voice features (similar to Skype) allowing users to make calls to other contacts. A great additional feature of Google's solution is the ability to leave voicemail. The voicemail is recorded in mp3 format (up to 10 minutes in length)and automatically delivered to the recipients inbox as an attachment. To use this feature you have to have a Google account (which is free), but the recipient can have any email address. I've prepared a short video showing how the system works:

Before you go rushing off to try this a couple of drawbacks of this system I've noted are:

  • you have to record your feedback in one take;
  • once you start recording there is no way of stopping your message from ending up in the recipients inbox; and
  • you do not get a copy of your sent voicemail (a way around this would be to use a program like Audacity to record your message as you delivery it)

An interested twist might be to get students to use this voicemail feature to submit assignment, which might be particularly useful for language courses.