Assessment

Last week my colleague, Kenji Lamb, and I were up in Inverness providing some support to the University of the Highlands and Islands (UHI) EDU Team. We were exploring the use/approach to assessment and feedback, sharing what is going on in the sector for the EDU Team to disseminate around UHI. Below are a couple of slide decks I used over the two days.

Having worked on the REAP project a couple of years ago there was a bit of material I recycled from that (as the ripples from this project are still resonating finding there way into publications like Effective Assessment in a Digital Age and workshop/design tools like the JISC funded Viewpoints project) Note to self: must write about Viewpoints once online tool is available 

I also took the opportunity to test drive my idea for a Google Form/Visualization mashup for electronic voting (couldn’t be bothered lugging voting handset up was my excuse ;). Technically it worked reasonable well. One major improvement would be to monitor how many people had voted in real-time.

1 Comment

Previously I’ve promoted the use of audio and video feedback on student work. Methods I’ve highlighted include creating audio and video files using a wide range of software tools and distribution methods. (At this point I would normally direct you to my Student Audio Feedback: What, why and how post but recently rediscovered ALT-C 2009 II: Audio and screen visual feedback to support student learning (and research methodologies), which is pretty good)

Recently a member of staff from one of our supported institutions interested in the use of this form of feedback contacted me with concerns over students reposting personal feedback in the public domain i.e. just as a tutor respects a student’s privacy in not publishing a student work without permission, shouldn’t students do the same. In particular they were wondering if any student declaration was needed to prevent this from happening.

My initial response was along the lines of that any feedback produced by the tutor would remain the intellectual property of the institution and any public reposting would automatically need the consent of the institution, therefore all the tutor needs to do is highlight the existing legal position rather than having students make any extra declarations. But as I wasn’t completely sure of my interpretation of IPR I put a query with JISC Legal and here was the response I got (Disclaimer: The following text is provided as information only and does not constitute formal legal advice):

The recording of the feedback given by the lecturer will either belong to the lecturer or the institution.  S.11(2) of the Copyright, Designs and Patents Act 1988 provides that the employer will be the first owner of copyright, unless there has been an agreement otherwise.  It could be that there is sufficient ‘dramatic’ content in giving the feedback too that there is a performer’s right in the recording too, which would stay with the academic, unless there is agreement to transfer those to the institution.

In any case, the student would need to get permission before doing any of the copyright-restricted acts, which would include copying the work, adapting it, and communicating it to the public by internet dissemination in this particular case.  It may be worth reminding the students of this, and I’d suggest including an explanation that the feedback is personal and given within the teaching relationship, and so dissemination of the work would be disrespectful as well as copyright infringement.  Beyond the legal issue, it might also be worthwhile addressing the underlying reasons why the student or students might want to share the feedback – is there a need for more generic feedback that can be shared more widely?

So generally speaking my guidance was along the right lines, but the information from JISC Legal not only identifies particular nuances of the legal implications but also highlights how the risk of getting into problems can be mitigated and addressing some of the fundamental pedagogy. I hard to see how advice like this could get any better.

This isn’t the first time JISC Legal have provided some first-rate guidance and if you haven’t checked out their service it’s well worth an explore. Before you think this level of support is only available to other JISC Advance and JISC related staff it’s not. JISC Legal endeavour to support anyone in the UK tertiary education sector “to ensure that legal issues do not become a barrier to the adoption and use of new information and communications technologies”.

As well as individual guidance JISC Legal have a wealth of support material. Recent goodies include:

JISC Legal = pure quality btw

RSC-MP3 LogoIt’s been a while since I’ve done a podcast but whilst at the  Enhancement Themes conference last week in between promoting JISC and the Services I managed an impromptu interview with Colin Dalziel from Pebble Learning to ask him about the next version of PebblePad, version 3, which is due out by the end of this year.

Below are links to the recording and a summary of what we talked about (sound quality isn't the best - I only had the internal speaker on my laptop for recording):

[podcast]https://mashe.hawksey.info/wp-content/uploads/2011/pebblepad-interview.mp3[/podcast]
Download Link
Duration: 10 minutes
Size: 7.4 MB

Subscribe to our podacst via RSS Subscribe to our podcast via RSS
Subscribe to our podcast via iTunes Subscribe to our podcast via iTunes

Summary

00:56 - moving towards a personal learning system, tapping into external systems like blogger and twitter, not only pushing assets out to these systems but also pulling material in.

02:20 - talking about the blocks to integrate with different VLEs Moodle, Blackboard and Turnitin

04:00 - highlighting some of the planned features like a social layer to allow PebblePad users connect from around the world; audio and video recording directly within the PebblePad system

06:00 - planning a lightweight version of PebblePad optimised for mobile use then adding more specific functionality for different mobile devices

06:52 - is there an open source future for PebblePad? Not on the horizon, but will continue to work with open standards

07:55 - Alumni and personal PebblePad access - (there is already a personal signup option for PebblePad £20/year)

Share this post on:
| | |
Posted in e-portfolio, RSC-MP3 on by .

4 Comments

On the 28th May 2009  I wrote a post on Generating Student Video Feedback using ScreenToaster. As ScreenToaster is now ‘toast’ I thought I’d repost highlighting screenr instead. As the process for using ScreenToaster/screenr is so similar I haven’t re-recorded the demo video, but hopefully you get the idea (I’m glad I downloaded the original and put it on vimeo ;)

Flickr Tag Error: Call to display photo '4198881540' failed.

Error state follows:

  • stat: fail
  • code: 95
  • message: SSL is required

As I’ve recently revisited on generating audio feedback it seemed timely, particularly with a request from UHI coming into my inbox, to also have another look at video feedback. Russell Stannard recently won a Times Higher Education Award partly for his work in this particular area. In Russell’s work he uses screen capture software to record feedback on electronic submissions of student work. More information on this technique is available in a case study Russell prepared for the Higher Education Academy English Subject Centre on Using Screen Capture Software in Student Feedback. An example of using this technique is also available - click here for a short example of video feedback.

In my original post I highlighted Using Tokbox for Live and Recorded Video Feedback as a possible solution to distribute video feedback. At the time I felt there were two niggling issues with using Tokbox. First there was the requirement to install the ManyCams software to allow you to display your desktop and secondly Tokbox was very slow in uploading video you had recorded. For live video feedback Tokbox might still be worth considering, but shortly after publishing the post I discovered ScreenToaster., but for recorded feedback you might do better with screenr.

ScreenToaster Screenr allows you to record your desktop without installing any software. It’s very easy to setup and the videos you create can be immediately uploaded allowing you to decides how you want to distribute and share them [You can also publish them directly to YouTube and/or download the video in MP4 format. The following video shows you how easy it is to setup and highlights some of the useful features. Even if you are not interested in delivering video feedback to students this is still a great site to record other material like demonstrations of software.

ScreenToaster Screencast 
Example of using ScreenToaster to deliver video feedback on student submitted work from Martin Hawksey on Vimeo

3 Comments

Electronic voting (clickers) in the classroom have been used to engage students in learning for a number of years. As well as potentially being a fun way to learn there are great opportunities for tutors to identify misconceptions in knowledge and ‘close the gap’.  However, one issue with many voting systems is that results often get lost in the vendor's voting software. Although a number provide ways to bridge the gap with plugins it can often be problematic to synchronise user details. 

One manufacture, Optivote, has taken a different approach, to integrate their voting solution into Moodle. Using the Moodle Quizzes module instructors can author their questions and pose them to their class, collecting responses via the Optivote handset, the results being collated for presentation back to the class and stored in Moodle. The video below demonstrates Optivote:

If you are a Moodle user and you would like to try the Optivote system they have a partnership programme. In return for helping Optivote refine their solution you will receive £3,000 worth of kit and support. This includes:

  • 32 Optivote handsets
  • The Optivote/Moodle plugin
  • Full training for 10 members of staff
  • Three years support and maintenance

Deadlines for applications are the 20th Feb.

Click here for more details about the Optivote Partner Programme

3 Comments

First I should say I don’t think this is the best solution, in fact an earlier post from 2008 DIY: A wi-fi student response system is probably a solution, if perhaps needing more tidying up, but I’m posting anyway just on the of chance that this might inspire or help solve someone else's problem.

This post has come about as a result of a couple of things:

  1. I’m in a bit of a Google Apps run.
  2. I read and enjoyed Donald Clarks Clickers: mobile technology that will work in classes
  3. I saw and contributed to Tom Barrett’s crowdsourced X Interesting Ways to Use Google Forms in the Classroom (I added #39 Collaboratively building a timeline which I discovered through Derek Bruff’s class example.

Concept: Using Google Forms as an interface for a mobile/desktop based voting system.

Issue: If you are asking multiple questions the form needs prepopulating with options making it a long list for the student to navigate and potentially creating a predefined route of questions.

Solution: Use a generic form with one set of response options, the class using a unique question identifier for response graphs to be generated from.

The finished result

Below (if you aren’t reading this via RSS) is this Google Form. [You can make a copy of the related Spreadsheet and customise the text and options. For example, you might want to turn qID into a list option rather than free text.]

Loading...

And here is a page with a summary of responses, which allows the user to choose which response set to display (screenshot shown below):

Screenshoot of summary of responses

How it was done

Some of you might already be familiar with Google Chart. This service allows you to create chart images by encoding the data in the URL. I’ve used this service in a number of my mashups, in fact all of my previous voting mashups use it in some way, and not surprisingly in Generating charts from accessible data tables using the Google Charts API.

Google Chart works well if it easy for you to get the data and format it for the image URL. For more complex tasks there is Google Visualization. The advantage of Visualization is it gives you a way of querying a data source before displaying as a table or chart. To see what you can do (and the place where I aped most of the code for this mashup) you should visit the interactive gallery of Visualization examples.

Using the Using The Query Language example as a stating point I could see you could lookup data from a Google Spreadsheet and filter the response using Google Visualization API Query Language, which is very similar to SQL. What I wanted to do was SELECT the data from the spreadsheet WHERE it matched a question identifier and COUNT the number of occurrences for each GROUP of response options. An extract from the table of data is:

A B C
Timestamp qID Answer
- q1 A
- q1 B
- q1 A

My attempts to convert the SQL version of this query which is something like:

SELECT C, Count(C) AS CountOfC WHERE B = ‘questionID’ GROUP BY C

initially I was left with keyboard shaped indentations on my forehead trying to get this to work but Tony Hirst (@psychmedia) was able to end my frustration with this tweet. This meant I was able to use the following query VQL friendly:

SELECT C, Count(B) WHERE B = ‘questionID’ GROUP BY C

The next part of the problem was how to let the user decide which question ID they wanted to graph. Looking at the Simple Visualization example I could see it would be easy to iterate across the returned data and push out some html using JavaScript. What I wanted to do was GROUP the questionID’s and COUNT the number of responses, which is possible using:

SELECT B, Count(C) GROUP BY B

This returns a table of unique question IDs and a separate column with a count of the number of responses. A form list element is populated with the results using:

for (i=0; i
var ansText = data.getValue(i, 0)+' (No of votes '+data.getValue(i, 1)+')';
var valText = data.getValue(i, 0);
ansSet.options[ansSet.options.length]=new Option(ansText,valText);
}

And that’s it. If you want to play around with this the code is here. Enjoy and if you find this idea useful or if you spot any issues as always I value your comments.

7 Comments

Flickr Tag Error: Call to display photo '130164091' failed.

Error state follows:

  • stat: fail
  • code: 95
  • message: SSL is required

Previously I written about Using a Learning Apps (xLearn) textwall for SMS voting for £25/year, but what if you haven’t got £25 to spare? How about free SMS voting*, and when I say free, I don’t mean free for the first 15 votes like SMSPOLL.net or free for the first 30 votes like PollEverywhere.com, I mean free for as many responses and polls you like!

*excluding the price to send a txt msg

I’ve been think about free SMS voting for quite a while, 4 years in fact! Back in 2006 one of the first blogs I regularly read was David Muir’s EdCompBlog. At the time I worked at the University of Strathclyde in CAPLE and David was in the Faculty of Education. His blog was great to find out what was going on at the other end of the institution, something Brian Kelly regularly highlights.

In October 2006 David posted his experiences on Moblogging: Turn it on again where he was able to mash a free SMS textwall using intelliSoftware SMS gateway. At the time I left a comment asking if David had:

thought about parsing the text messages for voting? i.e. students text 'pgdes2blog Q1B' to answer B in MCQ for question 1 etc? (Anonymously said …)

As it happened David had but neither of us was in the position to come up with a solution back then. Roll forward 4 years (with a Twitter voting solution inspired by David in between) and the old grey cells get a jump start after David posted some reflection on his student induction 2010 style in What did they need to know?. David mentioned he used his free textwall solution again collecting responses on this blog.

Both of us realised that if David was collecting responses on a blog that it would be easy to reuse my earlier Learning Apps solution to grab and parse the responses (using RSS). In fact it was so easy all I needed to do was change one line of code.

So below is an alternate version of XVS – SMS voting using Learning Apps:

*** RSSvs – SMS voting using RSS ***

With this version you can submit any RSS feed and it will extract/graph the number of occurrences of an answer option after a question identifier in the post title. Here is an example of a response chart which is generated from this test blog

So potentially you could use anything for voting which somehow creates results as an RSS feed. But how can you use this for SMS voting?

How to use RSSvs with intelliSoftware  

Unlike the Learning Apps textwall it doesn't have a native RSS feed for the SMS inbox, but as David has already demonstrated it is possible to automatically forward messages sent to intelliSoftware as an email which can then be used to publish a blog post. This is possible because a number of blogging platforms allow you to create posts from emails (e.g. Blogger: How do I post via email?). Here is how to setup your intelliSoftware account:

  1. Create a blogger account and enable mail-to-blogger (taking a note of your personal mail-to-blogger address)
  2. Usual form filling. Important: Username will be your message identifier i.e. students have to start their response txt with your username so keep it short and meaningful
  3. Once registered login and select ‘Preferences’ in ‘My Account’
  4. In the ‘Forwarding’ tab enable ‘incoming message forwarding’, choosing forwarding type email and entering your mail-to-blogger address.
  5. In the Advanced Settings for this you can also modify the email template. Important: Make sure [Message_Text] is included at the end of the ‘Email Subject’, you should also remove [Message_From_Number] to prevent students mobile numbers being published.

Collecting and displaying responses

When you want to ask a question give users the options and instructions like “to vote for option ‘A’ send a text message to 07786 XXX XXX with ‘xyz #q1 A’ (where 07786 XXX XXX is the mobile number found in the Trial Service section and xyz is your username created with intelliSoftware).

The question identifier (in this example #q1) can be anything you like as long as it starts with ‘#’ and the options can be anything you like (a, b, c … 1, 2, 3 … etc).

To display a response graph visit the  RSSvs Site and enter the rss feed for the blog you are collecting responses on and the question identifier.

Important Tip: If you are using Blogger Blogspot you can increase the number of items returned by adding &max-results={and a number}. For example: http://rschetest.blogspot.com/feeds/posts/default?alt=rss&max-results=200 

Once the form is submitted you can swap between the live results and a static chart. (the url of this page can be included in PowerPoint slides allowing you to link directly to the results) Below is the format it uses:

https://mashe.hawksey.info/twevs/rssVS.php?id={see note}&tag=q1&options=-&type=live

idis an encoded version of your RSS feed url.The encoded id is fixed so can be reused

tag – your question identifier

options – optional number to restrict the number of options displayed

type – setting to ‘live’ displays the chart with realtime updates. Leaving blank displays the static chart

 

As intelliSoftware have been providing their SMS forwarding service for free since 2006 I would encourage you to have a look at some of there paid for services. Lots of developer tools to look at and the Multimedia Messaging Service MMS looks interesting too.

Flickr Tag Error: Call to display photo '3680135473' failed.

Error state follows:

  • stat: fail
  • code: 95
  • message: SSL is required

Probably the most unlikely blog post title ever, let me expand.

On Friday we had our ePortfolio Scotland 2010 conference at Queen Margaret University. The final presentation on the day was by Dr Gordon Joyce entitled ‘JISC Effective Practice with e-Portfolios – Where are we now?’. As well as giving an overview/update of the JISC ePortfolio programme he introduced the JISC funded e-Portfolio Implementations Study (ePI) which is investigating, analysing and documenting how intuitions go about large-scale portfolio implementation.

The thing that caught my attention was the framework they were using to capture this information, ‘Threshold Concepts’.

Threshold Concepts' may be considered to be "akin to passing through a portal" or "conceptual gateway" that opens up "previously inaccessible way[s] of thinking about something" (Meyer and Land, 2003).

This wasn’t the first time I’ve come across this theory (having worked with Ray Land at CAPLE), but it was the first time I heard it being used as a analysis framework. I’ve embedded Gordon’s presentation below, he starts talking about Threshold Concepts and ePI from slide 15, so you can find out how they are using this framework.

JISC Effective Practice with e-Portfolios – Where are we now?

At almost exactly the same time as Gordon was giving his presentation John Robertson from JISC CETIS published a blog post on Threshold Concepts And Open Educational Resources in which he:

considers the possible possible application of threshold concepts to open educational resources and the conceptual challenges faced by those advocating the use and release of OERs

Now what are the chances of that! Or is it just a case of ‘Morphic Resonance’?

Morphic Resonance I hear you ask. I first came across this concept on BBC Radio 4’s ‘Museum of Curiosity’. The basic idea, as I understand it, is that ideas can be shared without contact, a bit like collective unconscious. An example regularly cited is the observation that sheep in part of Australia discovered they could cross cattlegrids by rolling across them. At the same time thousands of miles away the same behaviour was witnessed in a different flock of sheep, morphic resonance.

So if you are putting a JISC bid in I recommend you reference Threshold Concepts ;-)

Last Friday I ran a Twitter workshop as part of ‘eAssessment Scotland 2010: Marking the decade’. In the programme I described the session as:

What’s happening? Twitter for Assessment, Feedback and Communication

Twitter is a social networking site which continues to divide personal opinion. Some believe that the micro-blogging service is just an opportunity celebrities to boost their ego with millions of followers or just full of people ‘tweeting’ what they had for lunch. Whilst some users do use Twitter for this purpose a number of academics are now discovering that Twitter has the potential to support teaching and learning, providing a means to enable students to discuss and share within their personal learning network. Before you dismiss Twitter there are some basics worth considering: the service is free to register, status updates can be made from the most basic mobile phone, and users can monitor conversations through multiple means including, for some users, sending free SMS updates.

This workshop uses some of the features of Twitter highlighted above to let participants experience and use this service as a free electronic voting system (EVS), for classroom administration (assessment notification/reminders) and to monitor real-time student evaluation. As this technology is relatively new the workshop will begin with an overview of basic Twitter interaction making it suitable for novice and expert users.

I was perhaps too ambitious to include ‘novice’ users and it would have been better if I either focused on beginners or intermediate/advance users. The workshop I delivered was probably more beneficial for the later, but hopefully novice users were tantalised by the utility of twitter.

During the workshop I really missed having Timo Elliott’s PowerPoint AutoTweet tool (which is broken because of authentication changes at Twitter). This would have been really useful to send out links during the presentation (this example shows how I used it for another presentation).

As a number of participants had only just created Twitter accounts the week before it looks like Twitter quarantines their tweets preventing them from appearing in search results (I guess they wait until you hit some threshold in terms of following/followers/tweets to make sure the account isn’t being used from spam).

In the sides you’ll noticed I’ve revived the Twitter voting tool (TwEVS). Previously this solution relied on using Yahoo Pipes to manipulate results from Twitter, which meant graphs didn’t always have the latest results because of caching. To get around this I’ve created a script which can be run from a webserver. Here is the new TwEVS interface (you can also download the code).

11 Comments

I’ve written about the different ways you can do electronic voting without buying clickers a number of times from creating a simple wi-fi system, to using services like polleverywhere.com, to even using Twitter (more on the latest on this one in a separate post).

For the ‘eAssessment Scotland 2010: Marking the decade’ conference we ran a poster competition and not wanting to collect lots of slips of papers we thought it would be good to have a SMS vote. Having seen the Learning Apps (formerly xlearn textwall) being used at other events and knowing it allowed data to be exported via RSS it was the ideal candidate. Using the same concept for voting via Twitter (TwEVS) of counting the occurrences of options after a hashtag it was easy to just substitute the feed from Twitter search with the one from Learning Apps.

Wanting to add a bit more than just a static Google Chart I was interested to see if I could get the graph to update automatically without browser refresh. After looking at a couple of options including the Javascript plotting library ‘flot’ I came across a post by Sony Arianto Kurniawan on Create Realtime Chart Without Page Refresh using FusionCharts Free and Ajax (prototype.js), which worked a treat.

The advantage of this home grown solution is it gives you some flexibility in how it is used in particular using the space before the question identifier for users to explain why they think their answer is correct. You can access the voting site using the link below (here is also the source code for download).

*** XVS – SMS voting using Learning Apps ***

Instructions

  1. Rent a textwall from Learning Apps (xlearn) for £25/year (this solution only requires you to receive messages so you won’t need any additional credit unless you plan on contacting students via SMS)
  2. Once created login to the xlearn admin panel and click either ‘Text Wall’ or ‘Inbox’ and note/copy the code after 'http://xdalearn.co.uk/rssfeed/Feed?id= (might be 12 random characters) Update: There's a new url so it's the code after http://www.textwall.co.uk/rssfeed/textwall/
  3. When you want to ask a question give users the options and instructions like “to vote for option ‘A’ send a text message to 07XXX XXX XXX with ‘xyz #q1 A’ (where 07XXX XXX XXX is the mobile number and xyz is the short code provided by Learning Apps).The question identifier (in this example #q1) can be anything you like as long as it starts with ‘#’ and the options can be anything you like (a, b, c … 1, 2, 3 … etc)
  4. On the XVS site enter your textwall RSS id saved earlier and the hashtag identifier without the ‘#’ (in this example it would be ‘q1’). You can also optionally set the maximum number of options to graph. The reason you would use this is to try and prevent any malicious uses like sending rude messages.
  5. Once the form is submitted you can swap between the live results and a static chart. (the url of this page can be included in PowerPoint slides allowing you to link directly to the results) Below is the format it uses:

https://mashe.hawksey.info/twevs/xvs.php?id={see note}&tag=q1&options=-&type=live

id - is an encode version of your textwall RSS id. It’s encode to try and prevent direct access to you entire text wall. The encoded id is fixed so can be reused

tag – your question identifier

options – optional number to restrict the number of options displayed

type – setting to ‘live’ displays the chart with realtime updates. Leaving blank displays the static chart

One last thought. As this solution uses RSS feeds to pull the voting results, just as with the Twitter voting example, it would be very straight forward to combine the two (already a feature of polleverywhere.com, but something I’m not interested in doing).