Don’t think about how you can ‘badge’ a course, think about the assessment design and what you want to achieve

It’s been interesting to follow Alan Levine’s (@cogdog’s) stroll through yonder as he wrestles with his new Creative Commons certification gig. Part of this has dipped into Open Badges first Seeking Evidence of Badge Evidence followed up by Last Week’s Ride with the Badge Alliance. Having dabbled in badging I increasingly find myself torn between ‘badges are good’ and ‘badges are bad’. Part of the challenge is everything can be done badly and more than often you don’t need to even try. Ale Armellini is particularly concise in his judgement of badges highlighting in his Badges debate – positioning statement:


Supporters of the use of badges usually frame their position on the basis of two primary principles: motivation and recognition of learning.

Regarding motivation Ale goes on to highlight:

Like other rewards (or punishments) they operate on the basis of ‘do this and you’ll get that’, a pop-behaviouristic approach ultimately aimed not at empowering, but at exercising control (see, for example, Kohn, 2000). According to Kohn and the hundreds of research studies referenced in his work, extrinsic motivators such as a gold star, a bribe or a badge, have a detrimental effect on performance, productivity and engagement.

And then we come to the evidence:

The claim that badges are ‘evidence-based’ and ‘verified’ is, at best, very weak. They can be awarded by pretty much ‘anyone’, to represent ‘any’ skills or achievements (see Github, 2015, among many others). Some students have strongly negative emotions towards badges, including ‘dying internally’ at the sight of them (Haaranen et al., 2014). Badges are childish and ‘gimmicky’: badge holders want the badge to ‘look like it was hard work’ (Mewburn et al., 2014). As they’re ‘stackable’, ‘collectable’, and achieving them does not seem to pose any major hurdles, their relative relevance is easily diluted ‘by allowing too many to be issued’ (Glover & Latif, 2013). Attempting to compare badges to formal higher education qualifications indicates a lack of objectivity and understanding.

So aware of the negative consequences why do I find myself getting giddy about badges. Partly I think there is a playful aspect. As negative as the ‘pop-behaviouristic’ approach is the badges I collect usually have a fun aspect about them.

My Stack Overflow badges

I think there are also indirect benefits that badging systems can lend themselves to. I talked/written previously about how we’ve used ‘check-in’ as a way for learners to discover who else is active in the course. Some would perhaps argue that these types of activities undermine the rigour of badging systems. I would argue learners understand that ‘checking-in’ is unlikely to impress an employer and consequently not even feature in a portfolio. So why then would they bother claiming it? In a distributed networked world we have to find other ways of saying hello.

The other aspect of badges I think is often overlooked is it’s an opportunity to revisit the assessment design. Within higher education there is still heavy reliance on end loaded high stakes summative assessment. Back in 2007/08 I was fortunate to work for Professor David Nicol on the Re-Engineering Assessment Practices Project (REAP). David’s work has gone on to inform a number of follow-up projects and other research. As a young learning technologist the following set of assessment design principles proved very useful:

“empower”
  1. Engage students actively in identifying or formulating criteria
  2. Facilitate opportunities for self-assessment and reflection
  3. Deliver feedback that helps students self-correct
  4. Provide opportunities for feedback dialogue (peer and tutor-student)
  5. Encourage positive motivational beliefs and self-esteem
  6. Provide opportunities to apply what is learned in new tasks
  7. Yield information that teachers can use to help shape teaching

“engage”

  1. Capture sufficient study time and effort in and out of class
  2. Distribute students’ effort evenly across topics and weeks.
  3. Engage students in deep not just shallow learning activity
  4. Communicates clear and high expectations to students.

Adapted from Nicol and Macfarlane-Dick (2006) and Gibbs and Simpson (2004)

You could incorporate all these principles in a course design that included some sort of ‘badging’. In particular I would argue the nature of badging should encourage more distributed effort. In turn this should ‘yield information that teachers can use to help shape teaching’ and so on. Back to the playful aspect it’s interesting to see the overlap in the principles above and some of the ideas in game based learning:

For me number 4 is key. Clearly badges ain’t going to be for everyone so they have to be voluntary. So my advice is don’t think about how you can ‘badge’ a course, think about the assessment design and what you want to achieve. 

I tried to convey some of these ideas in a presentation for #openbadgeshe embedded at the end.

5 Comments


  1. Thanks for sharing these ideas, Martin. I am far from having any kind of definitive position, and wonder if I will.

    I’d agree there is a value in seeing say your Stack Overflow profile collection or the screen of activity checkins. My first thought was “are these even badges” or just activity checks, but badges are some kind of representation of micro-achievements, right? They give a quick sense of activity in a way that does not require a lot of drilling down.

    My question is, do these add up to something bigger? There is an assumption that a lot of micro-achievements can indicate larger skill sets. Maybe they do. In school as is, we work through a series of micro activities in a class, whose results add up to a grade in a class, and grades in a class somewhat add up to a degree satisfaction. Is that the degree math?

    I’m curious about the statement “In particular I would argue the nature of badging should encourage more distributed effort.” — are you saying that people will be doing things in disparate places? Or that it’s done in more ways that can be used as indicators? And while activity indicators can help a teacher modify course structure, do they give us enough to know how students are understanding higher level concepts?

    Thanks for the food thought. And I note how many times badging appears in ‘quotes’ ;-)


    1. Stack Overflow is interesting – I rarely target a single badge but it’s nice to see for time to time my contributions add up to something. The issue I guess is it’s hard to unpack what they really mean so while the SO badges are fun I find individually they are meaningless.

      And there are other holes. SO can’t badge everything just as you’d be limited in a class or course. So while you may issue many badges you can never fully represent who that person is. My perspective is rather than starting the question with what can be badge, we ask what is it we’d like someone to learn and as part of that process if that can be captured in a badge lets give people the option to collect that badge.

      Where I think badges could fit well within a course isn’t actually from a learner perspective but as a teacher thinking about the assessment and feedback design. So when I say ‘distributed effort’ it’s in the context of regular opportunities for feedback rather than shoving everything into an exam at the end.


  2. Neil Armstrong did not take a copy of his degree to the moon. He did take two scout badges however and broadcast this fact to the world.


  3. Nope. Badges are *not* and cannot be a tool for formative assessment. Why? cf Black & Wiliam. The defining feature of formative assessment is not when it is given, but where it takes the learner. Badges can tell you how good you did, but formative assessment tells you how to do better. In fact, badges don’t even do that – you know how well you did before you get the badge. So actually, getting a badge tells you (the learner) absolutely nothing you didn’t already know.


    1. Hi Yishay – I agree with your point and whilst a badge by itself isn’t formative the process around awarding a badge can be. For example, the BadgeOS plugin we use allows student and tutor to provide comments on a badge submission and/or award … it comes back to the assessment design

Comments are closed.