Archive for the 'Google Apps Script' Category

In Notes on technology behind cMOOCs: Show me your aggregation architecture and I’ll show you mine I reached the point in my own mind that the key behind cMOOCs was how you aggregated and shared dispersed activity. At the time I also asked “Given the widespread use of Twitter in MOOCs are there tools/techniques required to aggregate and disseminate the course discussions?” and started looking at techniques to retrospectively analysis Twitter based discussions.  This activity hasn’t gone unnoticed and I was very grateful to be asked by Dave Cormier and George Siemens to do a weekly summary of Twitter data from their latest course Current/Future State of Higher Education (CFHE12) which started this week. This will be outwith my official CETIS work but given the increasing number of enquiries we are getting in this area it will undoubtedly feed in.

As I’ll be reporting on this course it made sense to sign-up. On one of the registration pages I noticed a couple of different hashtags left over from earlier course so asked the question:

So which hashtag is it #CFHE12, #Edfuture or #oped12? This page bit.ly/Rq6UZE seems to indicate #edfuture = posts, #oped12 = course

— Martin Hawksey (@mhawksey) October 8, 2012

If you visit the Twitter status page for this tweet you’ll see I got a couple of responses from AJCann and Jez Cope. If I had not sent you to that page how would have you known I got an answer? Did Jez know that Alan had already responded to me?

Given this type of dialogue, but at a higher level is a key aspect of learning and many a Greek has dined out on ‘knowing that they know nothing’ and started wondering how could this activity be aggregated and would this aggregation increase the situational awareness of participants and cause a shift in how the course community interacted with each other (I had recently read Tony Hirst’s post on Conference Situational Awareness and the example from the “London 2012 Olympic Games where it was identified that tweets relating to the congestion of the Olympic park entrances had a direct effect on crowd flow through the site” was still on my mind.

So after some late night code bashing here’s what I’ve come up with (this is very beta so your feedback is welcome – particularly if it doesn’t work). A Filtered Aggregation of #CFHE12 questions and responses (embedded below if you are viewing this post on my site):

What you have here is an aggregation of possible questions from #cfhe12 with buttons to filter for messages with and without replies. Because it’s linked to Twitter’s own embed code users can do the usual Twitter actions (reply, retweet etc). As noted there are some limitations perhaps the biggest is it isn’t 100% reliable in that I’ve got no way to include replies made without the #cfhe12 hashtag … in this version anyway.

I’ll let you go and play with and hopefully you’ll share your thoughts. Two things that spring to mind for me are: it would be nice if this page had RSS feeds just to keep the aggregation juices flowing; and wouldn’t it be interesting to use tweet favouriting to let the community curate questions/answers, a favourite representing an upvote (see Techniques for Live Tweet Curation)

Make your own

*** Open and copy TAGS v3.1Q ***

Run through the Basic and Advanced setup used in the TAGS v3.1 (you need to authenticate with Twitter).

In the spreadsheet open Tools > Script editor and follow the ‘To use Filter Questions Interface’ instructions

Upgrading an existing TAGS v3.1+ Archive

1. Open and copy TAGS v3.1Q and click on the ‘questionsFilter’ sheet active.
2. Activate the sheet tab menu and chose ‘Copy to…’.
3. Now find your existing TAGS archive spreadsheet and copy.
4. Once it has copied open the destination and rename the new sheet from ‘Copy of questionsFilter’ to questionsFilter
5. Open Tools > Script editor… in your old archive and select New > File > Script file. Call the new file TAGSExtras
6. In the new script tab copy and paste the code from here, then save
7. Run > setup twice (first time to authorise, second to fun the function)
8. File > Manage Versions and enter any description you like and Save New Version
9. Publish > Deploy as web app… and click Update
10. Run > getUrl and then open View > Logs… and copy the url into your browser address bar to view the result

How it was made (Non-techies you are free to leave ;)

The starting point was Twitter Archiving Google Spreadsheet TAGS v3. A hidden feature of this is to add a column to you Archive sheet called ‘possible_question’. When the archive collects tweets it looks for the text ‘? ‘ or ‘?’ at the end to identify the tweets might be a question and if so ‘TRUE’ is put in the archive column.

Having got a list of potential questions and associated tweet ids I could have put them in my failed lab experiment (and unfortunately titled) SpreadEmbed, but noticed that the embed.ly api doesn’t return a in-reply-to message with it embed code. To expand upon, because this is quite important, currently when you embed a tweet which is in reply you use something like this:

  
@mhawksey Most of us are using #cfhe12 ?
  — AJCann (@AJCann) October 8, 2012

Although this text doesn’t include the text of the message it is replying to Twitter clever bit of javascript renders it like this:

re-writing our little <blockquote> as:

  

    

      
   Martin Hawksey @mhawksey     8 Oct 12    
      

        
So which hashtag is it #CFHE12, #Edfuture or #oped12? This page bit.ly/Rq6UZE seems to indicate #edfuture = posts, #oped12 = course
      
      

      

    
    

      
   AJCann @AJCann 
        
      
      

        
@mhawksey Most of us are using #cfhe12 ?
      
      
   8 Oct 12 
        

          
Reply
          
Retweet
          
Favorite
        
      
    
  

Now you know why the page takes so long to render ;)

With this extra data we can use jQuery to find and filter tweets that have the class ‘twt-reply’.

To recap using TAGS we can identify tweets that might be questions and using a Twitter embed we can also automatically get the message it is in reply to. So to display a question and answer together we only need to find the answer and Twitter will render the question it is in reply to (still with me). The problem we’ve got is we can easily filter for questions (possible_question == TRUE) but not the answer. To do this I create a sheet of all the tweet id_strings that are questions (=QUERY(Archive!A:N,"select A WHERE N is not null LIMIT 50",FALSE))  and another where we know the tweet is in reply to something (=QUERY(Archive!A:N,"select A, K WHERE K starts with '2' LIMIT 50",FALSE)) . For the last bit I need to write some Google Apps Script which replaced any question tweet ids with the answer id, which gives us the ‘Combination of Qs and As’ column.

To render the tweets on a page we need to get the embed snippet using Twitter’s official oembed endpoint. Because getting the embed code need authenticated access I again used Google Apps Script to fetch this data and cache the result. Using Apps Script ContentService I can expose this by publishing the spreadsheet as a web app and serving up each tweets embed code in JSONP. For example here’s the JSONP wrapped embed code for #CFHE12. The last part of the puzzle is some good old fashioned HTML/JavaScript which renders the Twitter embed code and adds some UI (the code is here).