Photo credit : Chris Bull CC-BY-NC-SA ALT https://flic.kr/p/xx86UaLast week was #altc, once of the largest annual edtech events in the UK calendar. Each year we’ve seen a growth in the use of Twitter at the event and it makes sense for us to have the live Twitter stream displayed throughout the venue. In previous years we’ve relied on 3rd party services but this year I wanted to bring this in-house.  Having looked around I came across a nice example by Remy Sharp for a ‘A conference twitter wall with built in schedule & announcements’. This looked ideal as the schedule and announcements was a nice additional feature, plus it was coded in HTML/Javascript making it easier for me to modify.

Iteration 1 - Happy days…

If you are familiar with the nuances of the Twitter Search API you’ll know everything needs authenticated access. In Remy’s example he got around this by proxying the calls via a local NodeJS server. I don’t have a Node server so my initial thought was to use the TwtrService Apps Script library to make the calls and host the files on Google Drive (if you are unfamiliar with Google Drive Hosting you can publish a folder with HTML/JS and it’ll render as a webpage … but see the ‘but’ below).  This is basically the same recipe for the solution for keeping your official Twitter archive fresh on Google Drive. For this solution I could have created a Google Apps Script Web App (Ref: read more) to expose the data but as we were planning on multiple walls and with Apps Script quotas to bear in mind it looked better to use Remy’s debug option which reads copies of the data. Actually getting Remy’s code to work hosted on Google Drive takes very little modification (I’m claiming 4 lines), to write the data to Drive takes a bit more.

The result of this iteration can be seen at:


… and all the files for this are in this Google Drive folder:



Before I go into the details of setting this up there is a ‘but’. I initially did all this work in August in advance of our event planning to publish my work for others to use. But then Google announced they were Deprecating web hosting support in Google Drive. As per Google policy this means web hosting will be available until August 31st, 2016 so if you have an event running before then here’s how you can set this up.

  1. Create a new folder in Google Drive and note the folder id from the url:
    Getting Google Drive Folder ID
  2. Download my fork of the Twitterwall, extract the .zip archive
  3. Open config.js and edit the baseUrl so that it ends with your folder ID from step 1 Important: keep the rest of the baseUrl e.g. baseUrl: 'https://googledrive.com/host/YOUR-FOLDER-ID/' and your search term.
  4. Upload all the folders and files to the Google Drive folder you created maintaining the structure

At this point visiting your baseUrl should render the wall, next setting up the Twitter data collection.

  1. Open this Apps Script file and File > Make a copy
  2. In your copy  edit the FOLDERID in the script to match the folder id of the hosted twitterwall files (part 1, step 1).
  3. If you’ve never used TAGS/TwtrService enable Twitter API access via this web app.
  4. Back in the Apps Script Editor window Run > getSearchResults and authenticate to test.
  5. To setup regular collection of data from the Script Editor select Resources > Current project’s triggers and add a time-driven trigger to updated every 5 minutes:
    Setting up script triggers

Now every 5 minutes this script will get new data from the Twitter Search API and dump in the the /history/data folder. Also every 5 minutes Remy's script will check this folder for new data and if it's there render the tweets. You can also modify the index.html file in your folder to add your schedule and notices as per Remy’s README.md.

If you are not familiar with Google Apps Script you are probably wondering ‘what magic is this’ and you might want to have a wee peek at the documentation to see what else is possible.

In my next post I’ll explain how we can enhance the Twitterwall to use a Google Sheet for the notice and schedule information as well as how this application can be run without Google Drive web hosting. In the meantime please use the comments if you get stuck or need part of this explained more.

In a couple of weeks I’ll be talking about TAGS at OER15 (14:30-15:00, 15th April). Whilst parallel sessions aren’t going to be streamed I’ve got a couple of ideas for broadcasting my session. If I pull this off I’ll be co-tagging my presentation #oer15 #740.

My notes for structuring the session so far is:

  • Networks – shape, strength and characteristics
  • Networked education – making the connection
  • Footprints – seeing the connection
  • Twitter in Ed – activities Adoption Matrix (Mark Sample) and examples
    ”anyone to become an active participant in the conversation” Ross, 2012.
  • APIs – me speaky code
  • Twitter Search and the Twitter Search API
  • Anatomy of a tweet
  • TAGS/TAGSExplorer
  • TAGS in the wild
  • Context – SNA, ethics, vulnerability

You’ll see I’ve highlighted two items in that list and this is where you can help. If you have used TAGS to support your class/course I’d like to know:

  1. how does Twitter generally fit in to the course? Are you using directed activities or is there a more organically;
  2. how TAGS is used to support this? Post/ongoing SNA, situational awareness, …

As the session will mostly be me talking, preferably some video or audio would be great. If you’ve previously talked about Twitter/TAGS in education and a recording is available I’ll happily look at this and see if there is something I can use.


1 Comment

I previously posted about TwtrService: A Twitter API client library for Google Apps Script which makes it easy to interact with Twitter from Google Drive applications like Google Sheets. One of the nice things about TwtrService is that once you setup a connection to Twitter you can use it many times in different projects, basically allowing you to do stuff in one line of code. In the post I said I’d share some of the examples of things I make so here is the first one, EasyTweetSheet.

What it does

At the Association for Learning Technology we organise lots of events. We only have a small staff team so having someone sending out tweets during the event can be a problem. We could use a Twitter client like Tweetdeck or Hootsuite  to schedule tweets during the day. One issue is if something goes wrong like the livestream not working or a session starting late you can look a bit silly. The solution was to draft our tweets in a Google Sheet and have a link we click when we want the message to be sent. Below is a screenshot for the one we used at this year’s ALT Annual Conference:

EasyTweetSheet used at #altc 2014

How to get you own copy working

  1. Open this copy of the EasyTweetSheet templateand File > Make a copy
  2. In your copy open Tools > Script editor and follow the instructions
  3. Start filling the ‘text’ column with what you want to tweet which should enable the ‘tweet’ link

IMPORANT: If you’ve used my other tools like TAGS this template will use the Twitter account you used to set it up. To use a different Twitter account to send the tweets from
replace YOUR_CONSUMER_KEY and YOUR_CONSUMER_SECRET in lines 34-35 of the Script editor code with your Twitter application key/secret. When you Run > setup switch back to the Sheet view and follow the instructions.



As part of the latest release of TAGS (Twitter Archiving Google Sheet) I moved a lot of the code into a Google Apps Script Library. Libraries are a great way to bundle and release code allowing you to write your own classes and methods. To see some of the functionality already being developed by the Apps Script community you should have a look at the Google Apps Script Samples site and the excellent work Bruce McPherson has done which includes a EzyOauth2 library.

One of the things you can do with libraries is wrap one library into another. When rewriting TAGS it made sense to strip out a separate Twitter client library that I and others could use in different projects. Based on the work by Arun Nagarajan at Google, TwtrService provides  access to Twitter's REST API. The work I’ve done is to add some UI and configuration methods to try to streamline the authentication flow. As part of this developers can offer authentication routes using their own Twitter application or use an application created by users. This particular angle is a result of one of the design principles for TAGS, that every copy of the template should use a Twitter application owned by the user. The reason behind this is to distribute the risk. If Twitter were to suspend my data access because a TAGS user abused their API it would suspend access for all TAGS users. By requiring TAGS users to register their own application with Twitter the responsibility to abide by Twitter’s terms of service lies with them. So in TAGS the auth flow looks like this

The result is hopefully a flexible library that developers can integrate into their own projects or by getting users to register their own.

Over the next couple of weeks I'll be sharing some examples applications we've developed at ALT. In the meantime this post serves as a basic introduction to TwtrService and covers:

Overview of TwtrService

The TwtrService library for Google Apps Script centrally stores your Twitter access details allowing them to accessed from multiple script projects without the need for re-authentication. TwtrService is designed to allow you to directly use the Twitter’s v1.1 REST API GET and POST methods. For example to return Twitter search results for the search ‘Google Apps Script’ you would use:

var data = TwtrService.get('https://api.twitter.com/1.1/search/tweets.json', {q: 'Google Apps Script'});

The url string can also be abbreviated to:

var data = TwtrService.get('search/tweets', {q: 'Google Apps Script'});

Additional options can be passed in the parameters array. For example to return 100 results for the same search you would use:

var data = TwtrService.get('search/tweets', {q: 'Google Apps Script', count: 100});

The project key for this library is MarIlVOhstkJA6QjPgCWAHIq9hSqx7jwh and the TwtrService methods are documented here.

To use the Twitter REST methods TwtrService first needs authenticated access. TwtrService has some built-in methods to do this detailed below. Once a user has authenticated access the TwtrService library stores these as User Properties. This means when a user has authenticated once with TwtrService using the library in another container-bound or standalone Apps Script immediately gives them access to Twitter API results using the get/post methods. In terms of security User Properties are limited to the effective user of the current script.


Quick start: Personal access

If you would just like to use TwtrService for your Apps Script projects the easiest way to get started is to register a Twitter application and enter it’s details on this page (if you are interested here is the source code for the interface).

Note: If you are already a user of TAGS you’ll already be able to use TwtrService without the step above.

In your Apps Script project you’d like to use the Twitter API in the Script Editor window use Resources > Libraries and add the service  using the project key MarIlVOhstkJA6QjPgCWAHIq9hSqx7jwh.

In your project you can now use the TwtrService.get() and TwtrService.post() methods. The documentation for get() is detailed below (post is the same but uses HTTP POST):

get(string url, Object parameters)

GET wrapper for request to the Twitter REST API. For full documentation of API method endpoints and parameters see https://dev.twitter.com/rest/public. For example to get last 100 tweets containing 'Google Apps Script': var data = TwtrService.get('search/tweets', {q: 'Google Apps Script', count: 100});

urlstringTwitter REST API resource url. This can be either long form e.g. https://api.twitter.com/1.1/search/tweets.json or abbreviated e.g. search/tweets
parametersObjectadditional API parameters as detailed in the Twitter REST API documentation e.g. for search results a search string and count is specified by {q: 'Google Apps Script', count: 100}.
Return Values:
ObjectAPI response in JSON format.

Quick start: Personal access in Sheets and Documents

If you would like to replicate the TAGS authentication flow where users enter their Twitter application key/secret TwtrService comes with a number of UI methods. For TAGS the following code is used:

* Launches key/secret and auth flow
function setup() {
  if (TwtrService.isUserConnectedToTwitter()){
   var result = Browser.msgBox("Twitter Authorisation", 
                   "You appear to already be connected to Twitter.\\n\\nWould you like to run the setup again?", 
    // Process the user's response.
    if (result == 'yes') {
      // User clicked "Yes".
  } else {

* Used as part of setup() to process form data
function processForm(formObject) {

Quick Start: Shared Twitter Key/Secret

The earlier examples have assumed the user registers their own Twitter application. For scenarios where you would like to have the option for users to have authenticated access using a dedicated Twitter API key/secret it is possible to initialize these values. An example application code can be found here which is also deployed here.

Similar to earlier examples once a user authenticates access with your key/secret as long as these values are also initialized in other script projects the user will have access to the Twitter API via TwtrService.

Instructions for creating a Twitter Application to use with TwtrService

TwtrService requires you to have a registered Twitter application. If you are If you haven’t already done this here are some steps you need to get started:

  1. Register for an API key with Twitter at https://dev.twitter.com/apps/new (if you've already registered for a TAGS sheet you can reuse your existing API Key and Secret).
    • Name, description and website can be anything you like
    • Important Include the Callback URL https://script.google.com/macros/
  2. Read the 'Developer Rules of the Road' before clicking 'Create your Twitter application'

On the Twitter site your application should include a ‘Keys and Access Tokens’ tab which includes the Consumer Key (API Key) and Consumer Secret (API Secret) you’ll need.



Amazing to think TAGS has been going for over 4 years now. Version 6 is a major code rewrite and the most important new feature is a better setup processes. No more digging into the script editor to run functions, no more entering your Twitter API key and secret each time you create a new archive. Both these things are history thanks to some base code released by Arun Nagarajan at Google. Now you enter/register for a Twitter API key and secret once and each copy of TAGS you make will remember these. This is made possible by incorporating this functionality as a custom Apps Script library I’ve called TwtrService. TwtrService makes it easy to make calls to all of Twitter’s API and I’ll be explaining how it works in another post.

Version 6 comes with some other new features. The one that was most requested was archiving favourited tweets. There was a quick hack version of TAGS that did this but was limited to the last 200. Now when you setup a favourite archive you can get up to the last 3,000 favourited tweets. Another option with TAGS v6.0 is to use Google’s new version of Sheets. This gives TAGS more capacity and performance. One issue however with new Sheets is it isn’t very stable with the Google Visualisation API which is used in TAGSExplorer.

With TAGS v6.0 I’ve also created a dedicated support site. So if you have any questions or need help head over to http://tags.hawksey.info where you can also get the latest version of TAGS.

1 Comment

Twitter has recently frustrated a number of developers and mashup artists moving to tighter restrictions on it’s latest API. Top of the list for many are all Twitter Search API requests need to be authenticated (you can’t just grab and run, a request has to be via a Twitter account), removal of XML/Atom feeds and reduced rate limits. There are some gains which don’t appear to be widely written about so I’ll share here

#1 Get the last 18,000 tweets instead of 1,500

Reading over the notes for the latest release discussion/notes for NodeXL I spotted that

you now specify how many tweets you want to get from Twitter, up to a maximum of 18,000 tweets

Previously in the old API the hard limits were 1,500 tweets from the last 7 days. This meant of you requested a very popular search term you’d only get the last 1,500 tweets making any tweets made earlier in the day inaccessible. In the new API there is still the ‘last 7 days’ limit but you can page back a lot further. Because the API limits to 100 tweets per call and 180 calls per hour this means you could potentially get 18,000 tweets in one hit. If you cache the maximum tweet id, wait an hour for the rate limit to refresh you could theoretically get even more (I’ve removed the 1.5k limit in TAGSv5.0, but haven’t fully tested how much of the 18k you can get before hit by script timeouts).

#2 Increased metadata with a tweet

Below is an illustration of the data returned in a single search result comparing the old and new search API.

Old and new Search API responses

If you look at the old data and the new data the main addition is a lot more profile data. A lot of this isn’t of huge interest (unless you wanted to do a colour analysis of profile colours), but there is some useful stuff. For example in this example I have profile information for the original and retweeter. as well as friend/follower counts, location and more (I’ve already shown how you can combine this data with Google Analytics for comparative analysis).

Whilst I’m sure this won’t appease the hardcore Twitter devs/3rd party for hackademics like myself grabbing extra tweets and more rich data has it’s benefits.


Twitter Archive interfaceLike a growing number of other people I’ve requested and got a complete archive of my tweets from Twitter … well almost complete. The issue is that while Twitter have done a great job of packaging the archives even going as far as creating a search interface powered by HTML and JavaScript as soon as you’ve requested the data it is stale. The other issue is unless you have some webhosting where can you share your archive to give other people access.

Fortunately as Google recently announced site publishing on Google Drive by uploading your Twitter archive to a folder and then sharing the folder so that it’s ‘Public on the web’ you can let other people explore your archive (here’s mine). Note: Mark Sample (@samplereality) has discovered that if you have file conversion on during upload this will break your archive. [You can also use the Public folder in Dropbox if you don’t want to use a Google account]

The documentation wasn’t entirely clear on how to do this. Basically it seems that as long as there’s a index.html file in the folder root and links to subdirectories are relative all you need to do is open the folder in Google Drive and swap the first part of the url with https://googledrive.com/host/ e.g. https://drive.google.com/#folders/0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ becomes https://googledrive.com/host/0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ/

So next we need to keep the data fresh. Looking at how Twitter have put the archive together we can see tweets are stored in /data/js/tweets/ with a file for each months tweets and some metadata about the archive in /data/js/, the most important being tweet_index.js.

Fortunately not only does Google Apps Script provides an easy way to interface Drive and other Google Apps/3rd party services but the syntax is based on JavaScript making it easy to handle the existing data files. Given all of this it’s possible to read the existing data, fetch new status updates and write new data files keeping the archive fresh.

To do all of this I’ve come up with this Google Spreadsheet template:

*** Update Twitter Archive with Google Drive ***
[Once open File > Make a copy for your own copy]

Important: Google have changed some of their backend. If you took a copy of this template prior to 8th Dec 2014 the script will fail with a Script is using OAuthConfig, which has been shut down. Learn more at http://goo.gl/IwCSaV (line 277, file "Code"). To get your archive running again the best solution is to copy the template and setup using your existing Drive folder and API key/secret.

Billy has spotted an inconsistency with the way Drive searches for the files used in the script. This has been addressed in an update to the code but existing news will need to either take and setup a fresh copy or open their existing copy and then open Tools > Script editor and replace the code with version here.

Note: There is currently an open issue which is producing the error message 'We're sorry, a server error occurred. Please wait a bit and try again.' Hopefully the ticket will be resolved soon

The video below hopefully explains how to setup and use (Update The script uses a new authentication flow which makes setup easier (and API registration is not necessary if you already use TAGS v6.0):

A nice feature of this solution is that even if you don’t publically share your archive, if you are using the Google Drive app to syncs files with your computer the archive stays fresh on your local machine.

The model this solution uses is also quite interesting. There are a number of ways to create interfaces and apps using Google Apps Script. Writing data files to Google Drive and having a static html coded based interface is ideal for scenarios like this one where you don’t rely on heavy write processes or dynamic content (aware of course that there will be some sanitisation of code).

It would be easy to hook some extra code to push the refreshed files to another webserver or sync my local Google Drive with my webhost but for now I’m happy for Google to host my data ;s