Google Spreadsheet

Previously I’ve written about how we use Google Apps Script to extract data from one of the Association’s online systems. We use this technique to generate charts which give us some insight to our current standing. One limitation we had was to view the charts we had to open the Google Sheet with them in. There are various techniques for automatically distributing these using Apps Script and in this post I’m going to highlight one method for emailing inline charts. Below is what the result looks like, an email with an embedded chart, no need to download or preview attachments.

End product - inline charts

Below is the same chart in the Google Sheet. If we make changes to the chart appearance or update the data, when we send the chart we get the latest version.

Source chart in Google Sheet that will be emailed

I found the code snippet for doing this on Stackoverflow. This works well but there are two issues to be aware of.

Google Sheet needs to be shared with anyone with link

As noted in the SO comments for the chart to appear the Google Sheet containing it needs to be shared with ‘anyone with the link’. Without this all you get is a blank image. This potentially creates a data protection issue if using sensitive information. There are ways you can juggle this by such as using the IMPORTRANGE formula to only publish aggregated data. This looks like a bug which I’ve noted in this issue ticket.

Missing numeric labels

The second issue we discovered was that axis with numeric values end up prefixed with ‘General’  e.g. as seen below General88000.

Numeric label issue

The solution is fairly straight forward and actually opens the door to doing a lot more with charts. It uses the EmbedChartBuilder service to grab the chart object. The original Google announcement for the Chart services highlights you can:

  • Add/Remove the ranges this chart represents via addRange and removeRange.
  • Set options that modify how the chart will be rendered as well as change the chart type with setOption and setChartType.
  • Change where the chart will be displayed (the cell and cell offset of the chart container) via setPosition.

To achieve proper axis formatting requires some cross document reference lookup to find the correct setOption to use. In this case it means jumping to Google Chart API reference for Line Charts to see the configuration options and setting the format:

builder.setOption('vAxis.format', '#');

The modified version of the function we use is included below:

At the Association for Learning Technology (ALT) one of the core tools we use for membership management is CiviCRM. CiviCRM has a number of ‘out-of-the-box’ reports you can use and run to summaries and analyse memberships and contributions. Given the flexibility of Civi you can also with a bit of know how create custom extensions and reporting options enabling so very sophisticated analytics. At ALT we are keen to make more use of the data we have on memberships but at the same time have limited resources and time to implement these. In this post I’ll highlight how using Google Sheets we’ve developed a framework that allows us to rapidly develop custom reporting.

Problem

If you are already a CiviCRM user you are probably already aware of CiviReports and the ability create basic custom reports which allow you to view and download data. As part of this you can also schedule reports to land in your inbox. This is great but has it’s limitations. In particular, additional setup is required if you don’t want to just report on a daily basis; you end up with tables of data, with no graphical summaries; and combining current and historic data isn’t possible.

Solution

Scheduling reports at custom intervals

CiviCRM provides an interface to schedule a mail_report. The issue many people discover is this will send reports on set intervals usually hourly or daily. You can schedule individual jobs to run a specific periods but you quickly find yourself in the world of command lines and CRON jobs. Crons are scheduled tasks run by a web server. If you have dedicated admin support this is a fairly standard task and the instructions are easy to follow. At ALT we have the option to open a support ticket with our host but this seems like a waste on time and money.

Our solution is to use a Google Sheet… well a Google Sheet with a bit of ‘juice’. The sheet is shared with our team and anyone can add a CiviReport id to Column A and choose how often it runs in Column B using a data validation list option.

deciding what reports run when

But how does this trigger our civi install to run the job? The juice is Google Apps Script, a cloud based scripting language native to Google Drive. Apps Script is a free service from Google and fortunately for us has the ability to run scripts on configured time intervals. It also has the ability to call specific urls using the build-in UrlFetchApp (similar to CURL). I’ll give you a link to this Sheet so you can setup your own later and when you do you’ll see the entire process is managed with a couple of lines of code included below:

function doTasks() {
  var doc = SpreadsheetApp.getActiveSpreadsheet(); // get spreadsheet
  var sheet = doc.getSheetByName("Tasks"); // get sheet
  var data = sheet.getRange(3, 1, sheet.getLastRow(), COL.total).getValues(); // get values
  var now = new Date(); // time now
  // for each row of the sheet interate accross
  for (var i = 0; i < data.length; i++){
    if (data[i][COL.report_id] != ""){ // if there is instance id do something
      // collect row values
      var report_id = data[i][COL.report_id]
      var type = data[i][COL.type];
      var next_run = data[i][COL.next_run] || 0; 
      // check if it's time to run the report again
      if (next_run < now && type != "never"){
        // if it is ping the report trigger
        var new_next_run = callUrl(report_id, type, {format: data[i][COL.format], ss_id: data[i][COL.ss_id], ss_sht: data[i][COL.ss_sht]} );
        // ..and record when to run again
        sheet.getRange(parseInt(i)+3, 3, 1, 2).setValues([[now, new_next_run]]);
      }
    }
  }
}

What this does is read the sheet data and then iterate across each row. If the report is overdue to be run again it calls a another custom function callUrl which will run the CiviReport and return/write when next to run.

Creating graphical summaries and combining data

By this point you may be sensing that I’m partial to solving problems with Google Sheets. With Sheets it’s fairly straight forward to manually export different reports from Civi and analyse using formula and Charts. The manual export of CiviReports can get tiresome so how can we automate this? Again we return to Google Apps Script. One of the options in CiviReports is to attach the data to the emailed report as a .csv file. From the previous example we can see it is possible to read and write data to a Google Sheet. So if we can get the .csv file from our emailed report we can write it to the Sheet … right?

This is actually more straight forward than you may think as another feature of Google Apps Script is to interact with the script owner’s Gmail.  As part of this we can search for messages and get associated attachments. Using this we can read the latest report from Civi, write the data to a sheet and with a bit of clever formula building automatically get the latest summary or custom chart. As Apps Script runs in a pre authenticated environment, no oAuth handshakes here, the code is relatively straight forward:

function processInbox(){
  var PS = PropertiesService.getScriptProperties();
  var data = PS.getProperties();
  for (var key in data) {
    if (key.substring(0, 10) == "search_str"){
      var param_raw = data[key];
      var param = JSON.parse(param_raw);
      // get last 20 message threads using serach term
      var threads = GmailApp.search(param.search_str, 0, 20); 
      // assume last thread has our latest data
      var last_thread = threads.length-1;
      if (last_thread > -1){
        // get message in the last thread        
        var msg =  threads[last_thread].getMessages()[0];
        // get the attachments
        var attachments = msg.getAttachments();
        for (var k = 0; k < attachments.length; k++) {
          // get the attachment as a string
          var csv_str = attachments[k].getDataAsString();
          // parse string as csv
          var csv = Utilities.parseCsv(csv_str);
          // create destination object
          var doc = SpreadsheetApp.openById(param.ss_id);
          var sheet = doc.getSheetByName(param.ss_sht);
          // clear any old data
          sheet.clear();
          // write new data
          sheet.getRange(1, 1,  csv.length, csv[0].length).setValues(csv);
          // mark message are read and archive (you could also label or delete)
          threads[last_thread].moveToArchive().markRead();
          PS.deleteProperty(key);
        }
      }
    }
  }
}

NB

Data protection

There are a couple of things worth noting here. Google Sheets are a fantastic collaborative environment and with this solution we can still share spreadsheets to selected people in our organisation and beyond. Something to remember though is this script runs as the sheet owner so when configuring the CiviReport it needs to go to the email address of the owner. At ALT we benefit from being a Google for Education user so our email and Drive access comes as part of the Google Apps suite. This solution could also be setup to run on a regular Google account but there are data protection issues to consider sending reports to a non-organisation contact. As such you might only want to re-use this solution as an easy way to schedule reports rather than schedule and process the data.

ARRAYFORMULA, FILTER, QUERY functions are your friends

As our solution dumps .csv data in a sheet, clearing any previous data, any formulas you have in the sheet will also be lost. We get around this by doing all data manipulations in a separate sheet which references the imported data. To save copy and pasting lots of formulas we make extensive use of the ARRAYFORMULA, FILTER, QUERY functions available in Google Sheets.

Comparing CiviEvent registrations

One scenario we have is monitoring the number of registrations to an annual conference. It’s easy for us to export registrations for previous years as static data into a spreadsheet. For the analysis we want to group the number of registrations by week number. To get a week number for the live data we create a second sheet that references it. Using the ARRAYFORMULA you can enter the reference once which is then applied to all the values in the range. For example, we get all the registration dates in column A using =ArrayFormula('2015LiveData'!B:B) in cell A1 and then extract the week numbers in column C by using the formula =ARRAYFORMULA(IF(ISBLANK(A2:A),"",WEEKNUM(A2:A))) in cell C2.

ArrayFormula is your friend

Setting up your own copy of Schedule CiviCRM Reports

If this is a project you’d like to use and/or build upon you can copy a blank version of our template below:

Schedule CiviCRM Reports
[While signed in File > Make a copy]

Once you have a copy open Tools > Script editor to see all of the project code and instructions for setting up.

I’ve hopefully given you enough to go on to setup but feel free to leave a comment if you get stuck or have any questions.

Enjoy!

Example of pdf version from altc 2014At ALT we use Google Sheets as an easy way to share and collaborate on draft event timetables. Recent examples are the ALT Annual Conference 2014  and the OER15 timetables. One of the reasons for publishing draft timetables using Google Sheets is  we can get a static url for people to download it as PDFs but the contents can be dynamically updated (see recent post on doing this). The template we use for conferences is continually evolving which isn’t an issue as it’s easy to copy the last version. One headache is that our theme colour usually changes. This can be a bit fiddly change as we use empty cells to create a thicker grid:

Thicker borders

Faced with another cell background switch it made sense to actually do this with code rather than clicks and thanks to Google Apps Script possible in 19 lines of code and a couple of minutes:

function colorReplace() {
  var doc = SpreadsheetApp.getActiveSheet();
  // get all the existing active sheet background colours
  var cells = doc.getRange(1, 1, doc.getLastRow(), doc.getLastColumn()).getBackgrounds();
  var rows = cells.length;
  var cols = cells[0].length;
  // iterate accross
  for (var i = 0; i < rows; i++){
    for (var j = 0; j < cols; j++){
      if (cells[i][j] == '#feeff8'){ // first color to change
        cells[i][j] = '#f3f3f3'; // first color change to
      } else if (cells[i][j] == '#bf0875'){ // second color to change
        cells[i][j] = '#079948'; // second color to change
      } 
    }
  }
  // update backgound colours
  doc.getRange(1, 1, doc.getLastRow(), doc.getLastColumn()).setBackgrounds(cells);
}

Tip

To get the existing cell background colour I used the debugger setting a breakpoint before the loop to see the existing cell colour HEX codes:

debugger to inspect cell colours

At ALT we are a Google Apps for Education user and make extensive use of Google Drive for creating and publishing documents. Some of our documents, such as member lists or timetables, get regularly updated and updating links in our websites can be a chore. One of the nice features of documents in Google Drive is you have a couple of ways of publishing documents so anyone can view (no log in required). For Google Sheets the main options are getting a shareable link:

share with others  ... because sharing is good ;)

Or using the ‘publish to the web’ option:

publish to web new school

In the ‘old’ version of Google Sheets the ‘Publish to the web’ option included lots of file types, not just as a web page:

publish-to-web old skool

You can still get new Google Sheets to generate download links for other formats … it’s just a bit complicated. The complicated bit is working out which url tweaks you need. For example, with your magic goggles on you might start spotting a pattern in this url which I’ve line breaked to make easier to see:

https://docs.google.com/spreadsheets/d/1muAOv_chNyCqtx26chZz2RLZlP-wySYsJSFt8UuEO0I/export?
format=pdf&
attachment=FALSE&
size=A4&
fzr=FALSE&
landscape=FALSE&
fitw=TRUE&
gridlines=FALSE&
printtitle=TRUE&
sheetnames=FALSE&
pagenum=FALSE

Remembering all these switches isn’t easy so for our team I’ve shared a simple template with lets us drop in a shared Google Sheet link, choose options for layout and get a url. Here’s a copy in case you find useful:

PDF Link for Google Sheet Template
PDF Link for Google Sheet Template

4 Comments

My old colleague at Cetis, David Sherlock, posted a nice little  ‘Twitter Question/Revision Bot’. This uses a .csv file of questions and multiple choice answers which get randomly tweeted out using a Python script. David designed the project with a Raspberry Pi in mind but also highlights it can be easily run on any Unix like environment such as Mac OS X or Linux. As not everyone is going to have easy access to this here’s how you can do something similar with Google Sheets (if you don’t want to play copy and paste coding make a copy of this sheet).

1. Setting up a Google Sheets environment to handle it

  1. Start with a new Google Sheet and like David have six columns in each row with question, answer, three options (one of which the correct one) and an extra row to record if the question has been asked.
  2. In your spreadsheet open Tools > Script editor and when asked start a ‘Blank Project’
  3. In the new editor window select Resources > Libraries (this will first prompt you to give your project a name, I called mine TwitBot.
  4. In the ‘Find a Library’ box enter MarIlVOhstkJA6QjPgCWAHIq9hSqx7jwh and click Select
  5. You should now see ‘TwtrService’ listed as one of the libraries. In the ‘Version’ dropdown select the latest version (at time of writing 14), and click Save

TwtrService is a library I’ve written so interact with the Twitter API. You can read more about it here.

In the code window add the following code and click save:

function setup() {
  if (TwtrService.isUserConnectedToTwitter()){
   var result = Browser.msgBox("Twitter Authorisation",
                   "You appear to already be connected to Twitter.\\n\\nWould you like to run the setup again?",
                   Browser.Buttons.YES_NO);
    // Process the user's response.
    if (result == 'yes') {
      // User clicked "Yes".
      TwtrService.showTwitterKeySecret(SpreadsheetApp);
    }
  } else {
    TwtrService.showTwitterKeySecret(SpreadsheetApp);
  }
}

The above code uses the TwtrService to help you set up Twitter access if required

Your script window should look like this:

script editor

2. Create a Twitter App

If you’ve used my other TAGS templates you can reuse your details for this. To see if Twitter App details are required from the script window select Run > setup (if setup isn’t listed you need to first save your code). Running setup will start the first part of the authentication process. Click continue and review the authentication required and ‘Accept’ if you are happy.

Auth required

Going back to the spreadsheet you started there should now be a dialog window asking you to do something. If you haven’t setup a Twitter App before it should look like this:

Twitter app creation

Follow the instructions onscreen to create your app.

Important: The Twitter account you use to authorise access is the one that will send out the tweets

3. Write a Python Google Apps Script

Add the code below to your existing script project and save:

var doc = SpreadsheetApp.getActiveSpreadsheet();
var sheet = doc.getSheetByName('Sheet1');

function tweetQuestion(){
  var ran = Math.floor(Math.random() * (sheet.getLastRow()-1)) + 2;
  var row = sheet.getRange(ran, 1, 1, sheet.getLastColumn()).getValues()[0];
  if (row[5] !== ""){
    tweetQuestion(); // if already asked pulls another random row
  } else {
    var tweet =  "Q: " + row[0];
    var tweet2 =  row[2];
    var tweet3 =  row[3];
    var tweet4 =  row[4];
    
    TwtrService.post('statuses/update', {status: tweet});
    var options = [tweet2,tweet3,tweet4];
    shuffle(options); 
    TwtrService.post('statuses/update', {status: "A: " + options[0]);
    TwtrService.post('statuses/update', {status: "B: " + options[1]);
    TwtrService.post('statuses/update', {status: "C: " + options[2]);
    sheet.getRange(ran, 6).setValue(new Date());
  }
}

// http://stackoverflow.com/a/25984542/1027723
function shuffle(a,b,c,d){//array,placeholder,placeholder,placeholder
 c=a.length;while(c)b=Math.random()*c--|0,d=a[c],a[c]=a[b],a[b]=d
}

4. Time the script to run every hour or so

In the script editor select Resources > Current project’s triggers and click ‘No triggers set up. Click here to add one now.’. Select to run tweetQuestion every hour (or your preference), and also click ‘notification’ so you can get an email if the script fails. Finally click ‘Save’

Timed triggers

What will happen now is the function even if you don’t  have the spreadsheet or script editor open or even your browser.

Important: when this script runs out of questions it will go into an infinite loop. You can go back into the trigger window to remove the function at any point. If you don’t you’ll end up using all of your script runtime quota. You homework is to figure a way to get the script to bail if there are no questions left.

My homework...

function tweetQuestion(){
  var asked_col = sheet.getRange(2, 6, sheet.getLastRow()-1).getValues();
  // get unasked q's
  var unasked = [];
  for(var i=0; i < asked_col.length; i++) {
    if(asked_col[i][0] == "") {
      unasked.push(i+2);
    }
  }
  // randomly pick one
  var ran = Math.floor(Math.random() * unasked.length);
  if (unasked[ran]){
    var row = sheet.getRange(unasked[ran], 1, 1, sheet.getLastColumn()).getValues()[0];
    var tweet =  "Q: " + row[0];
    var tweet2 =  row[2];
    var tweet3 =  row[3];
    var tweet4 =  row[4];
    
    TwtrService.post('statuses/update', {status: tweet});
    var options = [tweet2,tweet3,tweet4];
    shuffle(options); 
    TwtrService.post('statuses/update', {status: "A: " + options[0]});
    TwtrService.post('statuses/update', {status: "B: " + options[1]});
    TwtrService.post('statuses/update', {status: "C: " + options[2]});
    
    sheet.getRange(unasked[ran], 6).setValue(new Date());
  } else {
    // no questions left - do something else
  }
}

1 Comment

+Tom Smith kindly reminded me that there are existing Google Form Add-ons to achieve a similar result without having to mess with code. So you might want to have a look at Choice Eliminator by Bjorn Behrendt and formLimiter by Andrew Stillman

You can imagine the scenario, you’ve got a Google Form perhaps setup as a booking form and a select option for picking timeslots. As the slots fill up you’d like the option to be removed. I was surprised not to find an example of this so here is a rough sketch of some code that might do the job. We start with a basic form with a ‘choose from list’ option:

Select option

and in the linked Google Sheet we have an extra sheet to keep tally:

tally

In this sheet I’ve got a column with quotas for each of my select options and a calculation for how many are left =B2-COUNTIF(responses!B:B,"="&A2)

To dynamically modify the Google Form we need to add some code. Here’s a basic snippet that reads our options and quotas and rewrites the select options (note the inline comments for bits you’ll need to edit):

function availableSlots(){
  var form = FormApp.openByUrl('URL_OF_YOUR_FORM'); // TODO add your form url
  // need to read what slots are available
  var slots = SpreadsheetApp
                .getActiveSpreadsheet()
                .getRange("slots!A2:C10")
                .getValues(); // TODO make sure getRange matches your quota range
  var choice = [];
  // loop through our available slots
  for (s in slots){
    // test if slot still available
    if (slots[s][0] != "" && slots[s][2] > 0){ 
      choice.push(slots[s][0]); // if so we add to temp array
    }
  }
  var formItems = form.getItems(FormApp.ItemType.LIST); // our form list items
  // TODO assumption that first select list is the one you want to change  
  // change formItems[n] if you have more than one select list
  // and we just rewrite all the options to ones that are free
  formItems[0].asListItem().setChoiceValues(choice); 
}

To enable this you need to use the Tools > Script editor in you Google Sheet and then Resources > Current project’s triggers. At this point to may be prompted for authorisation to read the data and change the form. Once that’s done add the availableSlots function to trigger on form submit.

Current project’s triggers

If you’d like to shortcut all but this last step make a copy of this template.

Notes

Some things to be aware of. If you have concurrent users submitting the form someone may sneak in. Apps Script has ways of preventing this but not as far as I’m aware for Google Forms. It’s worth remembering that do don’t need to use Google Forms at all for getting data into a Google Sheet and if you can host your own HTML form you can make POST/GET requests. If you develop this example further I’d be interested to hear about it. Enjoy!

1 Comment

In the Apps Script G+ Community Scott Marquardt asked:

Youtube API question.

Argh, I can't find a single example of anything like this. With my current knowledge, I have a hard time inferring from the API reference documents alone, to Apps Script.

My goal is a script to add an item to a Youtube playlist.  My intended workflow is a Google form to enter video IDs into a Google sheet, with a triggered script adding each new video ID to a hard-coded playlist.

I'm unashamedly panhandling here, on behalf of what would be a really valuable use case in a dozen of our special education schools.

I agree with Scott that the Apps Script Advanced Services documents (here’s YouTube) don’t give you much to go on. The solution is actually relatively simple once you get your head around the shape of the data required. So here is a form linked to this playlist (I’ll turn this off as soon as it starts  getting spammed). In the spreadsheet receiving responses the Tools > Script editor has the following code (in the script you’ll see where you need to hard code the playlist you want to add videos to):

Update: Scott asked about adding notes/comments to playlist items and I've updated the code to do this to the gist

To get this working there are a couple of hoops to jump through. As the YouTube API is an Advanced Service and it must be enabled before use. For this project to do this you need to be in the Script Editor then:

  1. Click Resources > Advanced Google Services…
  2. Scroll down to YouTube Data API to turn it on then click the ‘Google Developers Console link:
    Enabling Advanced Services
  3. In the Google Developers Console find and turn on the YouTube API. After it’s enabled you can close the Console window

You can only interact with YouTube Channels the account authorised to run the script is the owner of (see related post by me on this).

The final step is to setup the script to trigger when the form is submitted. To do this while still in the Script Editor  select Resources > Current project’s triggers and add a run addVideoToPlaylist event on form submit.

Current project triggers

If you would prefer to not do all the code copy and pasting you can File > Make a copy of this sheet (you’ll still need to enable Advanced Services and the form submit.

There’s obviously lots more you could do with this script like putting submissions into a moderation queue but I’ll let you go and play with those ideas.

You’ll find lots of blog posts explaining how to do this the other way, converting a table from Wikipedia into a spreadsheet. I even cover this myself in Feeding Google Spreadsheets: Exercises in using importHTML. This post looks at taking a .csv or a spreadsheet and formatting it as a Wikipedia table.

I’ve been on a bit of a Wikipedia journey expertly guided by a friend, Simon Knight (@sjgknight) … who happens to be Wikimedia UK Vice Chair. The journey has included Wiki ShootMe which has been covered by Alan Levine (@cogdog) and a wonderful Teaching Wikipedia Editing session you can still watch from Connected Courses.

One of the great things about Wikipedia is there is no shortage of information on how to do stuff. Before jumping in I had a quick look for some existing tools and came across this article on Importing (converting) content to Wikipedia (MediaWiki) format. Listed is the Spreadsheet-to-MediaWiki-table-Converter. This is some code you would more normally run on a server possibly used as a library within another project. Wanting something simpler I looked for and found guidance on creating a sortable table. The guidance helpfully includes an example copied below:

{| class="wikitable sortable"
|-
! name
! data
! more data
|-
| cats
| 273
| 53
|-
| dogs
| 65
| 8,492
|-
| mice
| 1,649
| 548
|}

And here is what it looks like:

namedatamore data
cats27353
dogs658,492
mice1,649548

Hopefully you can quickly see the pattern, you start with the column headings, one per line of wikicode and then each of the rows again one value per row for each cell. This type of pattern is easy to code … if you can code … … if you want to code. As I was composing my first line I remembered an old project Templated Export for Google Spreadsheets:

Templated Export allows users to reshape and selectively publish data from Google Spreadsheets in custom formats. The tool is powered by Google App Script which allows integration with Google Spreadsheets using existing authenticated access. Example uses include: turning data from a spreadsheet into a RSS feed and publishing data from a spreadsheet in JSON format to be used by other tools/services. The framework for creating a custom wrapper for your data means there are many more opportunities.

Templated Export lets you customise use a basic reference system to create you own output pattern:

Templated Export

There is some learning curve using this and original post has details on setting this up and using it. The basic concept is you use a cell reference as a way to build up your pattern. You can save your templates which means if you update your data in your Google Sheet you can open the Templated Export interface and download it again or use a saved ‘published url’.

Templated Export hasn’t been touched for two years and is perhaps a candidate for some ‘modern’ Google Apps Script techniques such as publishing as an Add-on. If the thought of playing with Google Apps Script doesn’t take you fancy, Templated Export is based on a feature of OpenRefine called Templating Exporter. OpenRefine can be downloaded and run from your desktop and can handle a variety of data file formats.

So go forth and fill Wikipedia with lovely tables for everyone else to reference in ‘Wikipedia table export’ posts ;)

1 Comment

I previously posted about TwtrService: A Twitter API client library for Google Apps Script which makes it easy to interact with Twitter from Google Drive applications like Google Sheets. One of the nice things about TwtrService is that once you setup a connection to Twitter you can use it many times in different projects, basically allowing you to do stuff in one line of code. In the post I said I’d share some of the examples of things I make so here is the first one, EasyTweetSheet.

What it does

At the Association for Learning Technology we organise lots of events. We only have a small staff team so having someone sending out tweets during the event can be a problem. We could use a Twitter client like Tweetdeck or Hootsuite  to schedule tweets during the day. One issue is if something goes wrong like the livestream not working or a session starting late you can look a bit silly. The solution was to draft our tweets in a Google Sheet and have a link we click when we want the message to be sent. Below is a screenshot for the one we used at this year’s ALT Annual Conference:

EasyTweetSheet used at #altc 2014

How to get you own copy working

  1. Open this copy of the EasyTweetSheet templateand File > Make a copy
  2. In your copy open Tools > Script editor and follow the instructions
  3. Start filling the ‘text’ column with what you want to tweet which should enable the ‘tweet’ link

IMPORANT: If you’ve used my other tools like TAGS this template will use the Twitter account you used to set it up. To use a different Twitter account to send the tweets from
replace YOUR_CONSUMER_KEY and YOUR_CONSUMER_SECRET in lines 34-35 of the Script editor code with your Twitter application key/secret. When you Run > setup switch back to the Sheet view and follow the instructions.

Enjoy!

3 Comments

As part of the latest release of TAGS (Twitter Archiving Google Sheet) I moved a lot of the code into a Google Apps Script Library. Libraries are a great way to bundle and release code allowing you to write your own classes and methods. To see some of the functionality already being developed by the Apps Script community you should have a look at the Google Apps Script Samples site and the excellent work Bruce McPherson has done which includes a EzyOauth2 library.

One of the things you can do with libraries is wrap one library into another. When rewriting TAGS it made sense to strip out a separate Twitter client library that I and others could use in different projects. Based on the work by Arun Nagarajan at Google, TwtrService provides  access to Twitter's REST API. The work I’ve done is to add some UI and configuration methods to try to streamline the authentication flow. As part of this developers can offer authentication routes using their own Twitter application or use an application created by users. This particular angle is a result of one of the design principles for TAGS, that every copy of the template should use a Twitter application owned by the user. The reason behind this is to distribute the risk. If Twitter were to suspend my data access because a TAGS user abused their API it would suspend access for all TAGS users. By requiring TAGS users to register their own application with Twitter the responsibility to abide by Twitter’s terms of service lies with them. So in TAGS the auth flow looks like this

The result is hopefully a flexible library that developers can integrate into their own projects or by getting users to register their own.

Over the next couple of weeks I'll be sharing some examples applications we've developed at ALT. In the meantime this post serves as a basic introduction to TwtrService and covers:

Overview of TwtrService

The TwtrService library for Google Apps Script centrally stores your Twitter access details allowing them to accessed from multiple script projects without the need for re-authentication. TwtrService is designed to allow you to directly use the Twitter’s v1.1 REST API GET and POST methods. For example to return Twitter search results for the search ‘Google Apps Script’ you would use:

var data = TwtrService.get('https://api.twitter.com/1.1/search/tweets.json', {q: 'Google Apps Script'});

The url string can also be abbreviated to:

var data = TwtrService.get('search/tweets', {q: 'Google Apps Script'});

Additional options can be passed in the parameters array. For example to return 100 results for the same search you would use:

var data = TwtrService.get('search/tweets', {q: 'Google Apps Script', count: 100});

The project key for this library is MarIlVOhstkJA6QjPgCWAHIq9hSqx7jwh and the TwtrService methods are documented here.

To use the Twitter REST methods TwtrService first needs authenticated access. TwtrService has some built-in methods to do this detailed below. Once a user has authenticated access the TwtrService library stores these as User Properties. This means when a user has authenticated once with TwtrService using the library in another container-bound or standalone Apps Script immediately gives them access to Twitter API results using the get/post methods. In terms of security User Properties are limited to the effective user of the current script.

Usage

Quick start: Personal access

If you would just like to use TwtrService for your Apps Script projects the easiest way to get started is to register a Twitter application and enter it’s details on this page (if you are interested here is the source code for the interface).

Note: If you are already a user of TAGS you’ll already be able to use TwtrService without the step above.

In your Apps Script project you’d like to use the Twitter API in the Script Editor window use Resources > Libraries and add the service  using the project key MarIlVOhstkJA6QjPgCWAHIq9hSqx7jwh.

In your project you can now use the TwtrService.get() and TwtrService.post() methods. The documentation for get() is detailed below (post is the same but uses HTTP POST):

get(string url, Object parameters)

GET wrapper for request to the Twitter REST API. For full documentation of API method endpoints and parameters see https://dev.twitter.com/rest/public. For example to get last 100 tweets containing 'Google Apps Script': var data = TwtrService.get('search/tweets', {q: 'Google Apps Script', count: 100});

Arguments:
NameTypeDescription
urlstringTwitter REST API resource url. This can be either long form e.g. https://api.twitter.com/1.1/search/tweets.json or abbreviated e.g. search/tweets
parametersObjectadditional API parameters as detailed in the Twitter REST API documentation e.g. for search results a search string and count is specified by {q: 'Google Apps Script', count: 100}.
Return Values:
TypeDescription
ObjectAPI response in JSON format.

Quick start: Personal access in Sheets and Documents

If you would like to replicate the TAGS authentication flow where users enter their Twitter application key/secret TwtrService comes with a number of UI methods. For TAGS the following code is used:

/**
* Launches key/secret and auth flow
*/
function setup() {
  if (TwtrService.isUserConnectedToTwitter()){
   var result = Browser.msgBox("Twitter Authorisation", 
                   "You appear to already be connected to Twitter.\\n\\nWould you like to run the setup again?", 
                   Browser.Buttons.YES_NO);
    // Process the user's response.
    if (result == 'yes') {
      // User clicked "Yes".
      TwtrService.showTwitterKeySecret(SpreadsheetApp);
    } 
  } else {
    TwtrService.showTwitterKeySecret(SpreadsheetApp);
  }
}

/**
* Used as part of setup() to process form data
*/
function processForm(formObject) {
  TwtrService.setUserKeySecret(formObject);
  TwtrService.showTwitterLogin(SpreadsheetApp);
}

Quick Start: Shared Twitter Key/Secret

The earlier examples have assumed the user registers their own Twitter application. For scenarios where you would like to have the option for users to have authenticated access using a dedicated Twitter API key/secret it is possible to initialize these values. An example application code can be found here which is also deployed here.

Similar to earlier examples once a user authenticates access with your key/secret as long as these values are also initialized in other script projects the user will have access to the Twitter API via TwtrService.

Instructions for creating a Twitter Application to use with TwtrService

TwtrService requires you to have a registered Twitter application. If you are If you haven’t already done this here are some steps you need to get started:

  1. Register for an API key with Twitter at https://dev.twitter.com/apps/new (if you've already registered for a TAGS sheet you can reuse your existing API Key and Secret).
    • Name, description and website can be anything you like
    • Important Include the Callback URL https://script.google.com/macros/
  2. Read the 'Developer Rules of the Road' before clicking 'Create your Twitter application'

On the Twitter site your application should include a ‘Keys and Access Tokens’ tab which includes the Consumer Key (API Key) and Consumer Secret (API Secret) you’ll need.