For a couple of years now to support my research in Twitter community analysis/visualisation I’ve been developing my Twitter Archiving Google Spreadsheet (TAGS). To allow other to explore the possibilities of data generated by Twitter I’ve released copies of this template to the community.
In September 2012 Twitter announced the release of a new version of their API (the spreadsheet uses this to request data from Twitter). Around the same time Twitter also announced that the old version of their API would be switched off in March 2013. This has required some modification of TAGS to work with the new API. The biggest change for TAGS is that all requests now need authenticated access.
So here it is:
*** Twitter Archive Google Spreadsheet – TAGS v5.0 ***
[If the first link doesn't work try Opening this Spreadsheet and File > Make a copy]
Instructions for setting up TAGSv5
Instructions are included on the Readme/Settings sheet of the template. If you are having problems it’s worth checking Instructions for Tags v5.0 written by Stacy Blasiola (@Blasiola) or this modified version by Karen Smith & Shanifa Nasser made for Open Data Day Toronto available as CC-BY-SA.
What will happen to my existing TAGS sheets that aren’t version 5.0?
When Twitter turn off the old API (test outages this March) all authenticated and unauthenticated search requests will stop working.
How do I upgrade existing versions of TAGS spreadsheets (v3.x to v4.0) to keep collecting beyond March 2013?
As I can’t push an update to existing copies of TAGS you’ll have to manually update by opening your spreadsheet, then opening Tools > Script editor… and replacing the section of code that starts function getTweets() { and finishes 134 lines later (possiblly with the line function twDate(aDate){ ) with the code here. [And yes I know that’s a pain in the ass but best I could do] … or you can just start a new archive using TAGSv5.0
More additional tips and info when I get a chance
Last updated by at .
Loading...
I use your script, but I wanted to create new one and it seems that something is wrong. I have only “running script” but no data is pulled from TW. Do you have any idea?
not authenticating is where most trip up
Thanks for this awesome tools! :D
Tweet mining is so cool too…
Hey Martin,
I just have a question about rate limitations. I know that it is possible to search up to 15,000 tweets or tweets from the past 7 days (which ever limitation is reached first), but would you be able to tell me if any further rate limitations exist specific to this software? For example, I know that I was having a problem at one point where I made too many searches in one 24-hour period, so my searches were unable to run any further. Is there an exact rate limitation in this regard? Maybe the 15K/7-day rate limitation applies in this scenario? Thank you!
All the Best,
Brendan
Hi Brendan – the api docs state 180 calls per hour. The script fetches 100 tweets per call. TAGS Advanced menu has a ‘Test Rate’ menu option for you to monitor remain calls
Ah, perfect! Exactly what I was looking for; thank you!
Hi Martin – I’ve been experimenting with the TAGS v5.0 script to explore how to archive our #cdnedchat sessions. I have had trouble getting the archive to show times during the turnover from end-of-day to next (23:59:00 June 3rd to 00:59:00 June 4th) in the search.
I’ve tried changing the settings up (2 days, 1 day, 3 days) but that particular hour is just not showing up in the search. As it happens, that’s the exact hour when our chat happens online.
Any way for me to fix this? What do I need to adjust in the readme/settings?
Thanks!
Okay, I went back to Tools, Script Editor and I ran the collectTweets function and now it is working – all tweets are showing up. Just have to figure out how to get the analytics to work with the “paged” archives. Slowly but surely! Thanks for such an amazing resource!
Hi Tanis – glad you got it working. The paged collection is a feature from the very early versions of this template and I haven’t really been looking after it. My recommendation would be to use the single archive version. If you want to give users easy navigation per chat the MLAA developer TAG Viewer is a great option. Here’s an example on how the TAGS sheet for #cdnedchat is rendered. Alternatively there is this searchable interface I’ve created
Martin
Brilliant! Thanks for the suggestions regarding archiving multiple chats in one archive – love these resources you have created!
Thanks for this great tool. Really a brilliant thing. I also got a question. I just ran a sweep but was wondering why it doesn’t seem to go very far back in time. Is there a way to make it sweep tweets from the past few days? Right now it only goes back for about a couple of hours even though I know that there is much more. Any ideas?
Thanks again.
Micha
One more question…does the API v1 Retirement by Twitter on June 11
(see https://dev.twitter.com/blog/api-v1-retirement-date-extended-to-june-11) impact the script to be working properly?
Thanks again for your good work.
Version 5 was developed specifically to address the api changes
Since yesterday the script is not working. There is a problem with the variable followers, that I am suppose that is due to the retirement of the API v1. I have been using your script for several months now, and I wanna thank you for that.
Martin this is the error that I get:
Apps Script
Line 157 Cannot read property “followers_count” from undefined.TypeError
I tried to set up a new gsheet but I get the same error. It was reported that twitter has made some changes on how to get the followers, but I didn’t have so far the opportunity to search for more details.
Thank you again, for your work.