Twitter Archiving Google Spreadsheet TAGS v5

Update: TAGS v6.0 is here! You can jump straight to the new site. If you are using version 5.x here are instructions on how to upgrade (there is an old Sheets version of TAGS which is recommended for use with TAGSExplorer).

For a couple of years now to support my research in Twitter community analysis/visualisation I’ve been developing my Twitter Archiving Google Spreadsheet (TAGS). To allow other to explore the possibilities of data generated by Twitter I’ve released copies of this template to the community.

In September 2012 Twitter announced the release of a new version of their API (the spreadsheet uses this to request data from Twitter). Around the same time Twitter also announced that the old version of their API would be switched off in March 2013. This has required some modification of TAGS to work with the new API. The biggest change for TAGS is that all requests now need authenticated access.

So here it is:

*** Twitter Archive Google Spreadsheet – TAGS v5.1 ***
[If the first link doesn’t work try Opening this Spreadsheet and File > Make a copy]

Important: Uldis Bojars (@CaptSolo) has highlighted an issue with TAGS V5.0 which meant it would not page all the results. This has been fixed with the release of v5.1. If you are running version 5.0 of TAGS it’s highly recommended you update the code.

Note! Search API isn’t 100% reliable:

We find that the search API over-represents the more central users and does not offer an accurate picture of peripheral activity; we also find that the bias is greater for the network of mentions – González-Bailón, Sandra, et al. “Assessing the bias in communication networks sampled from twitter.” Available at SSRN 2185134 (2012).

Instructions for setting up TAGSv5

Instructions are included on the Readme/Settings sheet of the template. If you are having problems it’s worth checking Instructions for Tags v5.0 written by Stacy Blasiola (@Blasiola) or this modified version by Karen Smith & Shanifa Nasser made for Open Data Day Toronto available as CC-BY-SA.

What will happen to my existing TAGS sheets that aren’t version 5.0?

When Twitter turn off the old API (test outages this March) all authenticated and unauthenticated search requests will stop working.

How do I upgrade existing versions of TAGS spreadsheets (v3.x to v4.0) to keep collecting beyond March 2013?

As I can’t push an update to existing copies of TAGS you’ll have to manually update by opening your spreadsheet, then opening Tools > Script editor… and replacing the section of code that starts function getTweets() { and finishes 134 lines later (possiblly with the line function twDate(aDate){ ) with the code here. [And yes I know that’s a pain in the ass but best I could do] … or you can just start a new archive using TAGSv5.0

More additional tips and info when I get a chance

352 Comments



  1. Thanks Martin – good job :-)



  2. Thanks for this Martin. But I’m getting the “Script function onOpen could not be found” error message when I try to use the spreadsheet. I’ve tried making a new copy but with no better results. Any ideas? Should I try again later?


  3. Wait – it is working, but I was pressing the wrong button. I’ve got two buttons – one is labelled Enable Custom Menu – that gave the error. There is a blank button next to it that worked (?!).

    But now I’m getting an authentication error: OAuth Error (line 376, file “Google(c) Code”).


  4. hi

    was trying to use TAGS explorer but getting error message “Error in query: Access denied Access denied”

    thanks


  5. Hi Martin,
    thank you for your tremendous work and the new version of your TAGs sheet!
    Similar to Sharon I also got the “OAuth Error (line 376, file “Google(c) Code”)” error. This error occurs when I try authenticateTwitter in the Script Editor.
    Thanks
    Jonas


    1. OAuth Error (line 376, file “Google(c) Code”)

      Sounds like your key/secret are entered incorrectly or a mistake in the callback url entered in dev.twitter.com


      1. Thanks for your reply! It was indeed the missing callback url. If anyone’s interested, the solution to this error is the callback url “https://spreadsheets.google.com/macros”.
        thanks again!


  6. Like Peelie I get errors with the 2 Web view links at the foot of the dashboard. I’ve also tried entering the URL/key at http://hawksey.info/tagsexplorer/- it just sits there looking at me!


    1. For those views you need to make sure the spreadsheet is File > published to the web


      1. Doh…now I feel really stoopid! Thanks for the speedy reply.


  7. How many Tweets does this archive? I can’t get it to archive past about 80, instead of the 1500 like the last spreadsheet.

    Thanks!


    1. I imagine you are hitting the Twitter Search API 7 days limit (anything older isn’t available). I’ve got v5.0 running on #edcmooc and it’s beginning to creak at 15k tweets


  8. Hi Martin,

    Thanks again for setting this up.

    I am running into issues exporting the spreadsheet. I’ve tried all of the options, but they all return the same error: ERROR: Export to .csv failed.

    Any ideas why this is happening?

    Thanks!
    Stacy


    1. It happens when the number of tweets is greater than 10k, more or less. What you can do is to copy the file, and delete half the tweets in the original, and the other half in the copy, and export both…
      Frederic



  9. Hi,

    Thanks for this. I may be a little confused, but do is the newest version of TAGS (ie v5) sufficient to work or do we have to change the script in like it says?

    Thanks
    Ashley


    1. Hi Ashley – I’ve clarified the text slightly. v5.0 requires no update to the script.


  10. THANKS a bunch for this work! Do you have a location for contributing to the work fund?! I definitely am happy to donate a bit to the cause.


  11. Just want to thank you very much for putting this up and making the public search interface available



  12. Hi Martin,

    I’ve gotten through everything smoothly, but when I try to perform a search, the “Running script collectTweets” box just remains on-screen indefinitely and nothing ever appears in the archive. Any advice on what I’ve done wrong?

    Thanks so much


  13. So, I followed all the steps, set everything up correctly… but I have a question about how much data is pulled. I am noticing an inconsistency in the number of tweets by a particular person via a particular hashtag. Also, the google doc “dashboard” lists a little over 1500 tweets and shows that the first tweet was made 2 days ago (Feb. 28). And the analyzer only shows from March 1st…
    And I just noticed that the google archive is missing a good number of Feb. 28 tweets… (also, is it possible to change the time zone?)

    Anyway, sorry to complain, want to say that even though I’m having problems this is amazing.


    1. The twitter API is limited by the last 1500 tweets and/or 7 days. So if you made a search on the 1st March it’ll pull the last 1500 tweets upto a 7day limit (which ever one it hits first). Do you think that explains your problem? Not sure about timezone, you might be able to go into Tools Script editor and change something in File > Project settings(?) but I’ve low confidence that will work (it may need to be tweaked in the code)
      Martin


      1. Thanks for the response…

        Curious, is there a way I can embed the archive or explored on my webpage?


  14. Dear Martin,

    I was so excited to find this and I have followed the instructions carefully and I am desperate to make use of it and share it with others. Two surprise things:

    1) at min 3.25 in your video the authenticate twitter step – I did that, but I got no popup to authenticate on twitter – I did get the ” the yellow “running function pop up at min 3.28, but no “authorization required at min 3.28 or the subsequent twitter authentication window.

    As a “hack”, I did see on the developer page that I could create my own token right from the Twitter Developers site – “Use the access token string as your “oauth_token” and the access token secret as your “oauth_token_secret” to sign requests with your own Twitter account. Do not share your oauth_token_secret with anyone.” and I Verified the following was in my twitter profile: – Julie Archive by
    A way to archive my tweets
    Permissions: read-only
    Approved: Sunday, March 3, 2013 9:34:02 p.m.

    -If this is not correct this may explain why my spreadsheet isn’t working (see #2 below).

    2. At mins 4.10-4.15 I ran a search for the hashtag #ihateschool – i went to menu TAGS, RUN NOW! and it’s just hanging at “Running a script collectTweets Dismiss”

    And Nothing is going into the archive window.

    Any help you could offer…?
    Thanks,
    Julie


  15. Hi Martin, any idea why the Twitter application returns “Not a valid URL format” when I enter the spreadsheet url set out above?


    1. Hi Martin – First thing I’d check is there is no space before or after the url. If it’s still not working let me know
      Martin


  16. Great resource, thanks!! Do the advanced functions such as get followers work? I did try and it looked like it was working but a new tab was not created with the data.


    1. Hi Mary – the sheets data are written to are hidden be default. Click on the icon bottom left with 4 thick black lines to unhide them
      Martin


  17. I have used your TAGS spreadsheets before. I followed all the directions on your video and have the new Twitter key and secret. When I try to authenticate twitter, it flashes quickly with “running function authenticate twitter” but then nothing happens after that.


  18. First of all, thank you for this tool. It is probably the most useful twitter tool I have ever come across.

    Second, is it possible to use the advanced search operators ‘near:’ and ‘within:’ ?

    I have tried this in several different combinations with single, double and no quotes and cannot seem to get it to work.

    Thank you.


    1. Hi Scott – Unfortunately the Search API doesn’t support near: and within: It is possible to add a geocode parameter manually in the source code. If you open Tools > Script editor around line 120ish you’ll see a number of params are set eg params.count = advParams.count || 100; you can add another line with something like params.geocode = “37.781157,-122.398720,1mi”; setting your own latitude, longitude and radius (see documentation for details)


      1. That worked perfectly, thank you!

        A great tool just got even better.


      2. Can you provide any more detail on this geocode solution. I edited the code as suggested and continually come back with no results. Any help is appreciated.


      3. The geocode filter was exactly what I was looking for. Thanks for such a great tool and for the support :)


  19. Martin, this is a *great* application – as a social media user and researcher it’s great to have a customized search that I can access through Google Drive. However, I just wanted to confirm that the -7 day limit was a Twitter API issue, yes? We’re trying to pull hashtags from catastrophic events (such as the High Park Fire in Colorado, #highparkfire) but it seems that we can’t get those, as they would have been from June 2012.

    We’re able to use the “regular” Twitter search to get access to them (https://twitter.com/search/realtime?q=%23highparkfire&src=typd) but it seems that there is no way to pull these into an Excel sheet or file such as Tag v5.0?


    1. You are correct the api is limited to last 7 days. Twitter recently started rolling out the inclusion of older tweets in it’s search results but this data is currently not accessible to anyone else. Datasift have a a historics sou for older data. The data isn’t free but very cheap


      1. Thanks so much for the reply, and for the great program. I’m having some trouble getting the visualizations to show up in the Google Doc (lots of “#name” errors) but the TAGSVisualizer is pretty slick.

        I’ll follow this project often and recommend it to others – and you’ve provided our lab with an invaluable Tweet archiving tool that we will be sure to credit you with in any publications. =)


  20. Thank you Martin for doing this. Brilliant resource.

    I have tried setting it up twice and when i run it i get the following message “Exceeded maximum execution time” after a while, using different search terms.

    Any ideas?

    Alan


  21. Hi Martin,

    I’ve set up one of these archives and it went fine, but today I’ve had another go with V5.0 and I’m getting an error message.

    After Twitter API Authentication is completed (I made a new app, but I’ve also tried it with the old app), I authorise the API, put in the search term (a hashtag) and click Run Now.

    I then get asked for ‘Twitter API Authentication required’, and a red erro message, “TypeError: Cannot read property “length” from undefined.”

    Any ideas what I’ve done wrong?

    Many thanks, Su


    1. Hi Su – when you run the script authenticateTwitter does it redirect you to Twitter where you approve the connection?


          1. Yes, made a new application as per instructions.


          2. The Twitter API settings have the correct callback url (in the callback field)?


        1. Hi, I am experiencing the same issue. Have been using the tool for about a month now and suddenly it has stopped working and is giving me the error “Twitter API authentication required” — my access token and secret are set, and I have triple-checked the callback URL.

          Then today I have also started receiving the error “TypeError: Cannot read property “length” from undefined.”

          Any thoughts?

          Thanks so much!
          Kelly


          1. Hi Martin,
            Yes all the settings were fine, i went through them again.
            Then this morning I clicked Twitter API Authentication and ‘OK’ again, and chose ‘Run Now!’ and the script just worked. No idea what I did, it just worked fine this time.

            Thanks for your patience!


  22. Yes it did, though I’ve just had another go via the script editor and it just sits there when I ask it to authenticate twitter.


  23. This is happening to me too. I just set it up to try to run it again and I got the same thing: TypeError: Cannot read property “length” from undefined



  24. Hi

    A previous version worked for edges and a simple network analysis.
    I tried to fill “edges sheet” but it didn’t work out. I know NodeXL does social network analysis, but it does not update archive.

    Thanks much for your excellent work!


    1. I’m also having some trouble with the “getCommunityGraph” function for filling the edges – it does fill the edges but i can’t make out what exactly is being displayed, whether they are “following” vertices or retweet vertices.
      Sometimes i come up with a “#Value” in the cell, but can’t figure what the error is.
      Didn’t understand how the “getCommunityGraphfromsaveddata” works, as well.

      What I’m actually trying to figure out is the best way to import retweet relations into Gephi, any advice would be much appreciated!

      Besides that, i too must pay homage to the incredible work you’ve done! You rule man! Thank you veeery much!





  25. A big thank you for this great tool. Please, can the archived data be exported into Gephi (or any other network analysis tool) for analysis? Once again, THANKS!



  26. Hi Martin

    I have a problem with authentication, and I can see that is not only my problem. I did everything exactly as described, but when I go to Tools > Script Editor, and run authenticate Twitter, nothing pops up to get me to the Twitter authentication page. Hence, the app does not work. Any idea what is wrong?

    Thanks

    Hossein



  27. Hi Martin,

    firstly thank you for sharing your work, I have been using using the various incarnations of TAGS spreadsheet (v.3 – v5.05) extensively in my day-to-day analysis. Lately, I have been looking at the code closely and have a question. I am currently searching 24 @keywords and limiting the number of tweets too 100. As soon as I add the 25th @keyword I get the “Execution Timeout Exceeded”. Changing number of Tweets doesn’t make a difference. After a bit of research I came across the following link;
    http://stackoverflow.com/questions/7854573/exceeded-maximum-execution-time-in-google-apps-script
    And wanted your advice as to whether in principle/theory it would be possible to modify the collectTweets function to run in a similar way to the runMe() function in the link above. A timed batch process of var SEARCH_TERM and do it in chunks.
    Hope this makes sense, and thanks again for your contribution.


    1. Hi, quick Q are you using a single spreadsheet for you 20+ keywords? If so the issue is probably the url sent to Twitter (which will include your search term) is too long.
      Thanks,
      Martin


      1. Morning,

        Yes, you are correct, the keywords are in a single spreadsheet (below keywords take 4secs to collect&getTweets). Here is my working keywords set, = 439 Characters;

        @Universal_WE OR @mci_group OR @PMMOurWorld OR @Zibrant OR @GPJ_EMEA OR @GeorgePJohnson OR @jackmorton OR @TROexperiential OR @CarlsonWagonlit OR @Banks_Sadler OR @aileen_reuter OR @BCDTravel OR @rpmltd OR @VenuesEventMgt OR @wandoevents OR @logistikgroup OR @cievents OR @FirstProtocol OR @WRGCreative OR @ubm OR @Clarion_Gaming OR @ImaginationLabs OR @aymtm OR @BSIEvents OR @cascade_alan OR @crownlondon OR @drpgroup OR @FirstEventLeeds

        Is there a better way of analysing multiple keywords? Should I look to break this down into multiple TAGS files and try to analyse across multiple files?

        Thanks,
        Michael.


        1. I’m not aware of an easy way to get around the query length other than breaking it up over spreadsheets. It would be possible to add some extra code to do this (unfortunately I don’t have time to do it)


          1. I’ll keep it to Top 20 for now, thanks for getting back to me.

            I monitor some fairly high traffic hashtags that required me to add a couple of functions that deleted tweets based on numRows & numDays. Hope its ok to include them here in case anyone wants to create a new script file;

            function deleteOldDays() {
              // THIS FUNCION DELETES ROWS OLDER THAN 7 DAYS
              // I SET AS A TIMED TRIGGER > DAILY
              var ss = SpreadsheetApp.openById("SPREADSHEET_ID"); // Number between key= & # in current spreadsheet
              var sheet = ss.getSheetByName("Archive");
              
              var datarange = sheet.getDataRange();
              var lastrow = datarange.getLastRow();
            
              var currentDate = new Date();
              var oneweekago = new Date();
              // CHANGE THIS NUMBER TO INCREASE OR DECREASE DAYS
              oneweekago.setDate(currentDate.getDate() - 7);
                
             for (i=lastrow;i>=2;i--) {
                var tempdate = sheet.getRange(i, 1).getValue();
                
                if(tempdate < oneweekago)
                {
                  sheet.deleteRow(i);
                }
              }
            }
            
            function deleteOldRows() {
              var ss = SpreadsheetApp.openById("YOUR-SPREADSHEET-ID"); //Remember to replace this with your ID
              var sheet = ss.getSheetByName("Archive");
              var rowsToKeep = 12000; //CHANGE TO YOUR DESIRED NUMBER OF ROWS TO KEEP. 
              var rows = sheet.getLastRow(); 
              var numToDelete = rows - rowsToKeep  -1; 
              sheet.deleteRows(2, numToDelete); 
            }

  28. Hello Martin. Thanks for what you have create.

    I would create two files for different keywords. Do I just need to make two copies in putting different keywords inside it and it works or there is another manipulation? I’m afraid that the two files with the same settings and authentication are not distinct. I need to do two separate searches with different keywords. It is for this that I want two separate files.

    (Sorry for my english, I’m french)

    Thank you !


    1. Hi, you can use the same file and in the search term use OR. For example, cats OR dogs



  29. Hi Martin,

    I’m getting the following error on V5:

    Line 157 Cannot read property “followers_count” from undefined.TypeError

    I am able to test the collection successfully.

    Any ideas why that might be happening?

    Thanks,
    Bill


    1. Is this using a copy of the template or trying to update an existing spreadsheet?


      1. A copy of the template. I created it by clicking on the *** Twitter Archive Google Spreadsheet – TAGS v5.0 *** link.

        Thanks,
        Bill


  30. Hi Martin,

    I’m getting the following error on V5 after running the authenticateTwitter function on the script editor:

    Request failed for returned code 404. Server response: {“errors”:[{“message”:”Sorry, that page does not exist”,”code”:34}]} (línea 376, archivo “Google(c) Code”)

    I have already authorised permission with my twitter account.

    Do you know why that might be happening?

    By the way, I have another problem with the conversation visualization in TAGSExplorer. I have already publish the Google Spreadsheet in the Web but when I try my URL in the TAGSExplorer, the web is loading for ever with the message “Loading data”. I think that there aren’t too many tweets to be loading so slowly… (only 246), so I suppose it may be another thing… Any idea to solve this?

    Thank you very much and good job!

    Victoria


  31. Hi, thank you for TAGS, you may save me weeks of work!
    I’ve followed the instructions (got special menus, authorized the app), but when I click on “run now!” it just says “Running script collectTweets Dismiss” and stays like this, i can still do things in the sheet, but it doesn’t give me any tweet in the archieve.
    What can I do to make it work?


    1. Hi, As far as I can see this is to do with Google infrastructure and have run out of optimisation solutions. For some reason on the initial collection I find better performance if I open tools > script editor and then Run > collectTweets. Once an archive is running they generally update without any fuss. Seems to be an issue around when the spreadsheet is first copied.


      1. thank you, I just opened the script editor and ran the collectTweets from there, a popup asked the app authorization and after it started working normally


  32. Hi there, can I check if this only pulls in organic tweets and not promoted tweets?




  33. Great job, @Martin and thank you for sharing it with the world!

    Couldn’t find an answer to this, so asking here- does TAGS support a combination of logical operators for the search term. Specifically, I am looking for something like: a AND (x OR y OR z).

    Cheers,

    Ned


    1. Hi Nedyalko – yes you can use combinations of operators. One thing to watch is your query string doesn’t get too long (the maximum is 1,000 characters – this includes some parameters sent by the script so it’s a little less than this)
      Martin



  34. I use your script, but I wanted to create new one and it seems that something is wrong. I have only “running script” but no data is pulled from TW. Do you have any idea?


      1. I’m having problems with this, too. I’ve authenticated several times, but cannot get the custom menu to install. It doesn’t matter how many times I click on the button, give authorization, and click “accept”, it does exactly the same thing, and I just end up in a loop. Baffled.


          1. I had the same problem with the message “running script” and no data pulled from the spreadsheet. In order to solve it I tried to reload the spreadsheet, then went to Script Editor and authenticated again. Then closed the editor and from the spreadsheet I re-ran the script to get the data, and this time it worked like a charm!





  35. Thanks for this awesome tools! :D
    Tweet mining is so cool too…


  36. Hey Martin,

    I just have a question about rate limitations. I know that it is possible to search up to 15,000 tweets or tweets from the past 7 days (which ever limitation is reached first), but would you be able to tell me if any further rate limitations exist specific to this software? For example, I know that I was having a problem at one point where I made too many searches in one 24-hour period, so my searches were unable to run any further. Is there an exact rate limitation in this regard? Maybe the 15K/7-day rate limitation applies in this scenario? Thank you!

    All the Best,
    Brendan


    1. Hi Brendan – the api docs state 180 calls per hour. The script fetches 100 tweets per call. TAGS Advanced menu has a ‘Test Rate’ menu option for you to monitor remain calls


      1. Ah, perfect! Exactly what I was looking for; thank you!


  37. Hi Martin – I’ve been experimenting with the TAGS v5.0 script to explore how to archive our #cdnedchat sessions. I have had trouble getting the archive to show times during the turnover from end-of-day to next (23:59:00 June 3rd to 00:59:00 June 4th) in the search.

    I’ve tried changing the settings up (2 days, 1 day, 3 days) but that particular hour is just not showing up in the search. As it happens, that’s the exact hour when our chat happens online.

    Any way for me to fix this? What do I need to adjust in the readme/settings?

    Thanks!


    1. Okay, I went back to Tools, Script Editor and I ran the collectTweets function and now it is working – all tweets are showing up. Just have to figure out how to get the analytics to work with the “paged” archives. Slowly but surely! Thanks for such an amazing resource!


      1. Hi Tanis – glad you got it working. The paged collection is a feature from the very early versions of this template and I haven’t really been looking after it. My recommendation would be to use the single archive version. If you want to give users easy navigation per chat the MLAA developer TAG Viewer is a great option. Here’s an example on how the TAGS sheet for #cdnedchat is rendered. Alternatively there is this searchable interface I’ve created

        Martin


        1. Brilliant! Thanks for the suggestions regarding archiving multiple chats in one archive – love these resources you have created!



  38. Thanks for this great tool. Really a brilliant thing. I also got a question. I just ran a sweep but was wondering why it doesn’t seem to go very far back in time. Is there a way to make it sweep tweets from the past few days? Right now it only goes back for about a couple of hours even though I know that there is much more. Any ideas?
    Thanks again.
    Micha



  39. Since yesterday the script is not working. There is a problem with the variable followers, that I am suppose that is due to the retirement of the API v1. I have been using your script for several months now, and I wanna thank you for that.


  40. Martin this is the error that I get:

    Apps Script

    Line 157 Cannot read property “followers_count” from undefined.TypeError

    I tried to set up a new gsheet but I get the same error. It was reported that twitter has made some changes on how to get the followers, but I didn’t have so far the opportunity to search for more details.

    Thank you again, for your work.


  41. Thanks for Tagsv5.0! It is of immense importance to me. I am PhD at the Wageningen University & Research centre (the Netherlands) and I am doing a research on the influence of twitter-acitivty on the governance of sustainability in agro-food systems. I’ve made five documents for archiving tags, but now I encounter two problems: 1) only my first two are working (1 shared, 1 private), the others do not archive data, and 2) Tagsexplorer isn’t working for any: it says ‘loading data’ (for 8 hours).. I hope you can give me any direction for solving these problems. Keep up the great work!


  42. I am facing same error as M.M. Script is not working. Plus, I get a redbox saying something about the length property.

    Thanks for your work


  43. Thanks for this great tool. I am a PhD researcher and this is of great value to me!

    If you have problems with TAGSExplorer: it might be because you’re using Internet Explorer, try Google Chrome instead. It worked for me :)

    I can’t make new archives. I’ve two running properly, collecting about 40 tweets a day for a few weeks now. When I make a new one and click ‘run now’, it says

    Apps script:
    Twitter API Configuration Required
    And then, when I close that window, it says in red:
    TypeError: Cannot read property “length” from undefined

    I’m sure I’ve followed all steps, including the Twitter API authentication procedure (you can use your key and secret multiple times right?) and authenticate Twitter via Tools, script editor (you need to do that for all docs?). I haven’t archived many data yet, so I guess it’s not the amount of data I am pulling in. Is there a limitation for the amount of archives to have running, or the times you do the Twitter API authorization configuration, or just for the amount of data pulling in?

    I hope someone can help me with this somehow. Many many thanks


    1. Problem Solved: When I did the Twitter API authentication via Tools > script editor > run > authenticate twitter it worked. I guess I did something different before..


      1. Thanks! You saved me a good amount of head scratching!

        This tool is wonderful! I’ve popped it on my blog.


  44. Any suggestions for getting historical data in the same format? I need very little and basic data starting from November 2011 with the same search term as I used with TAGS. Anybody already done that? I would love to hear any suggestion. Thanks a lot.


  45. Hi, I’ve just made a copy of version 5.0, but when I click on the ‘enable custom menu’ button a window opens and says, “Oops, Script function onOpen could not be found”

    Is this a problem with the software or is it my google drive causing it to do that?


    1. Hi Andy,

      Google sometimes needs a couple of seconds to catch up with the template copy to attach the script. The advice I’ve seen elsewhere is to keep hitting the damn button until t works ;)



  46. Thank you very much for this! I hope this is not a silly question but only the first search term I enter shows up in the archive. Is it possible to enter more than one search term or do I have to do separate sheets for them? Also, my understanding is that Twitter allows for searching further back than 7 days. Is this a possibility here? Thank you very much again!


  47. Hi Martin,
    Is there an option for language filter in the archive? I’ve notice the French are using the same hashtag as us. If I could filter out the french language tweets…


    1. Hi Su – requires some code tweaking. In the Tools > Script Editor change the lines around number 87 from

      if (sinceid){
      var data = getTweets(SEARCH_TERM, {"sinceid": sinceid}); // get results from twitter sinceid
      } else {
      var data = getTweets(SEARCH_TERM); // get results from twitter
      }

      to

      if (sinceid){
      var data = getTweets(SEARCH_TERM, {"sinceid": sinceid, "lang": "en"}); // get results from twitter sinceid
      } else {
      var data = getTweets(SEARCH_TERM, {"lang": "en"}); // get results from twitter
      }


      1. Hi Martin, that’s great – thanks for letting me have the code. I’ve made the changes and can’t see any more French tweets. I’ve deleted the ones from the weekend.

        I’m liveblogging the Government’s Construction Summit tomorrow – will be also promoting TagsExplorer and TagsArchive to help non-tweeters see what’s going on.




  48. Hi, very awesome script and interface! I’ve ran into a problem. I set up a couple of scrapes for my thesis research. All working fine, but one isn’t working and states dates from 30/12/1899 till 30/12/1899. What could this be?


  49. Hello Martin, when i run the time-driven triggers to collect tweets .. is there a way to filter the new ones from the old ones ? Basically .. i want to be able to capture all tweets on a file between 1pm – 3pm and another separate file from 3pm – 5pm and a separate file from 5pm – 10pm & so on and so forth . .. is this possible to do ? Or does TAGS just update the tweets to the spreadsheet ?

    ALSO, is there a limit to how many tweets it looks for ?

    Lastly, on the spreadsheet .. the times it says the tweets have been posted are incorrect as to when i open my tweet deck. How do i change the time zone so i see the correct time same as i see on tweet deck ??

    The time is off by like 7 hours or something like that ..

    Thanks,
    Bill


      1. This is very helpful!

        One question; how can I change the time zone so the times are accurate for EDT (the time for New York City)?

        Perhaps an ideal would be if there were a way to have the date in one column (in US format, which would be 7/12/13 for July 12, 2013) and the time (in EDT) in another column? Would this be something easy to adjust?

        Thanks for this wonderful script!


        1. As a couple of people have asked. One way to get different timezone date display is in the Spreadsheet go to File > Spreadsheet settings and select date locale and timezone and then in Tools > Script editor open File > Project properties and change the timezone there as well. This should convert any existing and all new date/time data in the time column. The created_at column is unchanged as this is the metadata passed back from Twitter and required for tools that visualise a TAGS archive





  50. Hi Martin,

    Thanks for the amazing tool. I have one question though. How can I collect tweets based on geolocation? I want to collect some data from Bucharest and Brasov, Romania, but not by using hashtags. Is it something that I need to modify?

    Thanks,
    Cristina




  51. Brilliant tool – thank you, Martin. I had been using “The Archivist” to archive #globalclassroom twitter chats but, now that it’s subscription only, your script is the perfect replacement!


  52. Fantastic tool, thanks Martin! I’m trying to figure out a way to strip line breaks out of the tweets when they are imported to the Google Docs spreadsheet.

    I’m exporting this data via a web query to Excel and line breaks in Tweets cause the text to broken in to separate cells.


      1. Thanks for your guidance Martin. I have managed to strip line breaks from the Tweet text by adding some code to the TAGS.gs script. Around line 187 (after //end of while loop), at the end of the gettweets function I added the following code:

        // remove line breaks

        for (var i = 0; i < data.length; i++) {
        data[i]["text"] = data[i]["text"].replace(/\n/g, " ");
        }

        Perhaps this might be a useful option to include in future releases?


        1. Thanks for this. Given the workflow Richard has highlighted this makes sense as an option.


  53. Hi Martin,

    This question has been asked before by Bill but I dont think found the answer to it. Is there a way just to scrape the tweets in a particular time frame. That is for instance from last Sunday 10pm to 11pm EST US time or Monday 2am to 3am UTC? When I started my search last night at 11.20pm I o