Keep your Twitter Archive fresh on Google Drive using a bit of Google Apps Script

Update: There is a newer take on this solution which dumps Google Drive web hosting and uses Github Pages instead. You can read more in Keeping your Twitter Archive fresh and freely hosted on Github Pages.

Important: A noted in the comments Google announced last year that it is killing Google Drive web hosting which this solution replies on. The good news is I’ve come up with a solution that lets you keep your archive on the web. If you’ve got an existing archive the only change is a new url  https://script.google.com/macros/s/AKfycbwrXr8ejYjHwGEO6kj8f4WHIh096ARDRHdNOgAXPqGltoa80FU/exec?folder_id=YOURFOLDERID (e.g. my archive is at https://script.google.com/macros/s/AKfycbwrXr8ejYjHwGEO6kj8f4WHIh096ARDRHdNOgAXPqGltoa80FU/exec?folder_id=0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ) Your archive is be a bit slower to load and will have a bar at the top to say it’s not a Google application. I’ve updated the template to include the new url.

Twitter Archive interfaceLike a growing number of other people I’ve requested and got a complete archive of my tweets from Twitter … well almost complete. The issue is that while Twitter have done a great job of packaging the archives even going as far as creating a search interface powered by HTML and JavaScript as soon as you’ve requested the data it is stale. The other issue is unless you have some webhosting where can you share your archive to give other people access.

Fortunately as Google recently announced site publishing on Google Drive by uploading your Twitter archive to a folder and then sharing the folder so that it’s ‘Public on the web’ you can let other people explore your archive (here’s mine). Note: Mark Sample (@samplereality) has discovered that if you have file conversion on during upload this will break your archive. [You can also use the Public folder in Dropbox if you don’t want to use a Google account]

The documentation wasn’t entirely clear on how to do this. Basically it seems that as long as there’s a index.html file in the folder root and links to subdirectories are relative all you need to do is open the folder in Google Drive and swap the first part of the url with https://googledrive.com/host/ e.g. https://drive.google.com/#folders/0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ becomes https://googledrive.com/host/0B6GkLMU9sHmLRFk3VGh5Tjc5RzQ/

So next we need to keep the data fresh. Looking at how Twitter have put the archive together we can see tweets are stored in /data/js/tweets/ with a file for each months tweets and some metadata about the archive in /data/js/, the most important being tweet_index.js.

Fortunately not only does Google Apps Script provides an easy way to interface Drive and other Google Apps/3rd party services but the syntax is based on JavaScript making it easy to handle the existing data files. Given all of this it’s possible to read the existing data, fetch new status updates and write new data files keeping the archive fresh.

To do all of this I’ve come up with this Google Spreadsheet template:

*** Update Twitter Archive with Google Drive ***
[Once open File > Make a copy for your own copy]

Important: Google have changed some of their backend. If you took a copy of this template prior to 8th Dec 2014 the script will fail with a Script is using OAuthConfig, which has been shut down. Learn more at http://goo.gl/IwCSaV (line 277, file "Code"). To get your archive running again the best solution is to copy the template and setup using your existing Drive folder and API key/secret.

Billy has spotted an inconsistency with the way Drive searches for the files used in the script. This has been addressed in an update to the code but existing news will need to either take and setup a fresh copy or open their existing copy and then open Tools > Script editor and replace the code with version here.

Note: There is currently an open issue which is producing the error message ‘We’re sorry, a server error occurred. Please wait a bit and try again.’ Hopefully the ticket will be resolved soon

The video below hopefully explains how to setup and use (Update The script uses a new authentication flow which makes setup easier (and API registration is not necessary if you already use TAGS v6.0):

A nice feature of this solution is that even if you don’t publically share your archive, if you are using the Google Drive app to syncs files with your computer the archive stays fresh on your local machine.

The model this solution uses is also quite interesting. There are a number of ways to create interfaces and apps using Google Apps Script. Writing data files to Google Drive and having a static html coded based interface is ideal for scenarios like this one where you don’t rely on heavy write processes or dynamic content (aware of course that there will be some sanitisation of code).

It would be easy to hook some extra code to push the refreshed files to another webserver or sync my local Google Drive with my webhost but for now I’m happy for Google to host my data ;s

169 Comments


  1. I tried making a copy of the spreadsheet, but that option was not available to me.

    Reply

    1. Hi Jedidiah, first thing to check is you are logged in to your google account when you open the spreadsheet

      Reply

      1. Hi Martin – I was logged into my google account. I have tried it since and been able to make a copy. Thanks.

        Reply

  2. I’m not getting any action, have triple checked settings. U’ve entered my folder ID twice but the status says “Folder ID not entered yet” When I runt he script, it just stays at Running script updateArchive – it looks like its not getting my folder ID?

    Reply

    1. Think it was my dodgy coding not updating the location on the readme. If you open the Log sheet has it recorded any updates?

      Reply

    1. [closed – for some reason authentication with Twitter didn’t work 1st time. Added some extra UI to help]

      Reply

  3. Looking for a link to archiving twitter on a website (other than on Google Drive)…

    Reply



  4. hi

    am getting Unexpected error: (line 35) when i try to authorize function?

    var result = UrlFetchApp.fetch(“http://api.twitter.com/1.1/account/verify_credentials.json”, requestData);

    thanks in advance

    Reply

    1. Sounds like you API key/secret are not set properly. Make sure there’s so spaces before/after them when you enter them into the configuration dialog (I’ve added a trim to prevent this in the future)

      Reply

      1. hi again

        used your new spreadsheet template and now getting a different error message says:

        Apps Script

        Unexpected error:

        have tripled checked the api key/secret.
        and ideas?

        thanks

        Reply

        1. ak ignore last message working now, i forgot to add the callback url, doh!
          thanks again great script!

          Reply

  5. Holy smokes, this is great! I had trouble getting it to work at first, and I wanted to share the solution, in case your readers encounter the same problem. Depending upon one’s Google Drive settings, your Twitter archive folder might be converted to “Google Docs” format during the uploading process. This breaks your files and makes them unreadable (as I found out!). So make sure that in your Google Drive upload settings, you have all the file conversion rules turned off.

    Here’s my own now-working archive: https://googledrive.com/host/0By7OircJ9labZktFR0xkT1ExUXc/

    Reply


  6. Wow, great. That even worked for a coding/web-admin dummy like me. Thank you so much. Because I was too stupid to set up Tweetnest.

    Reply

  7. Hi Francisco George,

    I had the same problem – if you do as Martin suggested and Run > authorize part of the script, you’ll get a dialog box (can’t remember what it said now) with a button. If you click the button, it takes you off to Twitter, and you’ll be able to authorize the script there.

    It fixed the problem for me at least – hope it works for you too.

    Nice work Martin!

    Reply

  8. Hi Martin,

    Thanks a lot, it worked!

    I somehow din’t quite understood this step…and now with your hint everything worked fine.

    You should maybe submit your solution to Twitter, it would ease their load if by having this solution implemented as they would not have every one downloading their archive too frequently.

    Very nice work!

    Francisco

    PS: is there anyway to change the frequency of the automatic update and set it for a for a every 8 hours update for example?

    Reply

    1. UX isn’t my strong point. I’m not sure what Twitter would make of this ;s

      [You can update frequency by opening Tools > Script Editor and then Resources > Current script triggers. In the dialog box you should see a function set to fun daily. Just adjust the frequency from one of the drop downs and save. I wouldn’t recommend going any lower than every 15 min to avoid api/usuage limits]

      Reply

      1. Hi Martin, sorry if I’m being dense but what is the current update frequency set at – daily? I tried the route you mention above Tools > Script Editor > Resources – I don’t have ‘current script triggers’, instead I get the option ‘current project triggers’. The dialog box this brings up shows one trigger but it isn’t editable at all and I can’t see anywhere that mentions update frequency

        It’s not the end of the world – I would be happy with once a day if this is what it is

        Btw, love it. Thanks very much :)

        Reply

        1. Hi Patrick – in the spreadsheet there should be a Sync Twitter Setup menu where you can toggle the auto-refresh

          Reply

          1. Hi Martin, thanks I found that in the end but thanks for getting back to me. Very chuffed with this – great bit of work, well done :)



  9. Is the formatting of the added (updated) tweets exactly the same as the original Twitter Archive? I had a script written to parse the original archive JSON by stripping the first line (per the README in the archive) but that seems to fail with the updated archive files. It seems that in the original archives, the first line reads as “Grailbird.data.tweets_2013_01 = ” whereas yours outputs “Grailbird.data.tweets_2013_01 = [” with that opening bracket appearing on the first line rather than the second. It’s probably a simple fix to your script but it does break JSON parsing if the first line is stripped.

    Reply

    1. ah hadn’t spotted that. I’ve updated the template. The changes are line 109 which is now

      tweet_file.replace(“Grailbird.data.”+var_name+” = \n”+ JSON.stringify(Grailbird.data[var_name], null, ‘\t’)); // replace old file

      and line 116 which is now

      new_tweet_file.replace(“Grailbird.data.”+i+” = \n”+ JSON.stringify(newData[i], null, ‘\t’)); // replace content with new data

      Reply

  10. So I think something may be broken now that a new month has rolled around. The script generated a “2013_02.js” file in the root of my Google Drive rather in the appropriate directory which resulted in future updates failing.

    Reply

      1. Very weird. I moved the file back into the appropriate location and everything seems to be working fine. Guess I’ll find out next month if it’s actually a problem.

        Reply

    1. but you are right too, the file “2013_02.js” appears in my root too but along it, it says it is located in the “Tweets” folder(the public folder I use)

      Reply

      1. Appeared to work okay for me. Like @paco229 the new file appeared in the root directory and data>js>tweets (which is surprising as I thought Drive had done away with files having multiple labels/locations). I’ve added an extra line in the template to prevent this. For existing copies of the script insert the following line after line 115:

        new_tweet_file.removeFromFolder(DocsList.getRootFolder());

        so it should look like:

        new_tweet_file.addToFolder(tweetsFolder); // move new file to the data/js/tweets/ folder
        new_tweet_file.removeFromFolder(DocsList.getRootFolder());

        Reply



  11. hi

    my archive is giving me a 404 not found error.

    when i try to manually update archive using your script i get this error message:

    TypeError: Cannot call method “getContentAsString” of undefined

    thanks in advance!

    Reply

      1. hi
        had to eventually upload new Twitter folder, and set share permissions to “Public on the web – Anyone on the Internet can find and view”

        thanks

        Reply

    1. Hi Willem, search is done via the browser (no clever indexing). When I tried doing a search in your archive the console said the file data/js/tweets/2008_05.js was missing
      Thanks,
      Martin

      Reply

      1. Thanks so much, you found the bug! Actually, the file was not missing, but existed two times! Ni idea how that happened. May be a hickup whith ftp.
        Anyways, search works now thanks to your attention.
        Lots of people here in the Netherlands tried your script already. It’s been retweeted quite a lot!

        Reply


  12. When starting “update Archive Now” I’m getting to see (in Dutch) an error message telling me that function getFolders cannot be found in object false. There the script stops. What’s going wrong ?

    Reply

  13. How to fix this Twitter Api not configured???

    Reply

    1. did you get a twitter api key and secret? If you did, did you add them via the Twitter sync menu? If you did, did you open Tools > script editor and run authenticate?

      Reply

      1. Yes, I took all these steps without getting any error message, everything went smooth as far as these steps are concerned.

        Reply

          1. what happens if you run the authenticate function again? (In particular does a dialog box appear taking you to Twitter to authorise the connection or nothing)


          2. Harmen – I’ve confused myself my responses were supposed to be for Mike. Your problem sounds like all the folders/files aren’t uploaded to Drive properly


      2. Sorry, I don’t know how to get the key and secret, I stuck there. I followed the instructions on video, by clicking authorize and run, but it appears a box – Twitter Api not configured.

        Reply

        1. In the spreadsheet menu bar there should be a Sync Twitter Setup menu option. From it click API Authentication for instructions on setting up the API connection

          Reply

          1. Thanks a lot! I just realized I haven’t create that things, that’s the reason I can’t authorize the app. Everything is fine and my twitter archive is able to be updated now. Thank you again. =)


          2. Between, it only updated the latest tweets, how about some old tweets that I have deleted, but it still appear on my archive?


  14. Hi, Martin. Great script and easy to set up. Thank you for sharing it. I credit you on my website.

    Reply


  15. Is it possible to edit the title tags within the index.html & how would I go about using a bespoke address instead of the long googledrive address.

    Great script and so easy to set up.

    Thanks

    Adam

    Reply

    1. Hi Adam, I’ve done some minor editing of the index.html for my own archive but haven’t dug that deep. I also use a redirect on my webserver http://tweets.hawksey.info (doesn’t cloak the address, I’m sure there are ways of doing it)

      Reply


  16. Hi Martin,

    I’ve tried this a few times, followed step by step and to a T, but every time I get an “oAuth Error” after running the “authorize” script. Any ideas?

    Reply

    1. i had the same problem. right click on “My Drive/tweets” folder
      click “Details”
      then edit the Sharing visibility and change the folder from private to public.
      if u succeeded you’ll see a small word beside your folder name “shared”
      then its done

      Reply

  17. Running this since weeks and I have to say I’m very satisfied with it. The search is great and really works. But there’s one question I worry about: Will this still works, when Twitter will change to its 1.1 API completely?

    Reply

    1. Fear not – when I wrote the script I was aware of the impending changes so it entirely uses version 1.1 of the API

      Reply

  18. I hope this isn’t a cheeky question, but does anyone know how I would go about putting my own Twitter background on this for when people are searching tweets? At the moment it’s pretty uninspiring and blank. Thanks

    Reply

    1. Easiest way is to change the img/bg.png image. For more tweaking most of the styling is done in the css/application.min.css file. It’s minified so not easy to navigate (sites like http://procssor.com/process can decompress it if you are doing lots of edits)

      Reply

      1. Phew, you were right about it being minified! Thanks Martin I’ll have a bash and thanks too for keeping up with everyone’s comments, it’s great style. Much appreciated

        Reply


  19. Hi, it’s me again! I would like to ask you whether that’s a way to remove DELETED tweets on twitter archive from google drive?

    Reply

    1. Easiest way is to re-request archive from Twitter and replace the files. The script will just continue updating the data files it has available

      Reply

  20. Hi Martin,

    Since this morning I’m getting this error report. Any advice?

    Details:

    Inicio Función Mensaje de error Activación Fin
    16/03/13 19:51 updateArchive TypeError: No se puede leer la propiedad “id_str” de undefined. (línea 95, archivo “Code”) time-based 16/03/13 19:51

    Reply

    1. The error code from Appscript has changed 3 hours ago now it states

      17/03/13 6:51 updateArchive ReferenceError: “tweet_index” no está definido. (línea 80, archivo “Code”) time-based 17/03/13 6:51

      Reply

      1. Hmm looks like your archive files have become corrupted (I’m guessing a write process wasn’t completed properly). Easiest solution is to request your archive from Twitter again and replace the files on drive. If you use the same folders setting up the script again is not required

        Reply

    1. Hi Rodney – I know a number of graduate researchers swing by this blog and I’m sure they find it a valuable resource :)
      Martin

      Reply




  21. Hi, Thanks for this wonderful guide. I have some issues though. When I get to the API Configuration step, I am not sure what to put in for the Application Details Page. I see: Name, Description, Website, and Callback URL. What is the website we should set to?

    Also will this work with a private twitter account? I don’t want to share my tweets and want my own personal archive.

    Thank you!

    Reply

  22. Hey, I have downloaded twitter archive several times, it was complete. However, my recent download was incomplete, it started at my latest 3000 tweets. Why it happened to me?

    Reply

        1. This sounds like a question for Twitter. I’ve no control over what is in your sorehead from them

          Reply


  23. I’ve just figured out how to create a twitter archive page hosted by Google Drive thanks to your great article and video. Just a quick question- is it possible to insert/include the actual web page- the one that the twitter archive is hosted on- in a new Google Document or Spreadsheet on Drive? Thanks for any advice!

    Reply

  24. When starting “update Archive Now” I’m getting to see (in Dutch) an error message telling me that function getFolders cannot be found in object false. There the script stops. What’s going wrong?

    Reply

  25. dear martin
    I tried above and seemed to get things working but not viewable. what i mean is that the twitter data was upadating each time I tweeted – the new tweets were supposedly being added but I couldn’t open the archive to see these no matter what I tried.
    I reloaded the archive up again and even upgraded my google apps account to make it possible to change settings of folders/files to public (did you know you could not do that on a google apps free edition?). I am now getting the same error as Seungow
    Cannot find function getFolders in object false. – despite going through all the steps of the spreadsheet.
    If you could shed any light on this I’d be grateful. Not so au fait with scripts unfortunately.
    thanks, Helen

    Reply

  26. please forget my previous problem. everything is working now and i think it’s amazing. have to get my head around how great this is and how much it can help my clients.
    you are a genius!
    thanks….

    Reply

  27. This is absolutely one of the most useful tools I use. Thanks!

    Has anyone tried using this system to add additional twitter handles to the tweets that are displayed? I currently archive my own tweets, but want to add my family members in one central archive. I may try on my own, but thought I’d see what other people thought of my chances before I destroy my current archive. ;-)

    Reply

    1. From recollection it is feasible but would require rewriting the collection routine. Another way to do it might be to use this with the search operator ‘from:mhawksey OR from:dhkeller’ etc. (without quotes) The display of data however isn’t as pretty and it won’t be a fully archive

      Reply

  28. Thanks for the fabulous tool, been using it for the last few months & it was working like a charm..

    Now for the past couple of days, I’ve been getting this error emails from Google drive

    Your script, Copy of Sync Twitter Archive v1.0, has recently failed to finish successfully. A summary of the failure(s) is shown below. To configure the triggers for this script, or change your setting for receiving future failure notifications, click here.
    The script is used by the document Update/Host Twitter Archive with Google Drive.
    Details:
    Start Function Error Message Trigger End
    11/20/13 12:11 AM updateArchive TypeError: Cannot call method “getContentAsString” of undefined. (line 73, file “Code”) time-based 11/20/13 12:11 AM
    Sincerely,
    Google Apps Script

    I went into the spreadsheet & tried to do a manual refresh, it doesn’t work with the same error. I am no expert by any means but it seems either the Twitter API or Google Apps API has changed?

    I’m just asking if you have encountered the same error and if you plan to update the sheet to fix it. :)

    Thanks and keep up the good work.

    Reply

    1. Hi Michael, I know of one other recent case similar to yours. It looks like a glitch Google’s end. Usually it appears to clear itself and start working again. One thing you can do is search your Google Drive for a file called user_details.js. If this is missing or corrupt the script won’t work. The fix is to get a fresh copy of you archive from twitter and replace the file (it’s in the data/js folder).
      Thanks,
      Martin

      Reply

      1. Heya Martin. Thanks again for this excellent resource. It’s really amazing, and I love being able to host an automatically updated archive of all my tweets.

        I’m having the same problem as Michael. I’ve tried replacing user_details.js, as you suggest, but I’m still receiving the same error. Is there something else I can try, or should I just delete everything and start again from step one, hoping that a fresh go will fix this odd and out-of-nowhere problem?

        Reply

  29. Hey Martin,

    This has been working great all year, and about 5 days ago I started to get this error when it tries to update:

    ‘TypeError: Cannot call method “getContentAsString” of undefined’

    I’ve requested a new backup from twitter but it seems to be taking longer than usual. Any ideas what’s going on?

    So far I have deleted my old set of files and tired starting from fresh but I still get the same issue:|

    any help would be appreciated.

    Cheers,

    Mikey.

    PS. Would there be any way to get it to download any images we post? I understand there might be storage issues after a while, just curious

    Reply

    1. Hi there,
      I have the same problem as Mickey and Michael. “TypeError: Kan methode getContentAsString van undefined niet aanroepen. (regel 73, bestand ‘Code’)” Is there a fix yet?

      Reply

      1. *UPADTE*

        Right, I’ve’ve finally received a new archive from twitter. I deleted everything and started again. Same issue persists. Over 4 weeks now so I’m not sure is it’s a permanent thing with drive or not :(

        Reply

      2. Hey Everyone having the “TypeError: Cannot find function getContentAsString in object” error, I figured I would put that string in for people searching for this FIX.

        First off, thank you Martin for the amazing script and idea, I love my Twitter archive, but just like many people I’ve seen commenting, it hasn’t been working for a little while now. I was having the error absolutely every time, so I decided to try some things out, and I figured out the cause of the problem, and a workaround for now, but I thought I would post things here to see if we could fix it properly.

        Around line 70 of Code.gs, there are the json calls to bring in the data from the existing archive, starting with this:
        var user_details_file = js.find(“title:user_details.js”)[0];

        The issue is, for some reason, the built-in .find() function is not liking the ‘title’ part of the query. If you remove ‘title’ from those three .find functions, and the one further down (around Line 90, just look for ‘find’ and ‘title’ near each other), the queries still complete, and the archive works properly again!

        Any idea why Google Scripts isn’t accepting the ‘title:’ argument for a search anymore?

        Reply

        1. Billy – Big thanks for taking the time to look at this and finding the bug. Seems like this is inconsistent as my archive has been happily updating using the tite: search. I guess it must be the way Google indexes the search in Drive.

          I’ve updated the code to try an take account of this (untested). Unfortunately this will only take affect for new copies of the script. Existing users experiencing problems should either take a fresh copy or open the new version then open Tools > Script editor and copy the code across to their own version.

          Thanks again,
          Martin

          Reply

          1. Looks like a good solution to me, but it looks like the .find on line 92 of your Gist is going to toss the same error. Since I’m finally playing around with git again after a little while, I quickly forked your code:
            https://gist.github.com/WillPresley/7772938/revisions

            Since I was having the issue constantly, I figure I’m a good test-case, and your updated code (with my tiny edit) works perfectly!


    2. I’d also be very curious about how to get the Twitter images pulled. The self-hosted archive is definitely cool, but as long as the images themselves are still hosted on Twitter rather than in my GoogleDrive, the “archive” is incomplete. Anybody gets it working, let me know! :-D

      Reply

  30. @Billy – well done mate. Went in and edited it manually (been years since I’ve gone anywhere near a piece of code) and it worked a treat.

    @Martin – I didn’t realise how much I rely on this to search old tweets. A big thank you for spending the time figuring out how to do it. I’ve been looking into ways of getting uploaded images (twitpic/yfrog/twitter etc.) and archiving them. Doesn’t appear to be any where near as simple as I thought it would be. Think I’ll leave it until I have some time, over Xmas maybe.

    Anyway thanks again to the both of you.

    Mikey

    Reply

  31. Hi Martin,

    After a looooong time working smoothly since 24h I get every hour(my refresh rate) this message from Google:”TypeError: No se puede llamar al método “getContentAsString” de undefined. (línea 73, archivo “Code”)”

    Is it a google error once again?

    Reply

  32. @Martin,
    Since two days I get an error each from Google that the script fails.
    What can I do to repair it?
    I really love my archive and use it a lot, so I hope this is temparary?
    Hope (and trust:) you can help me out…

    Reply

      1. @Martin,
        Thank you so much again!
        I had overlooked the updated script.
        Now works like a charm again.
        Cannot tell you how happy I am since I use my archive a lot.
        Many people are jealous of what I have!
        Of course I always point them to your script.

        Reply

  33. @Billy & Martin
    Modified Script works fine. Thank you for your attention.

    Reply

  34. I have secured a new archive, and am recreating my “updatable” archive using the new code from GitHub. But now I am getting a persistent “Twitter access has not been authenticated” error. This is *after* I had authorized it using the “authorize” function, which did accept a authorize click from Twitter. My consumer key/secret have been triple-checked, but nothing.

    But —> Now I see a message from Twitter on its OAuth settings page that says “api.twitter.com now requires SSL/TLS for all connections as of January 14th, 2014. | Read more →”

    Red More: https://dev.twitter.com/discussions/24239

    Could this be the problem?

    Reply

  35. Hi Martin,

    Just so you know: the ‘script failure’ is back:

    ReferenceError: “tweet_index” is not defined. (line 81, file “Code”)

    Reply

  36. It ran a long time smoothly, but now I got a ReferenceError: “tweet_index” is not defined. (line 80, file “Code”). What can I do about it?

    Reply

    1. Seeing this error myself – looking into a fix

      Reply

      1. Did you found a solution? I now update the file manual once a week. Doesn’t take much time, but fixing the automated solution would be awesome.

        Reply

  37. first i have to thank about this feature! but i have a trouble. when i click ‘update archive now’, after some secs, google messages “File exceeds the maximum file size.” and i cannot update it. I think this is related with google drive but i don’t know well. how could i fix it? thanks.

    Reply

  38. Thanks so much for this. I’ve been using this method since you first posted it. I do have a question though.

    When viewing my archive, any tweet made with an avatar other than the most current one shows the avatar as a broken jpg. Looking through your archive, it seems that it doesn’t do this for you, so I’m wondering if there’s any way around it.

    It’s not the end of the world, and I obviously don’t care if the CORRECT avatar is shown for older tweets, but I’d love it if my current avatar showed for all the tweets in my archive, not just the newer ones.

    Oh, and for the record, this also happens with retweets; if someone has changed their avatar since I retweeted them, the retweet shows a broken jpg as well.

    Any help would be appreciated! My archive can be found at http://napor.us/twit

    Reply

    1. hi – I’m not aware of a way around this. It would be interesting to see how if you re-exported your archive from Twitter if they use the latest avatars

      Reply

  39. Hi Martin – Tried to comment earlier but not sure if my comment is in moderation or just didn’t go through, so trying again:

    My Twitter history goes back to 2007, so the right-hand column now takes up the entire height of a smaller screen (e.g., laptop or netbook). I have text and a ‘follow’ button underneath the list of months/years, but one cannot see these on a smaller screen – scrolling down only scrolls through the tweets, not the side of the page.

    Any ideas for a workaround? Here’s a link to my archive in case I didn’t explain it clearly: http://goo.gl/IqDSGN Thanks!

    Reply

    1. Hi – in your js/applications.js file if you add the line $('.footer.muted').prependTo('.sidebar-nav'); after Grailbird.init(); at the end of the file it’ll move the description to the top of the sidebar. To move the follow button up it would be better if you could reference it <center id='follow-me-but'>... and then you could use another similar line $('#follow-me-but').prependTo('.sidebar-nav');

      Reply

  40. Hello, Martin.

    I have been using your twitter backup method. however, my twitter achieve page seems to be an empty space since yesterday.
    I checked a logs, the tweets seem to be backed up with no problems. But the archieves are not shown.
    I do not fix the problem, though I changed the twitter consumer key and consumer secrets.
    Now I also find the same problem your twitter achieve page.
    Do you know what’s the problem?

    Reply

      1. Thank you!
        I could fix the problem when i changed the file extension to .js →.txt→.js.
        After synchronized my tweets, I checked that the twitter archive is working without a problem.

        Reply

        1. … appears Google have resolved this issue and Drive is happy running the Javascript files associated with the archive

          Reply

  41. I apologize in advance for wasting your time with this question. I went through this tutorial and believe I successfully installed the archive on Drive. I forgot to bookmark the URL. I cannot find the archive online. I can find the source files on Drive, but can’t find it “online.” Do you know how I can find the URL without deleting and re-doing the hack? Thank you.

    Jonathan

    Reply

    1. Hi Jonathan – if you can still find the spreadsheet you used to set this up towards the bottom it includes the magic link.

      Reply

  42. Hello,

    First of all: Thank you!! so much for this hack. I have been relying on my archives for 2 twitter accounts for almost two years now and I love having them. This is the first issue I’ve had in all that time, which is pretty remarkable and great.

    They’ve failed to update for a week and a half (I have them on autoupdate, daily). Here’s the error I get when I try to manually update:

    “DocsList” is not defined.

    Any ideas?

    Thanks.

    Rebecca

    Reply

    1. Hi Rebecca – Google changed the way you interact with Google Drive. This has been addressed in an update to the code but for existing versions of the template you will need to either setup a fresh copy or open your existing version and then open Tools > Script editor and replace the code with the version here.

      Reply

  43. Hi Martin,

    Been a while since I’ve had to comment on here but I’ve hit a new issue. Backups stopped working with the following error

    ‘updateArchive – Authorization is required to perform that action.’

    The backup stopped working about a month ago and I’ve tried;
    – editing the code with the new bit from Billy
    – resetting API keys
    – deleting everything and starting from scratch

    As soon as i try and authorise i get this error;

    ‘Twitter Authorisation
    Please be aware that if you are logged into multiple Google accounts authentication may fail. You can either log out of other Google accounts or use an incognito window.

    Oops something went wrong. Twitter says:{“errors”:[{“code”:32,”message”:”Could not authenticate you.”}]} ‘

    I’ve tried setting up a new twitter app but i continue to get the same error… any ideas?

    Many thanks

    Mike

    Reply

    1. Hi Mike – in your Twitter app make sure sign in with Twitter is unchecked. Let me know if you are still having problems

      Reply

      1. Thanks Martin, it was my mistake not making the drive folder ‘public’ was causing the issue! bit random but somehow it defaulted back to private. All working now.

        Reply

  44. hi there

    i am getting this error message from Google:
    ——-
    updateArchive Script is using OAuthConfig, which has been shut down. (line 269, file “Code”)
    —–

    thanks
    mura

    thanks

    Reply

  45. Hi Martin,

    As of this morning July 7th your script stopped working, apperently due to oAuth changes. Error message:

    Script is using OAuthConfig, which has been shut down. Learn more at http://goo.gl/IwCSaV (line 277, file “Code”)

    The explanation being Greek to me, could you come up with a solution? :)

    Cheers,
    RJ

    Reply

    1. Google have changed some of their backend. If you took a copy of this template prior to 8th Dec 2014 the script will fail with a Script is using OAuthConfig, which has been shut down. Learn more at http://goo.gl/IwCSaV (line 277, file "Code"). To get your archive running again the best solution is to copy the template and setup using your existing Drive folder and API key/secret

      Reply

      1. I re-copied the template and pointed to my existing Drive folder but when I try to do the Twitter Auth nothing ever populates the pop-up window.

        Reply

          1. Yup. Using Google Chrome Stable (Mac). Tried it with extensions disabled and still not showing up. Swapped to Safari and it appeared so it appears to be something with Chrome for Mac.


          2. hmm – can’t replicate. When you first used a menu option did you get a ‘Authorisation Required This app needs authorization to run.’ prompt and authentication request? If not I’d try again with a fresh copy. Also check if any popups get blocked


  46. Thanks Martin! I saw the notice last night, you are quick. I spent a bit of time turning off updates on a pile of old twitter TAG sheets too,

    FYI, the new sheet works with existing authorizations if one is using your other fine sheets, so I did not need to enter the Oauth keys used on the old archive sheet.

    This is one of your finest magical bits of internet wizardry. Twitter’s search is like spinning some magic wheel of fortune. I use my archive so often to find archived tweets or conversations.

    You deserve an extra G&T for this.

    Reply

    1. @cogdog Mark Sample (@samplereality) needs to take a bow for alerting me. Thanks for the note on if you are already using TAGS the ‘magic’ follows you .. :)

      Reply

  47. Thanks Martin, great to have this, and great to get the update so promptly.

    Do you have any idea why the main tweet panel (left hand side, under the date) doesn’t render in Firefox (though not Chrome or IE). I had a look in the comments above, but can’t find anything, so not sure it has been mentioned before. It feels like a CSS issue (ie not your code) but thought I would ask anyway.

    Reply

      1. Hello
        Your tweets panel looks the same as mine – missing – so the problem must be with my Firefox. Shall investigate!

        thanks again for updating the archive script. All the best

        Reply

          1. Sorry for the delay – holidays ‘n all.
            The good news is that I tracked down the problem to AdBlockPlus (current firefox, windows 7+ updates). Disabling it for that page displays the archive.


  48. hi Martin,

    I’ve found this tool incredibly useful over the past couple of years, but I’ve been having trouble getting it up and running again after the OAuthConfig shutdown. I ended up just starting from scratch with fresh downloads / installs of everything, but when I do a manual “Update Archive Now” I get the following pop-up message:

    “No item with the given ID could be found, or you do not have permission to access it.”

    Any ideas as to why that might be? Thanks in advance for any thoughts!

    R

    Reply

      1. I’m a massive doofus. I wanged the whole URL in the Archive Path instead of just the folder ID. Apologies. Many thanks!

        Reply

  49. Hi Martin:

    Just a little note of relieved appreciation. I used your sheets for ready access to my Twitter archive back in early 2013. In November 2013 my sheets stopped syncing. Figuring that this was due to a change at Twitter, I suspected that I would need to troubleshoot/upgrade my sheets. I had no time to do so and so for a year and a half this stayed a very back burner/back of my mind task.

    Today I finally dug back into your blog/comments and was able to successfully restore functionality of my searchable Twitter archive. I am overjoyed!

    Thank you again for this wonderful contribution and for your ongoing support!

    You are truly an Internet Hero! :-)

    All the best!

    Kelvin

    Reply

      1. Seems to work indeed, thanks! Though Google Drive was still working today as well so we’ll have to see :)

        My url that forwards is tweets.mackaaij.org

        Reply

      2. Thanks for this, I just get the Red Alert from Google (who must be bored and feel the need to yank another service just for grins)– it seems to be working well.

        Is there another approach for hosting on Dropbox? It’s mentioned, but not sure how you did that.

        Reply

        1. Going down the Github Pages route. I’ve a prototype script that is working, just adding some interface so that it’s just a button clicking exercise for the user. Biggest part of the curve will be getting got novices to commit their initial archive.

          Nice thing about Github Pages is if have your own donation registered you can add a CNAME to your DNS.

          I’ll drop a line in this comment thread when there is something to try

          Reply

  50. Martin, this is great. Thanks so much for sharing with us all. I set it up a few weeks ago and it’s working great. Would you happen to know how to have the date & time display along with each tweet? I realize it would be a change to twitter’s code and not necessarily yours. I just started browsing through the JavaScript code and the solution is not apparent to me.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *