Google Reader API

Dec26

Google Reader API

Google Reader is an online feed aggregator with heavy use of JavaScript and pretty quick loading of the latest feed data from around the web. Google's AJAX front-end styles back-end data published in the Atom syndication format. The data technologies powering Google Reader can easily be used and extended by third-party feed aggregators for use in their own applications. I will walk you through the (previously) undocumented Google Reader API.

Update 10:40 p.m.:Jason Shellen, PM of Google Reader, called me to let me know that Google built its feed API first and the Google Reader application second as a demonstration of what could be done with the underlying data. Jason confirmed my documentation below is very accurate and Google plans to release a feed API "soon" and perhaps within the next month! Google Reader engineer Chris Wetherell has also confirmed the API in the comments below.

A reliable feed parser managed by a third party lowers the barrier to entry of new aggregator developers. Google and its team of engineers and server clusters can handle the hard work of understanding feeds in various formats and states of validation, allowing developers to focus on the interaction experience and other differentiating features. You can also retrieve and synchronize feed subscription lists with an established user base that could be in the millions, providing a better experience for users on multiple devices and platforms. Google Reader's "lens" provides only one view of the available data.

Google Reader users are assigned a 20-digit user ID used throughout Google's feed system. ~~No cookies or session IDs are required to access this member-specific data.~~ User-specifc data is accessible using the google.com cookie named "SID."

Feed retrieval

/reader/atom/feed/

Google converts all feed data to Atom regardless of its original publication format. All RSS feed post content appears in the summary element and unlike the My Yahoo! backend I found no additional metadata about the feed containing full posts but Google does publish content data where available.

You may request any feed from the Google Reader system using the following URL structure:

http://www.google.com/reader/atom/feed/ + [Feed URL]
Niall Kennedy's Weblog (RSS 2.0)
Niall's Flickr feed (Atom 0.3)
del.icio.us popular (RDF)

You may specify the total number of feed entries to retrieve using the n parameter. The default number of feed items returned is 20 (n=20).

Google strips off all the data it does not render in Reader. Stripped data includes namespaced data such as Apple iTunes podcast data and Yahoo! Media RSS, additional author data such as e-mail and home URL, and even copyright data.

Subscription list

/reader/atom/user/[user id]/pref/com.google/subscriptions

Google Reader's feed subscription list contains a user's current feed subscriptions as well as past deleted subscriptions. Each feed is contained in an entry complete with feed URL, published and updated dates, and user-specific tags, if present. Current subscriptions are categorized as a reading list state. You may request the full list of feeds by setting the complete to true.

Here is a copy of my Google Reader subscription list with my user ID zeroed out. I am not subscribed to my RSS feed (index.xml) and I have added tags to my Atom feed. Each listed feed contains an author element which appears to be empty regardless of declarations within the original feed. Perhaps Google plans to add some feed claiming services, but its own Google blog has no affiliated author information.

Reading list

/reader/atom/user[user id]/state/com.google/reading-list

My favorite feature of the Google Reader backend is direct access to a stream of unread entries across all subscribed feeds. Google will output the latest in a "river of news" style data view.

Here is a sample from my limited subscription set. You may specify the total number of entries you would like Google to return using the n parameter -- the default is 20 (n=20).

Read items only

http://www.google.com/reader/atom/user/[user ID]/state/com.google/read

You can retrieve a listing of read items from Google Reader. You might want to analyze the last 100 items a user has read to pull out trends or enable complete search and this function may therefore be useful. You may adjust the number of items retrieved using the n parameter -- the default is 20 (n=20).

Reading list by tag

/reader/atom/user/[user id]/label/[tag]

You may also view a list of recently published entries limited to feeds of a certain tag. If you have tagged multiple feeds as "marketing" you might want to request just the latest river of news for those marketing feeds. The returned feed contains both read and unread items. Read items are categorized as read (state/com.google/read) if you would like to hide them from view. The number of returned results may be adjusted using the n parameter.

Starred items only

/reader/atom/user[user id]/state/com.google/starred

Google Reader users can flag an item with a star. These flagged items are exposed as a list of entries with feed URL, tags, and published/updated times included. You may specify the total number of tagged entries to return using the n parameter -- the default value is 20 (n=20).

Google treats starred items as a special type of tag and the output therefore matches the tag reading list.

Add or delete subscriptions

/reader/api/0/edit-subscription

You may add any feed to your Google Reader list using the Google Reader API via a HTTP post.

/reader/api/0/edit-subscription -- base URL
ac=["subscribe" or "unsubscribe"] -- requested action
s=feed%2F[feed URL] -- your requested subscription
T=[command token] -- expiring token issued by Google. Obtain your token at /reader/api/0/token.

Add tags

/reader/api/0/edit-tag

You may also add tags to any feed or individual item via a HTTP post.

/reader/api/0/edit-tag -- base URL
s=feed%2F[feed URL] -- the feed URL you would like to tag
i=[item id] -- the item ID presented in the feed. Optional and used to tag individual items.
a=user%2F[user ID]%2Flabel%2F[tag] -- requested action. add a tag to the feed, item, or both.
a=user%2F[user ID]%2Fstate%2Fcom.google%2Fstarred -- flag or star a post.
T=[special scramble] -- three pieces of information about the user to associate with the new tag. Security unknown and therefore unpublished.

Conclusion

It is possible to build a your own feed reader on top of Google's data with targeted server calls. You can power an application both online and offline using Google as your backend and focus on building new experiences on top of the data. Advanced functionality is available with a numeric Google ID and some variable tweaks.

Google has built the first application on top of this data API, the Google Reader lens, and judging from their choice of URLs the lens may not be Google's last application built on this data set. I like the openness of the data calls and think the Google Reader APIs are simple enough to bootstrap a few new applications within Google or created by third-party developers.

Tags:

Posted at 4:15PM
Updated at 12:44AM
Permalink
Comments [41]

41 Comments

Commentary on "Google Reader API":

Subscribe to new comments

Greg Linden on December 26, 2005 at 5:39 PM wrote: #

This is pretty cool, Niall. Nice work tracking all this down. But, is it really true that people can build on this? This was undocumented by Google and seems to be intended for their own apps. Using this seems risky, I'd think, with high likelihood of getting shut down if you built anything that was available publicly or generated noticeable levels of traffic.
mparaz on December 26, 2005 at 6:40 PM wrote: #

My educated guess: Google will open up the Google Accounts login infrastructure. Once logged in, you can make these calls legitimately. For non-logged in users, there could be an server-side HTTP equivalent of the Maps API (which is for Javascript rendering) Nice work, pitting G against Y! in the race for openness!
? Niall Kennedy on December 26, 2005 at 7:52 PM wrote: #

I think Yahoo! and Google are interested in opening up their feed backend in the same ways they expose maps data and other information. The first step is to expose the data, ask "why not?" and allow other developers to express their interest in using the data.

I sent a note to Scott Gatz, My Yahoo! lead, last week but I have not heard back from anyone at Yahoo! about my previous post. I also sent off an e-mail to Jason Shellen, Google Reader PM, this evening inviting conversation.

Google Maps API came after the JavaScript had been reverse-engineered and posted across the Internet with significant interest. I'd like to start a conversation about feed APIs and generate some interest in the data before a formal program is announced.
Chris Wetherell on December 26, 2005 at 10:31 PM wrote: #
Hi, Niall. Nice work on the API analysis! I'm certain Jason would be happy to jump into the conversation - I can as well, if that's all right.

So, to clarify. To date the API has been undocumented by us to date to help prevent those oh-my-gawd-the-url-changed headaches that would've happened if we'd changed something. So developers should note that some of the URLs might still change but that we're getting close to stability and we're happy for anyone to build atop the framework who is interested (and doesn't find managing auth issues daunting) as that's been our goal and intent with the API for Reader.

Other neat stuff I didn't see listed here:
- There's also a way to exclude tags/labels by appending another query to the URL as xt=[tag]. For example, you could exclude read items from some collections of items by appending: ?xt=user/[000...]/state/com.google/read
- Some tags include specific domain information because we've been planning on an approach that's flexible for other people's (read: non-Google) uses.
And I can confirm that you're right about the lens - it's only one of several planned approaches. Based on a series of user studies during early development we've been planning from the start a system that can support several distinct UIs for feed reading that would be developed by us - I actually have two other views on my dev build of Reader - but am more excited that others would be able to build clients that work best for them. One of the most exciting ideas for me for any feed reading API is that we (and perhaps other big companies) could possibly work on the more non-obvious and experimental solutions for syndication transformation since third-party developers are going to be excellent at supplying oft-requested and well-understood modes of interaction.

A short (and incomplete) list of stuff to do includes:
- Passing media enclosure data
- Atom 1.0 compliance (if you notice we're getting close)
Some of the other Reader team might post here, too, as everyone's been directly engaged with the API - it's been a chief part of the discussion, planning, and design for this li'l project. After we finalize the remaining bits we'll definitely post to our blog and document the framework.

- Chris Wetherell
Google Reader Engineer
Mark on December 27, 2005 at 5:42 AM wrote: #

For Atom 1.0 compliance, I recommend my own unit tests.
ihavenoname on December 28, 2005 at 7:29 AM wrote: #

What happens when you retrieve all unread items? Do they all get marked read? If that's the case then the API is pretty much useless from a synchronization perspective. If I try to sync to my handheld and then only read some of the items (typical usage scenario) and then try to continue reading on the web there will be a problem because items that I have not read in actuality have been marked as read. I don't see any facility for marking individual items read/unread.
Laurence Gonsalves on December 28, 2005 at 11:23 PM wrote: #

Items are not marked as read automatically, so it's possible to fetch an item without reading it. You can mark them as read using the same facility that's used for starring. Look for "flag or star a post" above. Just replace "starred" with "read" when passing the "a" parameter. By the way, the comment "No cookies or session IDs are required to access this member-specific data" is incorrect.
? Niall Kennedy on December 29, 2005 at 12:07 AM wrote: #

Thanks Laurence. I rerequested all URLs without an SID cookie and Google returned a status code 403 and "permission denied" in the entity. I updated my original text to reflect this information.
jcburns on December 29, 2005 at 8:18 AM wrote: #

The value of my SID cookie from Google.com is a string that is 161 characters long and starts with DQAAAGgAAABRnt... ...so where is this 20 digit value?
? Niall Kennedy on December 29, 2005 at 8:37 AM wrote: #

jcburns, Google assigns each user a 20 digit user ID that would not change over time, and a session ID (SID) you can delete, regenerate, and expire.
RichB on January 2, 2006 at 8:56 AM wrote: #

Notice the Google RSS->Atom conversion is recursive. eg: http://www.google.com/reader/atom/feed/http://www.google.com/reader/atom/feed/http%3A%2F%2Fdel.icio.us%2Frss%2Fpopular%2F
Nick Lothian on January 11, 2006 at 10:24 PM wrote: #

You may be interested in some futher reverse-engineering I did, and a Google IG module I build on top of the Reader API: http://www.mackmo.com/nick/blog/tech/2006/1/5/Google-Reader-Homepage-Module.html
Andrei Lopatenko on February 9, 2006 at 2:56 AM wrote: #

2 Chris Wetherell Hi, Chris is there any information which kind of Google Reader API is going to be provided by Google? Will you support a streaming model (event-based)? I am curious since i like to develop intelligent interfaces and I am RSS-addict. I am quite unhappy about flexibility and search capabilities of many RSS-reader and I want to develop a query interface for Google Reader
Steven on February 14, 2006 at 3:42 PM wrote: #

I hope the xt=[tag] functionality makes it into lens soon. The ability to show only things NOT tagged nsfw is a vitally important feature.
Mike on February 15, 2006 at 4:56 PM wrote: #

One more comment.... Has anyone figured out how to get a complete list of unread entries for all feeds you are subscribed to? This /reader/atom/user[user id]/state/com.google/reading-list seems to just give me a copy one entry from each feed I am subscribed to. I am looking to get a feed of all entries from all feeds (read or unread).
Sergio Nunes on March 2, 2006 at 3:13 PM wrote: #

Hi,

It seems that URL based authentication works. Try: http://[username]:[password]@[url] Replace [username] and [password] with your Google Reader data and [url] with one of the above urls.

This is great for server side development.

Regards!
Kannan on March 8, 2006 at 9:34 PM wrote: #
I am having some problems with google reader, although i must say that the google reader is extremely fast.
1. I want an entire feed to be labelled for example, rugby.com/atom.xml should be labelled as sports and etc. This would mean that I would read my "tech-blogs" labels first and then others.
2. Although it is not very desirable like the above thing, it would be great if we can have rules like gmail to categorize stuff.
Thanks (And as usual Google Rocks)
mparaz on March 10, 2006 at 6:54 PM wrote: #

It looks like even retrieval requires a login now. I was hoping to use this backend for an AJAX login-free aggregator.
mohan on March 14, 2006 at 11:30 AM wrote: #

Hi, can you please tell about the user id in the URL /reader/atom/user/[user id]/pref/com.google/subscriptions wher can i get it? thanks mohan
Nitin Nanivadekar on March 31, 2006 at 9:53 PM wrote: #

Thanks a million i am building an app that user feed reader as an auxiliary function. This fits my bill perfectly. -nitin nanivadekar
Darren Kulp on June 5, 2006 at 6:13 AM wrote: #

@mohan, jcburns: View source on your Google Reader page and search for _USER_ID.
Jamsi on July 31, 2006 at 1:53 AM wrote: #

I'm looking at creating a feed reader that works with the Google reader API (mainly so I can download a list of items to read on the train ..) but has anyone made such an application? I can't imagine it to be too hard.
Muzaffer Coruh on August 4, 2006 at 4:29 PM wrote: #

Hi,
The number of returned results may be adjusted using the n parameter."
this is really nice but I wonder which parameter is used to call next 20 items...
Mark on August 4, 2006 at 5:46 PM wrote: #

This information is great. Thanks so much. I'm having a little trouble understanding how to star or add tags to entries as these links seem to be hidden somewhere deep inside the javascript, css, or otherwise. Can you provide any further information that will help me piece this together? I'm working on a pre-reader that will filter duplicate news items from different sources as well as other unwanted content, since this doesn't seem to be in lens yet. This page has been a great help. Thanks!
Yasha Davidov on November 5, 2006 at 2:39 AM wrote: #

Just few things more: 1. You can access only unread items using ttp://www.google.com/reader/atom/user/[user ID]/state/com.google/unread 2. You can access all items via http://www.google.com/reader/atom/ (i suppose you must be loged in)
Yasha Davidov on November 5, 2006 at 12:46 PM wrote: #

It seems that com.google/unread always gives you an empty list. That was my mistake. But you can omit user id and replace it with "-". If the user is logged in this will work.
Rick on November 22, 2006 at 10:38 AM wrote: #

There is a way to request the number of unread feeds?
Venu on November 29, 2006 at 2:43 PM wrote: #

For shared items:

/reader/atom/user/[user id]/state/com.google/broadcast
Edwin Khodabakchian on January 22, 2007 at 11:22 PM wrote: #

Hi, Does anyone know if there is a way to apply multiple tags to a feed as part of a single HTTP request. I have tried the approach of calling multiple times the request which adds a single tag to a feed and I think that I am running into a concurrency bug because I do not get a consistent behavior from the backend. Thanks, Edwin
ptman on January 24, 2007 at 9:14 AM wrote: #

They still haven't released the API, have they?
jack on March 1, 2007 at 8:23 AM wrote: #

when I try the /subscriptions feed it says "oops! that wasn't supposed to happen." does anybody have any solution to that ?
ptitcali on April 11, 2007 at 6:06 AM wrote: #

Hi

actually I've tried the same, it doesn't work. I think this is a so-called "oh-my-gawd-the-url-changed" ^^

by the way, I've the same error (oops, that wasn't supposed to happen) with edit-tag and edit-subscription, even if I put the right parameters.

Do we have any info about when google is going to officially release the API?

Thank you Nail for your great work!!
ptitcali on April 12, 2007 at 3:44 AM wrote: #

Hi @jack I guess they changed the address... I've found this /reader/api/0/subscription/list maybe it is the same!! ^^ ++
Jack Tanner on June 5, 2007 at 9:25 AM wrote: #

Did the API change? The n=[int] parameter doesn't work for me on /reader/atom/feed/[url] any more.
anjanesh on June 9, 2007 at 5:34 AM wrote: #

ptitcali is right. /reader/atom/user/-/pref/com.google/subscriptions doesnt work any more. /reader/api/0/subscription/list does. How can we find the list of parameters & their functions for the new api ?
Craig on August 9, 2007 at 5:57 PM wrote: #

I've found a Google Doc that defines the API and explains how to get the SID cookie. http://code.google.com/p/pyrfeed/wiki/GoogleReaderAPI
HB on August 14, 2007 at 2:07 PM wrote: #

does anyone know if we can use Authsub with Google Reader API -- as opposed to using the the Auth Cookie which would required getting the user login/password.
Manoj Bist on August 18, 2007 at 5:16 PM wrote: #

Hi Chris,
Any plans to 'unofficially' support Google's authsub for Google Reader any time soon?

Supporting Authsub will remove huge burden of handling google user's password for anyone trying to develop applications over google reader.

MB
Tim on August 20, 2007 at 2:26 PM wrote: #
How are you guys authenticating against /reader/api/0/subscription/list?

I tried some basic http authentication but its not working; it returns a 400 status and this msg:

Server Error:
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.

(this method works for getting say a list of google bookmarks or something from search history)
```
import httplib2
login = "timothy.broder@gmail.com"  
password = "*****"  
url = 'http://www.google.com/reader/api/0/subscription/list'  
  
h = httplib2.Http()  
h.add_credentials(login, password)  
 
resp, content = h.request(url, "POST", body="nt", headers={'content-type':'text/plain'} )

print content
```
Joe Lazarus on March 16, 2008 at 12:53 PM wrote: #

Niall, do you know if there's a way to use the Google Reader API to get the RSS contents of a particular item within a feed? I see that you provide an example above of how to view the feed for this blog. I'm wondering if there's a way to view just this post, for example, without your blog header, sidebar, or all the other posts in your feed.
Ken on April 14, 2008 at 8:13 PM wrote: #

This is phenomenal. When will the API be released or stabilized?