So I recently found myself using Yahoo! Pipes.  I wanted to aggregate all the RSS feeds from local newspapers, filter the articles for those that mentioned the Company (which is what I will henceforth call the place at which I hold a steady job), and consume it in the Company’s new web app I’m working on.  I’d heard of Pipes a few years back, but I never had the reason to use it.  It’s a great tool and dead simple to use.  We could probably use it to replace the (what I’m sure is expensive) news aggregator service that culls industry news and articles that mention the Company from publications across the country.  Well, maybe eventually.

Anyway, creating the pipe was easy and using the interface was actually quite fun.  I’m pulling from almost 40 different feeds, filtering for mentions of the Company’s name in both the title, and the articles themselves, and now have one single feed I can now use.  On the web app end I’m using the ever-powerful and omnipotent jQuery to consume the feed in the JSON format.  Now, I’m not going to go in-depth in how this is accomplished, but rather I want to throw out this “gotcha” moment I experienced and hopefully, help at least one of the two people who will ever see this post avoid my mistake.

The JSON link I received from Pipes looked like this: http://pipes.yahoo.com/pipes/pipe.run?_id=[Some_ID]&_render=json.  Now in the jQuery documentation for the getJSON function, it says that in order for cross-domain JSON requests, you must use a JSONP callback by appending a variable to the url’s querystring, which is “callback=?”.  jQuery will then replace the question mark with the correct callback function that will then wrap the JSON object that comes back to your app.  So I created a link that looks like this: http://pipes.yahoo.com/pipes/pipe.run?_id=[Some_ID]&_render=json&callback=?.  That should work right?  Well, at least I thought it should.  (Others with more experience with JSONP callbacks will probably already see where I went wrong.)  Instead of my new aggregated feed in my web app, I got an error from Firebug instead.  The message said “INVALID LABEL”, followed by the JSON string.  Well, I can see that I’m making a JSONP call, or else the error from Firebug would have been “ACCESS TO RESTRICTED URI DENIED” and I wouldn’t be able to see the JSON data.  So the cross-domain call works.  So, then, what’s wrong?  Well, jQuery is trying to parse JSON data as JSONP.  Looking back at the Firebug error, the JSON data that’s returned isn’t wrapped in the JSONP callback, but I know that the callback is being appended to the Pipe’s JSON URL querystring.  What gives?!

Insert an hour or so of misery and woe, as I Google every possible combination of “jQuery”, “JSONP”, “callback”, and “JSONP is teh suxxorz”.  I never found the answer.  Instead, I ended up staring at the Pipe’s JSON URL and I notice something.  Each variable declaration in the Pipe’s URL querystring is prefixed with an underscore.  Could it be that easy?

Yes. We. It. Can.

Enter the new JSON request URL: http://pipes.yahoo.com/pipes/pipe.run?_id=[Some_ID]&_render=json&_callback=?.  Now the request works perfectly.  Why does the callback variable need to be prefixed with an underscore?  Who knows?  Something to do with Yahoo’s processing of the requests I’d guess.

Hopefully, this can help you avoid the sheer frustration this caused me.