Seek And Ye Shall Find - Keeping Abreast Of Your Topics

As RSS/ Atom web feeds become more popular to subscribe to, many bloggers are increasingly admitting to reading them as a source of information and inspiration for their own content. But for certain topics such as technology, it's incredibly time consuming to track down all tech sites and then subscribe to their feeds.

It would be nice if there was some way to grab search results by the neck, shake them a bit, and produce a list of auto-discovered web feed URLs, then automatically subscribe to them. Well, there is. Except there isn't any handy package by which you can do this. But there could be.

I had a brainstorm a couple of days ago. It goes something like this:

Use the Technorati API to programmatically build a search feature that gets results for a search term. You enter a keyword or keyphrase, and your code returns a list of results. (You could use Google's BlogSearch, but I haven't checked to see if they have an API.)
The results are then stripped down and compiled into a unique list of URLs.
Each URL is visited by a mini-webbot and any web feed is auto-discovered. The content of the feed is passed through a relevancy filter. If the feed passes, its URL is added to an OPML list.
The generated OPML list is imported into a suitable feed reader, and voila! You're subscribed to a whole whack of feeds with very little effort on your part.

Now, does that sound like a RSS-based tool you could use? Anyone who wants to read a lot of decentralized feeds but has not started compiling a list could benefit. So could any new feed reader developer who wants to compile a list of default or starter channels for their users.

It's not very hard to do, relatively speaking. Technorati already has a library of API code for several programming languages. Stripping a list of search results to produce a unique list of URLs is just a few lines of code for any language that allows regexes (regular expressions).

The hardest part might be the relevancy ranking for each feed. Many web browsers already do auto-discovery of web feeds. In fact, some websites and blogs include the URL of their feed in an HTML <link rel> tag in the header of each web page.

If I can figure out the best way to do the relevancy ranking, I might take a stab at coming up with a tool to produce the final OPML list. But my attempts aside, I have a feeling that as RSS matures and achieves an even wider adoption, we'll see lots of 2nd- and 3rd-generation RSS-based tools that build upon other web services to massage data. In other words, keep your eyes open for exciting RSS-based tools, some maybe even AJAX-based, in the next year or two.

>> Raj Kumar Dash, http://www.chameleonintegration.com/

Technorati Tags: rsscases, rss cases, web feeds, content syndication, finding content, OPML, RSS, discovering feeds