How BlueOrganizer Works

We just launched our best and most exciting version of BlueOrganizer - our smart browsing technology that focuses on bringing semantic web to consumer market in pragmatic way. This release is packed with cool features and things that make it easy for people to interact with things around the web.

In this post we want to tell you all about the internals of BlueOrganizer. Why? Two reasons: first, we are geeks and we love to talk about technology. More importantly, we believe in being candid about what BlueOrganizer does, how it does it and why. We want to tell you how it works so that you know that this piece of technology is working hard for you.

BlueOrganizer Fundamentals

At the heart of BlueOrganizer are two idea: Things and Shortcuts. The simplest way to think about them is to think about nouns and verbs. Things like books, music, movies, stocks, recipes and people are nouns or concepts that BlueOrganizer “understands”. Shortcuts are like verbs that can be applied to the nouns.

For example, consider a book: you can flip through it, you can put it on a shelf or return it to a library. Or if we are talking about a movie, you can rent it or watch. If its a restaurant you can reserve it and go eat there. Shortcuts in the BlueOrganizer are like verbs but in the digital world. When you are looking at a book you can read reviews or find out more about the author. When you are looking at a movie, you can look up its clips or pictures of the movie stars and if you are looking at a restaurant you can look it up on the map or find nearby movies.

Things and Shortcuts are not hard wired into the BlueOrganizer, instead they float dynamically from AdaptiveBlue.com. Because of this design, we are able to roll out new concepts and shortcuts without asking the users to update the product.

How BlueOrganizer Represents Things

Each thing, like a book or a movie is described in an XML file. First for each kind of thing we define its attributes. For a book, for example, we define title, author and ISBN. For a movie, we define director and stars, for restaurant address and cuisine, etc. We also specify key and required attributes. A key uniquely identifies the object in our system. They keys are really important because they allow BlueOrganizer to know that, for example, that a link on Amazon and a link on Barnes and Noble is actually about the same book.

In addition to the basic attributes each file defines keywords - frequently used words that define the concept. For a book, for example, these words are book, reading, author, etc. The keywords are used to help BlueOrganizer recognize the content of the page. There are a few more meta data elements but in essence this is it. You can find a complete sample model file here.

How BlueOrganizer Recognizes Things

BlueOrganizer deploys various strategies for recognizing things in pages in text. It uses a mix of APIs, Microformats and home grown recognition techniques. When user navigates to a page, BlueOrganizer looks at the URL of the page, and maps it onto a recognizer. For example, for Amazon, Netflix and Google Finance URLs there are specific recognizers that know how to deal with the information in these pages. There are also specific recognizers for blogging platforms like Wordpress and Typepad. And there is a generic recognizer that knows how to best deal with the information on any page on the web.

Each recognizer, specific or generic, consists of a set of agents. Each agent is concerned with being able to recognize just one attribute of an object. For example, when user navigates to Barnes and Noble page, a recognizer for Barnes and Noble books is paged in on the fly. Inside, it has agents that know where to look for the book title, author, ISBN and other book attributes. When user navigate to a movie page on IMDB, a different recognizer takes over. This one is looking for movie title, director, stars, etc.

So each page is handled differently depending on the kind of thing that it describes. When user presses the BlueOrganizer bookmark button in toolbar, the agents run through the page and grab the information about every attribute. Even though information about a book maybe presented in a different way on different sites BlueOrganizer works to normalize it and make sure its the same. The sort of cleaning that it does, includes removing brackets from titles, colons, dashes, words like paperback, etc. to make sure that the resulting information is the same across different sites.

The recognition of information inside pages is similar, except instead of analyzing the whole page, BlueOrganizer looks at a small piece of the page. AdaptiveBlue technology for recognizing objects inside text is based on microformats and home-grown recognition algorithms. Every time when user selects a piece of text, a set of agents is applied to it. One agent is looking to see if there is an hCard microformat present. Another one is looking for hCalendar microformat. There are agents that “understand” US addresses and popular people first names. All of these little agents run through the selection to figure out what you are looking at. When the match happens, BlueOrganizer “knows” what object user is interacting with and so it then can be helpful.

How BlueOrganizer Shortcuts Work

Once BlueOrganizer “understands” what user is looking at, it can offer contextual help. For pages and text, BlueOrganizer serves shortcuts that makes sense based on the type of context. So there is a set of shortcuts for books, music, movies, stocks, recipes, people, addresses, etc. For each type of thing, there is a set of shortcuts. Like definitions of things, the definitions of shortcuts are not hard wired into the BlueOrganizer. They are paged in dynamically from AdaptiveBlue.com. This allows us to dynamically add new shortcuts to BlueOrganizer without asking the users to update the software.

Each shortcut is a small template that essentially describes the URL and parameters. The example below shows a shortcut for author search on Barnes and Noble. The %author% means take the author attribute from the current book object and substitute it in place of the variable.

There is a set of shortcuts defined for each kind of context. A shortcuts for a page are different from shortcuts for a text. The page shortcuts are focused on the object itself, while the text shortcuts vary depending on what is selected. For example, if the user selects author name on a book page there will be author shortcuts. For an address, there will be location shortcuts and for plain text there are standard set of shortcuts that makes sense and is useful on any page.

How BlueOrganizer Personalizes Shortcuts

Not only BlueOrganizer shortcuts are contextual, they are also personalized. It is the combination of context and personalization that makes the shortcuts so cool and effective. BlueOrganizer recognizes that a user is looking at a book, and it also “knows” what book sites the user likes to visit. Although this may seem magical, it is actually really simple. To personalize shortcuts, BlueOrgainzer relies on the Browsing History. The history is used as a filter to turn on a subset of possible shortcuts.

BlueOrganizer never stores any of the user history nor does it send it back to AdaptiveBlue.com. The algorithm, which the users can re-run themselves, simply does a single history scan to decide what shortcuts should be automatically enabled. In addition to the automatic shortcuts, there are also defaults - a set of shortcuts recommended by AdaptiveBlue for each type of object. These recommendations are based on analysis of traffic popularity using Alexa, as well as user feedback and staff picks.

How BlueOrganizer Stores Information

When users bookmark things, BlueOrganizer writes these bookmarks into a local file and replicates them to Amazon S3. Each user has her own copy of the things she saved. On each computer that has BlueOrganizer installed bookmarks are stored in the Firefox profile directory under the blueorganizer folder. Inside this folder there is a folder for each account which in turn stores the actual bookmarks as XML files. Books, movies, music, etc. each are stored separately.

On Amazon S3, a secure storage service, the bookmarks for each user are stored individually. Currently, these individual bookmarks are not public, that is there is no way to access them from outside or via API. To make their information public, the users need to publish RSS feed, which is done by creating widgets.