Life in the startup lane

Several weeks ago, at Filtrbox, we shared some of our internal functionality with the public via the Filtrbox Twitter Influence scoring page.  The Filtrbox Twitter Influence scoring page, which has turned out to be a hit among many, allows anyone to check the Twitter influence of any Twitter user. Some of our users have had some good fun with it for the purposes of ego boosting or ego busting. While we appreciate the versatility of purpose of our technology, the purpose of the Filtrbox Twitter Influence scoring page goes beyond a bragging rights tool.  The Filtrbox Twitter Influence scoring page provides a means to gauge the “reach” of mentions on  Twitter by measuring the influence of the “mentioner”  (Twitter is only one of the many conversation venues whose participants’ influence Filtrbox tracks). In this blog post, I would like to impress upon the reader that, going forward, the measurement of “influence” in social media conversation venues, such as Twitter,  should be integrated as part of all “message reach analysis” activity that a company performs.

Given the fact that conversation venues, such as Twitter, democratize the notion of “reach”  by providing a venue where anyone can mention anything (including your brand) to an organic audience (original target audience+viral audience), it is imperative that  brand protecting companies,

1) Track mentions of the company’s brand (s)

2) Analyze the influence (“reach analysis”) of the people who mention a company’s brand(s)

As social media networks become entrenched conversation venues where participants discuss anything under the sun including company brands, “reach analysis” needs to be expanded beyond messages that originate from a company’s marketing department.  This is the first step in acknowledging that there are other messages that are emanating from places other than your marketing department.  Those messages you cannot control. However, you can manage the conversations that the messages produce. In order to manage messages that result in conversations about your brand, regardless of their origin, brand mentions need to be effectively monitored and the message reach effectively analyzed.

Consider the following example: Every brand protecting company’s nightmare is seeing the following brand mention (message) on Twitter (or any conversation venue e.g. Facebook, Blog comment, Online newspaper comment)

“(put your brand here) sucks!!”

The next time people Google your brand; you do not want this to be the first brand mention they see. It well can be, if you do not properly manage the conversation that emanates from this mention. Therefore, before you react to the mention, it is important that you perform a “reach analysis” of the mention (measure the “influence” of the “mentioner”) in order to understand the authority of the person who made the brand mention, the nature of the venue in which it was made and the number of people who potentially saw the mention.  Performing such a reach analysis gives you the ability to assess an appropriate entry into the conversation and gives you a basis for formulating an approach on how to manage the conversation going forward. Products like Filtrbox simplify the “reach analysis” determination through Twitter Influence scoring and FiltrRank scoring.

In closing, it is important that ALL companies pay attention to “influence” in social media conversation venues. Think of “influence” as good old “reach analysis”, except the message whose reach needs to be analyzed is not coming solely from your marketing department - its coming from anyone, its coming from everywhere and, in a real-time information environment, its coming fast.

A couple of months ago, I watched an ex-driver of the President mention that as the President’s driver, he had to keep track of all the outlets to the safe houses and hospitals while he was driving the Commander-In-Chief.  I thought to myself, that’s what I do too, except, I have to keep track of all the Wi-Fi hotspots on the Denver-Boulder corridor while riding public transportation in case I need to tend to the servers. However, I have recently discovered a cool little iPhone app, TouchTerm, that has saved the day for me.  I now have complete SSH access to all my servers from my iPhone, no more keeping track of Wi-Fi hotspots.

Having used all sorts of monitoring tools for the servers, there is nothing like direct shell access from anywhere. I recommend this iPhone app to developers who are on the go.  This saves you time from spending time writing monitoring tools that you can access from the iPhone; TouchTerm gives you direct access to the shell.

TouchTerm has the following features:

• Complete server, connection, and password management.
• RSA/DSA Key-based authentication and public key distribution via e-mail.
• Rock-solid SSH implementation based on OpenSSL and OpenSSH.
• Wi-Fi and EDGE/3G support: access and administer your servers from anywhere.
• VT100 Terminal Emulation: Use top, screen, emacs, vi — virtually any console application.
• Landscape mode; Full-screen Mode; Configurable UI transparency, font size and color.
• A polished, intuitive, iPhone-standard interface.

TouchTerm screen shot

Several weeks ago, while purchasing a commemorative copy of the Rocky Mountain News, I came to the realization that two distinct stories, symbolic of the shift in media landscape, were playing themselves out on both ends of US-36. In Denver, The Rocky Mountain News, a symbol of traditional mainstream media, was closing down after almost 150 years of publishing. In Boulder, at Filtrbox, a young new media company, we were celebrating the release of the latest version of our service, Filtrbox G2. While the people at the Rocky Mountain News were probably not aware of Filtrbox, I had a keen eye on the daily goings on at The Rocky and I looked at the whole situation at the Rocky as a symbolic passing of the media torch.

As a long time resident of the state of Colorado, its was tough buying the last copy of the Rocky. As the CTO of Filtrbox, I lamented the loss of yet another mainstream content source. Contrary to what many may expect, in my opinion, the loss of content source like the Rocky is no cause for celebration at Filtrbox. The reason is that the death of a medium, such the newspaper, is a natural cycle; media have come and gone over the years. However, one thing that has remained constant is the content.  There is no substitute for good content. Whether Mike Littwin’s dispatches from the political stump or Dave Krieger’s Broncos inside scoop or Penny Parker’s celebrity sightings around town are delivered via pony express, the telegraph, the tabloid, the broadsheet, the web or a Filtrbox Daily Briefing, its all all great compelling content that I want to read on a regular basis. Thus, the death of the Rocky was by no means of verdict on content, it is a verdict on the medium in which the content is delivered. 

Filtrbox is providing new ways for discovering and delivering content using new media. Instead of a newspaper being delivered to your porch every morning, Filtrbox delivers a daily briefing to your inbox every morning.  In addition, Filtrbox provides various other means of consuming the content. But at the end of the day Filtrbox has to deliver content, quality content. The death of the Rocky results in one less source of content for Filtrbox users.  Content diversity is paramount if our users  are to be be well informed. Many have said, mainstream content will be replaced by blogs. However, that assumption is not reflected in the information consumption patterns that we see on a daily basis. At Filtrbox we interface with a variety of consumers of content and observe that information consumers like diversity.  Just as much as they want the thought stream in the blogosphere, they also want to know what is being said in mainstream media and on micro blogs and other sources. People simply want good content that keeps them well informed.

So, to the journalists who were at the the old media companies like the Rocky Mountain News and the Seattle Post Intelligencer, I say, there is still demand for your content; newspaper as a medium to deliver your content may be dying but other means to deliver your content are on the rise. Keep writing great content, the content industry is not dead.

While there have been many wishes and predictions for 2009, mine is simple, it’s HTML 5.  The adoption of HTML 5 specs by browsers, rendering engines and content publishers in this coming year will make 2009 a good year for me. As I have written in the past (here and here), content extraction is an often overlooked challenge that gets in the way of deriving web content semantics. This is an issue that often gets overlooked but for those of us who are passionate about extracting web content semantics, we understand how much it gets in the way of making much of the good work being done now even better. As we have seen recently, this is not an issue that is challenging only the small players, some of the major applications that rely on content extraction such a Google Alerts are seeing a degradation in content quality as they provide articles that have keyword hits in the navigations bars, ads and other non-content related text on web pages.

HTML 5 had taken steps in specifying how web content (e.g. news story, blog entry) should be represented in a page. The specification has attempted to structure a web page by separating different parts of a web page such as headers, footers, navigation, content etc.  The elements of HTML 5 that will help with content extraction are <section> and <article>.

The <section> element is described in the HTML 5 specification as follows,

“The section element represents a generic document or application section. A section, in this context, is a thematic grouping of content, typically with a header, possibly with a footer.

Examples of sections would be chapters, the various tabbed pages in a tabbed dialog box, or the numbered sections of a thesis. A Web site’s home page could be split into sections for an introduction, news items, contact information.”

Having an HTML element that groups content is very welcome.  The <section> element can be used to contain content such as a news article.  HTML 5 has gone one step further to make this possible by introducing the <article> element which the specification described as follows,

“The article element represents a section of a page that consists of a composition that forms an independent part of a document, page, or site. This could be a forum post, a magazine or newspaper article, a Web log entry, a user-submitted comment, or any other independent item of content.

An article element is “independent” in that its contents could stand alone, for example in syndication. However, the element is still associated with its ancestors; for instance, contact information that applies to a parent body element still covers the article as well.”

 A structured implementation of the <section> and <article> elements by content publishers will go a long way in making content extraction simpler thereby providing for a small step in making web content semantic analysis easier.

The HTML 5 specification has been out there for some time, its time for rendering engines to start implementing some of the new semantic oriented elements in the specification (some rendering engines have already started implementing parts of the specification). 2009 sounds like a good year for rendering engines, content publishers and content generation software to come together and help chart the course for web semantic analysis-based applications.

NOTE: HTML 5 contains other descriptive elements that help with the expression of semantics of textual data. I will get to those in future posts.

A couple of weeks ago, I attended the Yahoo Open Hack Day at the Yahoo Campus in Sunnyvale, CA.  At Open Hack Day, Yahoo opened up all their technologies for a few chosen hackers to play with and evaluate for a weekend.  The technology that I was most interested in was BOSS (Build your Own Search Service). BOSS is “Yahoo!’s open search web services platform”.  Simply put, this means Yahoo has opened up its web index for anyone to use using the BOSS API.  This is unprecedented and opens up a ton of opportunities to advance some of the topics that I have discussed on this blog, primarily NLP: Unstructured thinking for unstructured data and 2008 Web Search is still in 1979.

As I have said in the past, the goal of the semantic web is still a long ways to be realized. However, rather than wait for every website owner to build semantic web conforming website (or retrofit their past content to be semantic web compliant), we should seek to derive web semantics at the application level using a whole new set of applications, web semantic analysis-based applications. Yahoo’s BOSS can be one of the missing components that pushes the ball forward towards this goal.

Surprisingly, one of the challenges of deriving web semantics is as simple as programmatically identifying and extracting the content from a web page (I have talked about this in a previous post: A case for standardizing blog templates).  Before semantic analysis can be performed on a web page, the proper content must be extracted fom the web page first. As humans, when we look at a web page, we can readily distinguish the “main content” of a web page from  navigation bar, header, links or ads.  This is not so easy for computer programs to accomplish.  At Filtrbox, we have developed algorithms to accomplish this with a very high success rate only because we have devoted time and resources into the algorithms because they are core to our business.  Other application developers wishing to leverage web content semantics may not have the time and resources to build such algorithms because that is not core to their business. This is where Yahoo BOSS comes into the picture. We know that Yahoo has built its massive Web index by indexing the “main content” extracted from web pages.  Yahoo has  invested time and resources to solve the content extraction problem. In addition, they have built a massive infrastructure to index and store web content.  Therefore, instead of re-inventing the wheel, developers of applications that leverage web semantics can take advantage of Yahoo’s content extraction through the Yahoo BOSS API. However, Yahoo needs to open up a little more for this to be possible.

Here is where Yahoo needs to open up: Although Yahoo currently performs content extraction and content indexing, unfortunately the Yahoo BOSS API is not geared towards applications that analyze web data semantics.  The Yahoo BOSS API in its current form is geared towards web searches.  It is keyword query-based and returns at least TITLE, URL and ABSTRACT/EXCERPT.  Unfortunately, to move towards web semantic analysis-based applications, the ABSTRACT/EXCERPT alone is not enough.  Instead, the Yahoo BOSS API should return the WHOLE “main content” (not links,ads and navigation etc) of a web page.  Returning the whole content enables applications to perform semantic analysis on the data from millions of web pages that is stored in Yahoo’s web index, thereby adding value to the data and moving the ball forward towards unlocking the hidden value in web data using web semantic analysis-based applications.

(NOTE: This article was originally published at my old blog)

One of the questions that I am often asked is how Filtrbox is different from traditional RSS readers and aggregators.  The following are the major differences:

Closed Search Domain vs. Open Search Domain

When using traditional RSS aggregators, the user supplies the list of RSS feeds. This means that the domain of information gathered by a traditional RSS reader/aggregator is limited to the RSS feeds that are known to the user.  I call this a closed search domain. However, in an environment such the one we have today where thousands of new content sources are being created on a daily basis and anyone can potentially become a publisher, it is unrealistic to put the burden on the user to keep up with the thousands of new content sources that are sprouting up each day.  Filtrbox takes this burdensome responsibility away from the user and discovers the new content sources for the user because Filtrbox’s search domain covers all the new content sources. I call this an open search domain. The user can also add RSS feeds to the search domain, thereby guaranteeing that their RSS feeds of interest are searched. This approach leads to the user discovering new content sources.

Publisher centric vs. Content centric

Traditional RSS readers/aggregators present to the user all the content that is published by a specific publisher regardless of whether the user is interested in the content or not. Thus, the traditional RSS readers/aggregators implement a publisher centric information consumption model. On the other hand, Filtrbox implements a content centric information consumption model.  Rather than deliver to the user all the content published by a specific publisher, whether its relevant or not, Filtrbox allows the user to filter for the content that they are interested in from ANY publisher by providing contextual keywords. The content centric model implemented by Filtrbox greatly reduces information overload because each piece of content is examined and filtered for contextual relevance before it is delivered to the user.

No filtering vs. Contextual relevance filtering

As indicated above, traditional RSS aggregators do not filter the content.  All content published by a publisher in the user’s closed search domain is delivered to the user regardless of whether it is relevant or not.  Filtrbox applies algorithms that filter content from an open search domain of publishers for contextual relevance.  Filtrbox uses multiple factors to determine the contextual relevance of content and assigns a score called FiltrRank.  The most important feature of the algorithm is that the contextual relevance algorithm learns from a Filtrbox user’s implicit interests and applies the implicit interest to future contextual relevance filtering. This means that the content delivered to the user is content that that specific user is interested in and not content other people are interested in.  Contextual relevance filtering plays a large part in the reduction of information overload.

Beyond RSS

 
One of our favorite past times at Filtrbox is figuring out fun but useful things to do with our technology.  So in an effort to showcase our robust content filtering technology, we decided to put together FREE widgets that can be used to track news on each member of Team USA as well as the individual sports in which Team USA is competing.  

For example, if you only want to follow Lopez Lomong, you can set up your widget to show news about Lopez only or if you care about Women Gymnastics only, you can set up your widget to show you news about Women Gymnastics only.   In addition, you can set up different combinations of athletes and sports.  Most olympic content consist of every bit of news about the Olympics and you have to do the filtering for the news that you are interested in. At Filtrbox, we do the filtering for you.

We have created two types of widgets, a blog widget to embed on your blog and a desktop widget that runs on your desktop.  Both widgets can be found here.

My favorite widget is the desktop widget and because its my favorite, it has an additional special page for itself here.

Download the widgets and enjoy the Olympics.

Java + LAMP Developer

Join a dynamic, growing software company in Boulder, Colorado
Basic requirements are:

* Solid experience with Java and LAMP
* System administration skills (a plus)
* Working knowledge of Information Retrieval and/or NLP (a plus)
* Must be energetic, motivated and creative

Please send your resume to TOM AT FILTRBOX DOT COM

NOTE: Prima Donna, high maintenance Rockstar developers, please do NOT bother sending your resumes!!!!

This is a pic of the new “Hop 2 Chautauqua” route map that I took this morning at the bus stop on Pearl and 23rd.

On Thursday (04/24/2008 ) last week, I had the privilege of talking to Dr. Jim Martin’s Natural Language Processing (NLP) graduate class, at the University of Colorado at Boulder, about the work that we are  doing at Filtrbox and the role that current NLP students will play in the future of information technology.  This blog post is the basis of my message to the class.

As I have written before, the problem that we face today is how to harness the data that is available on the web so that we can apply meaningful interpretation to it using applications.  This problem is rooted in the assumption that the data that is stored on the web is “unstructured”.  Unlike the majority of the data processed by applications today which is stored in some form of a structure e.g. a relational database, the data on the web is not so, as its is perceived as discrete pieces of data scattered all over the web.

I told the class that part of what I am doing at Filtrbox is an attempt to prove that the data on the web is not as “unstructured” as we may think today.  Within that data, there is a lot of structure, relationship and general interconnectedness no matter how “discrete” we may think it is.  With effective mining of the data and good applications, we can apply interpretation to the data and produce meaningful information.  However, we are still far from applications that can apply effective interpretive meaning on this data.  The reason for this is that we have to address the problem of information retrieval (IR) first before we can get to the writing of applications. 

To recognize where we are today on the continuum of web data information retreival and applications; a look at the evolution of enterprise applications gives us a great analogy:

Enterprise applications are where they are today primarily because they have a structured data storage model (Relational Database or RDB) and a standard access model (Structured Query Language or SQL).  Before there were enterprise applications that we know today, there were only RDBs and SQL.  While RDB work dates back to the 1960s, the RDBs that the majority is familiar with today had their beginnings in the 1970s.  The first (or widely believed to be) commercially available implementation of RDB+SQL was Oracle, then known as Relational Software, in 1979. This provided the ability to query an RDB for data using SQL but no applications as we know them today.  Analogizing this with the web, this is where we are today. We can go on Google or our favorite RSS readers (RDB analogy) and query for web data using a weak REST API or search form (SQL analogy) but we have no applications comparative to what is in enterprise today to interpret that data.  So simply put, today we are where enterprise applications were in 1979.

My message to the class was that applications like Filtrbox are starting to barely scratch the surface with respect to the implementing of applications on top of web data.  That is because, although its 2008, we are still in 1979.  The stumbling block is the perception of the “unstructured” nature of web data. Today’s NLP students will play a large role tomorrow in identifying and establishing structure in the “unstructured” web data in order to move us beyond 1979.