This week we released the beta version of the open API for our new Calais Web service, and debuted our new developer community at www.OpenCalais.com.  

The Calais Web service is a free metatagging service that enables both publishers and content consumers – search engines, news portals, RSS Readers, etc. – to automatically metatag the people, places facts and events in online content. Doing so adds an important layer of Semantic intelligence to that content while increasing its relevance and accessibility on the Web. (You can read more about the speicifcs of this initial API release in our press release here.) 

Why does the Semantic Web matter?   Let me go back to the beginning of my interest in the Semantic Web – some ten years ago. The Semantic Web – often referred to as ‘the Data Web’ and considered part of ‘Web 3.0’ –enables computers to access and use the meaning of words, instead of just noting their presence on a page or mapping that page’s relationship to others.  Also called the ‘Great Global Graph’ by Tim Berners-Lee, the Semantic Web adds a new layer of context to content, commerce and community sites that enables computers to detect existing connections and make new ones. Once this new layer of context is created, ‘intelligent agents’ can be built that can access data across a variety of applications. For instance, a simple movie bot might scan the Web for film titles, returning links to synopses, credits, reviews, ratings, listings and tickets in one easy place.  The basic promise of the Semantic Web is aligned with the Data Portability initiative, which posits that individual Web users should be able to share their identities, photos, videos and other personal data between chosen and trusted tools and vendors.  Both initiatives are about enabling applications to share data and communicate amongst themselves – with the requisite privacy protections in place – to automate the exchange of information where possible and let human beings focus on bigger picture tasks. 

Why do we need this much help? Despite the plethora of cool Web 2.0 apps – in fact, in part because of them – people are still suffering from information overload.  Web 2.0 has set content free to roam the Web via feeds, bookmarking services, widgets, social networks, etc.  And while this “opportunistic” distribution of information has increased the accessibility of insight overall, it hasn’t helped us find exactly what we need when we need it.   Semantic tagging offers superior sorting and filtering of content for more targeted and timely delivery. 

What’s the hold up? Good question. 

The Semantic Web has historically struggled with two problems: 

  1.) The Semantic Chicken & Egg  •   Semantic applications require publisher metadata in order to attract users and gain popularity •     Publishers will only bother to create metadata when Semantic applications get popular 

  2.) Metadata Cost & Transport  •   Creating quality metadata entails costs and resources that most publishers can’t afford.•          There is no way to transport metadata with the content as it moves thru feeds, widgets, etc.  

The good news: Today, not only are Semantic search tools cropping up – like Powerset, Spock and Wikia – but consumer-facing Semantic applications have emerged, like Twine, Freebase and TripIt.  Publishers and providers now seek a simple, effective way to tag their content, make it accessible to these new applications and export it liberally across the Web. 

OpenCalais.com  Drop by OpenCalais.com (www.OpenCalais.com) and check it out.  Note that this is a beta, and our initial release, so you can expect to see additional languages and functionality as we evolve. I would love to get your feeback, which will help guide the addition of new features and additional developer support as we go forward.  

“There are very few monsters who warrant the fear we have of them.” —  André Gide

I have the amazing opportunity to lead a global team – within a global company. Not just a company that has figured out how to take Western products and services and “localize” them, but a company that is truly global. Within the top 2 tiers of my team there is representation from 7 time zones, 3 continents, spread across cities like London, Paris, Bangkok, Beijing, New York, Boston, Tel Aviv, Bangalore and St. Paul, Minnesota.

This is not done for wage arbitrage. In fact, a simple economic principle nearly dashes all expectations of cheap tech-labor: Market equilibrium. Ultimately, a java developer who lives in Bangkok will be earning a wage substantially equivalent to a Java developer in Palo Alto. There are a few mitigating forces providing downward pressure on wages - efficiency in communication and management of the “remote” developer, liquidity of the market with respect to job alternatives for the employee… a whole range of factors come into play. Many of which are short-term. Communications will be worked out. Time shifting will become natural, remote places will fill up with companies competing for talent. Just look at Bangalore. I am not expecting a 1:1 ratio any time soon, but far more importantly, I believe companies expecting to continue 3:1 economics better prepare their balance sheets for an upset.

Additionally, the concept of offshoring is old – and pejorative. It locates the center of the action in the Western world. How imperialistic of us all. The effective way to think about it is creating a Global mix. My experiences over the last 4 years with my previous team in Bangalore taught me one thing: Talent is universal, but it takes work to establish the relationships, trust and working rhythm to harvest it. I also learned another thing – cultural “barriers” are befuddling. These things pop up unexpectedly and disappear unexpectedly. I could write a book about the situations where I have scratched my head and asked “what just happened???” Diligence always required…

So why build a global workforce?
- First of all we live in an increasingly global world. There is a very good argument to be made that a global economy (and all of its inefficiencies) is becoming efficient. Foreign currency exchanges are very efficient. Investments can now be shifted from one country to another very easily – even for the casual investor. The world is truly a small place, and to be competitive in the new world, you need to be operating in it. (By the way, if you want to read about the potential negative cultural effects, read John Hagel’s post here). This will show up in greater success in global markets. I will pick this topic up in more depth soon…
- Second, talent is scarce. My leaders can’t hire enough developers in New York, London, or even St. Paul to keep up with the demands of the business. Regardless of price. To have access to a large market of potential talent can dramatically change a project’s (or a company’s) prospects. In my world, this pays immediate dividends in time-to-market.
- Third, a globally diverse team will be resilient to the movement of any one economy, cultural situation and, in unfortunate situations, disaster. Basic portfolio theory, applied to people. Good, responsible management.

What have I found to be the critical factors?
- Critical mass in locations: A small team – or even a large team without diversity in its tasks, will burn out within 2 years. Each center of presence needs to be thought of as an integrated, yet independent organic system. This shows up as the opportunity, or lack of opportunity, for sustained career growth for the team’s individuals. A team (anywhere) without career advancement opportunities will begin to turn on itself over time. It’s universal. Additionally, diversity in roles in a location provides a healthier system. There is a magic spark when a product person, a program manager and a developer work together to solve a tough problem.
- Getting over the temporal hurdles quickly. Communication barriers, time zone differences, creating a shared understanding of “quality” – these are all problems that can be largely solved once, and built into the DNA of the team. Without solving them a team will languish. By solving them well, collaboratively, it becomes competitive advantage. In my experience this is best done by getting the team set up and then giving them something very hard to accomplish – something that reaches across the geographical boundaries of the extended team.
- Multi-cultural leadership. There is no way around it – success in starting up a team requires a leadership “bridge” – someone who is considered to be an accepted member of both cultures. There are many ways to get there, but it’s critical that while a nascent team is establishing there’s an accurate, trusted translation of leadership between cultures.
- Recognizing and driving forward into “phase 2.” Once a team is operating relatively well, it’s time to loosen the reins, encourage more collaboration and independence within the team. While phase 1 is about fostering a new shared culture and carefully managing things, phase 2 is about getting out of the way. Trust your new leaders and continue to give them hard things to do. Much more to write on that one, too.
- Fostering innovation. This is the tough one. Not even going to begin to tackle it today.

I originally sent this post simply as an email to friends and former colleagues. I was encouraged to post it for posterity, so here it is. 

John McKinley recently posted on his blog about AOL Search - it’s a positive view of the progress made over the last several years, and the fact that it largely went unnoticed in the market. 

It seems that the story is that “we did a lot with a little, and made some positive gains though few noticed.“ That story works for me in retrospect, but I also think a view of what it was like inside will provide a view to the future of search.

The team believed and many of us still passionately believe that delivering actual content rather than just links to content is a more satisfying and valuable experience. Research, business intelligence and revenue supports the fact that it’s a winning proposition. Full stop.

AOL Search pioneered a database-driven, Widget-driven search capability, one that offered content rather than links to content. It started 6 years ago with “starship” and grew into FullView. More than a veneer, it was a powerful, sophisticated semantic web platform. Web results were a simple modular part that could have been replaced at any time (given the right acquisition, etc.).

This experience had the potential to be a top 3 market player had there been a moderate amount of marketing and branding effort beginning in about 2003. AOL was not in the position/of the mind to do it. Understandably, there were many other projects drawing the focus of the company.

In retrospect we can think that Google’s ability to solidify the market around a “blue links” model was quite possibly unstoppable. It wasn’t always so, and I don’t think it will last forever. Users will gravitate to the more valuable, useful, “saves me time” experiences. ASK is on the right track to be the innovative counterbalance to the links-only experience. I hope Jim Lanzone and his folks keep it up.

The operative question is what could have been done differently in order to deliver a different fate for AOL.

Simple answer: reinvest a moderate portion of the vast margin into growing the business. It never happened.

So, after a half-decade of underinvestment, AOL had a choice of what to do with the property.

It was the same old question: Invest and build a brand - get behind the opportunity to be a player in search - which will be an (ever more) expensive battle and will have to become the company’s primary or secondary focus, or minimalize, enjoy the margins and a great partnership with Google and count the revenues.

I applaud the Falco/Grant/Cahall team for finally stepping up to the decision that had gone long-unmade. I am disappointed at the outcome, but not sure, given the years of underinvestment, that it wasn’t inevitable.

G

I tend to reject buzzwords, and semantic web is a term that’s as buzz-y as they come. The best ideas end up buzz-ified, though… I’ll do my dissertation on the Phases of the Web here soon, providing a different view on Web 2.0, etc. But today I am thinking about data, content… how to unlock it, associate it and describe it… so we can build more insightful ways for users to get their information. 

As a part off my quest I went back and re-read Tim Berners-Lee’s seminal paper on the Semantic Web. The clarity of insight into the problem is awe inspiring:

“The Semantic Web is not designed just as a new data model – it is specifically appropriate to the linking of data of many different models. One of the great things it will allow is to add information relating to different databases on the Web, to allow sophisticated operations to be performed across them.”

Yes. That’s what I want to do. The term Semantic Web just took on renewed meaning for me. Not buzz, but a path, a de-hyped recipe for getting good stuff done. In fact, there’s a sort of pragmatism that’s very appealing:   

“We remove the centralized concepts of absolute truth, total knowledge and total provability, and see what we can do with limited knowledge.”

I can work with that. The world is messy. Data, content and metadata are messy.

Yes. Now get on with it: create systems that can adapt to the chaos, take advantage of the specific design of each data source.

When you’re handling content from the Web this is a no-brainer. It’s very obvious. When thinking about a company with purpose-built databases/models, each with domain-specific content - with significant expense in creating, maintaining and cleansing highly structured data it’s a very different proposition. That’s the challenge of vertical search. Health, autos, financial services, shopping… we’ve all got the problem.

Who’s doing it well? I’m going to be digging in to get underneath this and I’d love to hear people’s thoughts if you happen by this post.

I’ve been thinking about blogging for a long time, but haven’t jumped in until now.

Seth Goldstein and I started a new conference series - Open Data - which is all about opportunities and challenges around the proliferation of API’s, feeds, etc.

OpenData2007 was beyond my expectations, way beyond. The group was diverse - journalists, technologists, VC’s, entrepreneurs and corporate folk. Despite the various perspectives and interests, conversation was passionate, vibrant and insightful.

One outcome is that we’re going to do more of it - a series of discussions - maybe even bi-coastal.

Another outcome is that I am finding my voice. That’s what I’ll do here.

As for my day job, I run a new group within Reuters. From the outside, I thought it was a news company. From the inside what I’ve found is a company that is focused on delivering the utmost quality and timeliness in financial services information. Underpinning that is a set of trust principles that inform and guide the company to be independent and free from bias. My team’s mission is to unlock the vast content within the company and put it in the hands of our customers. It’s the perfect opportunity for a guy like me who loves technology, markets and creative problem solving. There’s a lot of untapped revenue potential.

I don’t know how often I’ll post. That’s partially why it’s taken me so long to make the jump to blogging. We’ll see where this goes…

i seem to have lost my original post. will redo this today. until then, welcome! look back for more later.