Information Vending and the New Web
Posted by indroneel on March 21, 2007
Traditionally, the Internet has always been about information hosting, with the power to contribute in the hands of a select few. The Web 2.0 paradigm has changed all that, bringing the power of publishing to the masses. With such a volume-driven approach, it is only natural that every submission to the write-enabled Web cannot be an original that appeals to a large section of readers. In fact, most of the information floating around is second-hand in nature: repeated and referring to facts and ideas expressed elsewhere.
Second-hand information is not all that bad, so long as it is possible to trace back to the original. The primary benefit is that of information reaching a larger audience, with less effort and diligence on part of the individual. Yet another interesting effect of such repetition is that of ‘information vending’.
Information vending can be defined as the set of processes and techniques by which relevant information from multiple sources reach their intended audience with a high degree of accuracy. Often, the content that reaches an end-user includes the original juxtaposed with inputs from other contributors collected during propagation. There is an element of mediation involved, both automated and human. Automation involves software-based services and hosting platforms while the human element is provided by the ‘wisdom of crowds.’
Critical Success Factors
From an end-user’s perspective, the critical success factors for an information vending scenario can be summed up into:
- Implicitness: Provide an answer to the question: “Where do I start looking?” Maps personal interests and broad-level queries onto information sources.
- Relevance: Associate content with each other to achieve maximum significance for a given context. Provide a measurement of relevance in absolute terms.
- Automation: Separate information from content. Reduce the number of steps, and hence effort, required to obtain information.
Concepts and Mechanisms
The following sections provide an overview of the different paradigms that have evolved within the scope of information vending. These paradigms are qualitatively graded on the basis of critical success factors mentioned above.
Level Zero: Internet Search
Search engines like Altavista, Excite and Lycos were the original information vendors on the World Wide Web, predating Google, Yahoo and MSN. Newer concepts are continually being introduced to improve relevance of search results and offset information overload brought about by the new Web.
Internet Search Engines in their basic form are the most primitive of information vending solutions with minimum automation and wide variations in relevance.
Level One: Feed Aggregation
Aggregation of syndicated content is the first level of information vending that utilize the features provided by Web 2.0. The information vendor is the platform or the solution that subscribe to pushed content from multiple sources and present the same through a unified interface.
The protocol for syndication is any one (or a combination) of RSS, RDF and ATOM. Presentation is usually through HTML with RSS smartfeed and email integration provided as value additions.
Level Two: Search and Syndication
A combination of search and content syndication usually manifests itself in the following two ways:
- XML search feed services: generic search services that present their output in a syndicated format. This allows end-users to create custom feeds based on search engine results. Examples of such services include Google Blog Search and Gigablast.
- RSS search engines: back-end aggregators with a search engine front-end. Examples include Feedster, PubSub (now defunct) and Bloglines.
Level Three: Rule of the Masses
The community-driven approach to classify, refine and popularize published information introduces the human factor to an otherwise automated information vending space. As a new compass for the Web, this approach makes use of collective intelligence to pick and choose the right content for a given context. Common techniques to harness this intelligence include,
- tagging and bookmarking for content classification (del.icio.us, Technorati)
- voting for popularity-based ranking (Digg)
- collaborative authoring (Wikipedia)
- domain-based searching and participative refinements (Swiki)
Level Four: Wires and Chains
Popularized by Yahoo Pipes with a possible origin in Dapper, wires and chains add a whole new dimension to the concept of mash-ups (data mixing). Through a variety of visual programming environments, end-users can create extremely flexible data mixers that take input from disparate sources and output in multiple formats. With the added abilities to share and reuse mash-ups definitions in a hosted environment, every mash-up author ends up being a potential information vendor.
So far, on the basis of critical success factors, wires and chains represent the highest level of refinement and build on top of all other paradigms described so far.
The Future of Information Vending
Future trends in information vending would be along the lines of data homogeneity and interoperability. The human factor would extend to include expert-driven mediation during information interchange.
All sources of information shall be interconnected. The Web shall end up being one huge database with a manageable and finite number of data representation semantics. Consequently, information vending services and applications will cooperate with each other and other services to form one large eco-system.
Enterprise and Intranet data will integrate seamlessly with online data. More and more private domain information will go online.
Public-facing expertise will be institutionalized in the form of information services. Non-trivial information related operations shall be mediated and driven by human experts. In such cases, collective intelligence shall be used to grade and choose the right experts for a given context.
This entry was posted on March 21, 2007 at 2:15 pm and is filed under community, internet, portal, syndication, web. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.