Will We Censor Information by Seeking Relevant Search Results
Posted by Mike Bijon October 08, 2005
The root of the whole relevant-results problem is the concept of open vs. closed networks.
The original-closed “What’s new on the web” lists from 1993-1995 (I think) and most human-edited engines (like the original About.com, now just an editorial engine) have been supplanted by Yahoo and Google search. The lists were closed and the search engines were open networks. With too much information on the web it’s too big and broad to rapidly apply context to, so closed networks will constantly lack the capacity to classify that info. Open networks that auto-classify can always be grokked, after all they’re only as smart as the last time a programmer/team of programmers updated the algorithms that run them.
Search roles will never be the end game. They’re inherently flawed as long as they’re part of an open network. Robert’s inability to find links to HDTV manufacturers is a great example to use. Even with Clusty and Yahoo’s Mindset there are few or no relevant links. It’s likely there never will be very many relevant results in an open network. Additional filters like Mindset aren’t the answer either. After all, if Mindset were tuned to accurately filter manufacturers & distributors - how long would it be before those results were dirtied?
Closed networks provide a different, but equally flawed, answer. For example, if electronics.memorandum.com existed then it might provide a good subset of news and manufacturer-related info. Nonetheless, that would only be a SUBSET of what exists. I’ve noticed that Tech.Memeorandum has caused a narrowing in the topics discussed by many of the feeds I read (some of which are definitely Memeorandum sources). As an example, there isn’t one mention of Yahoo’s Mindset on Tech.Memeorandum despite all the discussion about search, Google vs. MSN/Yahoo, and even “The Search” that’s happened recently.
What we’re really facing, as information becomes both increasingly electronic and malleable, is a dilemma of exposure vs. technology. Will we rely on technology to filter irrelevant information and possibly to block exactly what we’re looking for? Will we restrict our exposure to information in order to view only good, relevant information from a smaller pool? Will we choose to use unfiltered technology to view volumes of results and somehow draw our own conclusions of relevancy?
The whole set of questions may be unnecessary, since the savviest of us will continually be ahead of any technology. With the tools we have I will always find what I want, and only in a few instances will I lose anything by not getting exactly that info immediately. Before the internet I would have had multiple newspaper and magazine subscriptions and still headed to the library or bookstore regularly. The majority of internet users, like paper-media readers before them, have and will be overwhelmed by volume of information but may never step beyond misleading articles or ads to know more than the latest diet trends or tomorrow’s weather. Despite all the efforts of search programmers, bloggers, and new media companies the most popular tools will be the most profitable ones. Just as search results are skewed by commercialism, our tools will be too.
In response to comments on Scoble’s post “My Brother and I argue about search roles”
Mike Bijon: I work as senior technology manager and technologist for 
