The author of the web's first worm-virus, teamed with a man who dresses as a medieval warrior and goes to battle on the weekends and a woman who follows World of Warcraft, acupuncture and ballet, have raised $24 million dollars to storm the gates of the Google Castle. They got incredible press coverage when their new search engine, called Blekko, launched this week - but they are probably going to get slaughtered.
In the meantime, they have provided an opportunity for countless other freaks and geeks to use the magical tool they've built to grow our stature wherever we work; to cut through information overload, to shine a bright light on opportunities and to augment our minds with the snap of a finger. Read on for my advice about how to use Blekko and we'll use it well - for as long as it lasts.
Sponsor
What is Blekko?
Blekko was the name of company CEO Rich Skrenta's first networked computer. Skrenta was 15 years old when he wrote the Elk Cloner virus that infected Apple II machines in 1982; it is believed to have been the first large-scale self-spreading personal computer virus ever created. Skrenta went on to work on the Amiga at Commodore, then at Sun Microsystems, then co-founded the Netscape-acquired Dmoz and the Tribune/Gannett/Knight Ridder-acquired local news search engine Topix.
Now he's raised funding from a group of investors that include Marc Andreeson, to build Blekko. His band of freaky geeks include CTO and Society for Creative Anachronisms member Greg Lindahl and community manager Cheralyn Watson, who prioritized being ready for the WoW and ballet communities in the days leading up to Blekko's launch. (I found that quite charming.) The entire Blekko team includes more than 20 people.
What have these people built?
Blekko, simply put, is a social Custom Search Engine creation service with RSS feeds. It lets users curate and subscribe to mini-search engines that return results only from selected websites, thus increasing the signal to noise ratio and tightly controlling the context of search results.
If you've used Google Custom Search, really used it, that very powerful tool has been improved upon in Blekko because the latter was built to search large groups of sites and to have those groups shared and edited.
I used Google Custom Search all day, every day, to query my defined sets of blogs from technology analysts, geolocation specialists, semantic web scientists, youth marketing bloggers, English-language Asian tech bloggers and more, about whatever topics we're writing about here at ReadWriteWeb.
These Custom Search Engines are like dynamic reference books, populated by the blog archives of topical experts. They are things of magic - but Google CSE has never caught on beyond its widespread use as an embedded search box for a single website. That's tragic, if inevitable: the most powerful magic on the web will never be appreciated in its making, only in its results. Those of us who appreciate its making get to make some it ourselves. But more on that in a minute.
Blekko launched this week to big write-ups in the Wall St. Journal, the New York Times, TechCrunch and a flood of other outlets all over Techmeme. Frankly, I think it got all that press because it's backed by Andreeson, makes big claims about improving on Google, talks smack about red-hot content farm Demand Media as part of its pitch, has founders with very credible pedigrees and has good PR. But the general response to the company's product has been confusion and doubt.
As a regular and heavy user of Custom Search Engines, I've got some ideas about why most people are unlikely to appreciate the beauty and power of Blekko. But cynicism is cheap. Strategic advice on how to most effectively use a good tool is far more valuable and interesting.
How to Be a Blekko Power User, For Fun and Profit
I'd like to share a few thoughts about how I'm planning on using Blekko, based on the strengths and weaknesses of Google Custom Search. But first some assumptions:
- I'm going to assume that you work in a field where high-quality information is valuable.
- I'm going to assume that you can imagine the value of first mover's advantage, when it comes to information.
I believe that everyone works in a field like that, most people just don't know it.
First step, compile yourself a magic collection.
A searchable collection of your industry's top blogs and websites allows you to quickly put any topic in context, based on what the most knowledgeable people in your field have written about that topic in the past.
I've built scores of Custom Search Engines, but the first one I put into production is something I call my Magic Search. It's a collection of 25 of the top Web 2.0 news and review blogs on the web - all our competitors here at ReadWriteWeb.
I do a Magic Search whenever I see a new company or website, to see if and what our competitors have written about it. Last week, for example, someone sent us a link to the incredible website MapCrunch (like StumbleUpon for world-wide Google Street View views, try it!) and I was impressed. I did a Magic Search for it and with a snap of my fingers found that two of our biggest competitors had already written about it 3 weeks prior. (I hate to tell you how the sausage is made, readers, but if any of the leading tech blogs finds out that something has been written about by a competitor 3 weeks ago, then that would mean that we are 3 weeks behind if we write about it afterwords. There is nothing a leading tech blogger hates more than being considered behind the times. So I tweeted about it, and I'm telling you about that wonderful site now.)
If I'm writing about a company and I am being a good blogger who wants to mention competitors in providing the same type of search, I Magic Search it. For example, this week Nick Bilton wrote about the new email snooze-button service Nudgemail at the New York Times and got slammed in comments for failing to mention another service that had been providing the same kind of service for years. I reviewed Nudgemail after Bilton, and I made sure not to make the same mistake. I Magic Searched the neglected competing company and found that it had been written about by my competitors a number of times, along with still other companies providing the same type of service. So I included links to all of them in my write-up and I think it made for a pretty snazzy article.
If I want to know what startups provide a particular kind of service, I Magic Search it. If I want to know what my competitors have written about a person in our sector, I Magic Search them. If I want background on anyone or anything I'm looking at, background drawn from the archives of specialists in my field, I Magic Search it.
It's awesome. I've got it on my phone, I've got it on my browser toolbar and I've got it a couple of other places I won't mention here.
I've built that and my other Custom Search Engines with Google CSE, but Blekko is a very nice alternative. It provides a clean, sharable URL. It can be collaboratively edited. It handles search across hundreds of domains in one collection very nicely. It offers date-based search in a click. Blekko is a better service than Google CSE and I plan on moving all my engines there, as soon as the company offers a way for me to share them privately with my co-workers and no one else. Blekko's management says that's on the development road-map but not available yet.
Quick tips:
- Build a collection of top blogs on any topic using any of these methods, or others - or just Google for 'top blogs in X' and grab someone else's list to turn into a custom search engine.
- If you find a list on a page - try putting that page through LinkLeecher or some other way to quickly scrape the URLs off the page and into your Custom Search Engine creation tool. Sometimes I'll open a text editor next to a browser page, drag the URLs onto the text editor, then find and replace all instances of 'http' with '[linebreak] http'.
- If you've got a collection of the top blogs on any topic built, there are any number of things you can do with that, if you're creative.
Next step: Building Collections on the fly.
Content on the public web is searchable and the savvy searcher will manipulate that content to search it the way they want it.
Both Blekko and Google CSE are well-suited for carefully curated collections. Blekko in particular encourages the creation and maintenance of high-quality custom search engines with its easy collaborative editing, subscription to updates and more.
Sometimes you've got to do it quick and dirty, though.
Here's a story. This Spring I was hired to be one of the official event bloggers at a crazy conference called Techonomy. Bizarrely creative and accomplished people from all around the world came to speak at the event about how to use technology to solve the world's great problems (water, food, poverty, pollution, etc.)
The night before the event, a co-worker and I scraped the names and associated URLs from the pages listing the speakers at the event. I then threw those URLs into a Custom Search Engine that I used to do super-fast research while blogging on the conference floor. 'Speaker X says A,B and C. That's something that Organization Y, also participating at the event today, has posted the following white papers about in their website archives.' That worked really well.
While I was at it, I also put the names and URLs into a .csv file, uploaded them into Amazon's Mechanical Turk and paid $50 to have people around the world collect the Twitter handles, country-of origin and gender for each of the several hundred event participants. It took them about 90 minutes. I then created Twitter lists of all the participants, the women participants and the international participants, in part so I could put all three lists into Tweetdeck and make sure that as I was covering the event, I could track what people with different backgrounds were saying about the day. I also envisioned those lists being loaded into Flipboard, so you could have an iPad magazine made up of all the links shared by women who had attended the Techonomy conference, for example.
A few years prior, I did the same thing (and more) with companies launching at the DEMO conference, so whenever I was thinking about one, I could quickly search the websites of all the others and determine who else was working in the same sector.
Last week my mother emailed me to ask if I knew the educational technology speaker visiting her school that day. I didn't, but it took about 5 minutes to create an EdTech CSE and find what some of the most respected edtech bloggers online had said about the man. He and my mom ended up having lunch together and talking about the internet, which was pretty cool.
These on-the-fly CSEs are particularly well-suited to Blekko because anyone can then suggest revisions later. More importantly: it's a social community where you may find that someone else has already built the engine you're looking for and you can just grab it and go.
Quick Tips:
- I've trained myself to see a list of anything and think CSE. If you've got, or want to create, folders in an RSS reader - it's relatively simple to export your subscriptions, open the OPML file in a text editor, see how simple it is and delete all the subscriptions except for the topic folder in question. Resave it, then you can upload the new .OPML file into Blekko. That's a very nice feature, OPML import.
- The more you think in terms of URLs, the content they hold, the other URLs they are linked-to and the feeds that are associated with them - the more permutations for searching you can think of. The sky is the limit and it's a very rational system.
Finally, CSEs plus RSS equals a key to any inner sanctum. RSS feeds have been my bread and butter since becoming a professional blogger, and they are really what captured my interest more than anything when I first discovered social media. When I was just getting started (ok, sometimes I still do this), I would take the RSS feed of one or more blogs I really admired and I'd run them through an RSS to Instant Messaging service. These days, my favorite is Notify.me.
I'd get an IM within minutes of any post going up on a blog I really admired, and if I could make a meaningful value-add to the conversation, I'd jump right over and be the first person to leave a comment on the post. I'd do that just often enough to get their attention but not often enough to be annoying. (But where do you find interesting things to add to the conversation? If only there was some way to quickly search for keywords and tidbits in the archives of a collection of topical experts in any field!)
I used to do a whole lot of hitch-hiking and developed the ability to listen and converse with just about anyone, regarding just about anything. Participating in blog conversations is a much safer way to do that.
So - let's say you've got a good collection of topical expert blogs built up and you've got it put into Blekko so you can search against their archives. Guess what? Blekko offers RSS feeds for your search queries! That's awesome.
Take what parts of this you will, but I'll tell you what I'm going to do: I'm going to make sure that all the hundreds of geolocation experts who blog online know that I'm interested in geofencing and history because I'm going to subscribe to a feed for those two search terms in my big Blekko search engine of geolocation blogs. I'm going to read everything they write on the topic and I'll probably leave a comment whenever I can do so appropriately. Thank you, Blekko.
Google Custom Search does not offer RSS feeds. It has to be given a cookie and told to roll over just to get it to try to show you something close to the most recent search results inside your collection.
What Does it Mean?
These are just the ways that I intend to use Blekko, based on my experience using Google Custom Search and the ways that the two services are different. Maybe you've got other strategies in mind.
But here's how I see the three strategies above:
This is how I will assemble a magic reference book, a Dictionary, Encyclopedia and Atlas of the Past, Present and Future of...any topic I want.
This is how I will see a field, or a group of people and companies described online, and with a quick scrape be able to query deep into their online histories. Imagine looking at a set of people's names online, using find and replace to build a set of Google Social Graph query URLs to discover their blogs, bookmarks, Twitter accounts and more, then putting those URLs into Blekko. If that works, that's like 5 minutes of set-up to enable yourself to ask 'who in this room has ever bookmarked or tweeted a web page about topics X, Y or Z?' Hello dreamy social search engine!
This is how I will hone in on conversations in the future, among any set of experts, on any topics of interest to me. That's going to be very high signal and very low noise. That's like pulling a sliver of knowledge from the future and putting it straight into your brain - then sliding down it to arrive exactly where the conversations emerge, so that you can participate in them.
Will Blekko Succeed?
Clearly, only the ten people still reading this article and I are ever going to do the above - and we're going to have to click on a whole lot of ads to keep this puppy in business!
The rest of the world is unlikely to build Blekko-collections of their own because the cognitive overhead of curation is of a fundamentally different quantity and quality than the simpler requirements of consumption. People have compared Blekko to the recently launched Cuil because of the hype, but better comparisons could be made to the valiant but failed roll-your-own search engines like Rollyo, Swicki and Hittery. Not enough people with good intentions (not spammers) were willing to put the time in to use these tools well.
Blekko will attempt to compensate for this by automatically serving up search results from hand-curated custom search engines on big popular topics. That will help direct searchers to the Mayo Clinic's webpage for 'cure for the common cold' instead of Demand Media's eHow page on the subject, which is of clear commercial intent and questionable worth if you ask the Blekko team. Mayo Clinic doesn't have SEO people on staff, but Demand Media does, Blekko points out - so they built a health 'slashtag' that includes Mayo's site but excludes Demand sites. Health search queries are performed inside that slashtag by default.
Will that matter, though? Will millions of people rush with open arms into the unfamiliar, even if it's clearly superior upon the slightest investigation? Will people choose what's best for them? Will they choose what's most powerful and beautiful, instead of that which is easy and must remind itself constantly to 'Not be Evil?'
I would submit that if people were likely to make such choices, the world would be a very different place than it is today.
But in the meantime, fellow benevolent hackers, fellow dreamers of antiquity who find ourselves on the web, fellow fans of ballet, acupuncture and other romantic, unorthodox pursuits --- when we find ourselves with a need to search, in a cruel and banal world, may we use tools like Blekko. May we use them to dive deep into the archival knowledge of the finest writers and thinkers in our respective fields. To survey quickly what any set of people online have said before. And to track what our most trusted sources say regarding any topic, anytime in the future.
Even if no one else understands. They don't all need to understand, because in the hyper-linked, flash-curated, streaming social world of the future - those of us who know how to dance among the links and the fleeting, beautiful startups that overestimate most of humanity's capacity for intellectual engagement before crumbling into craven ad networks just to keep the lights on - those of us who can do the power user's dance in those circumstances will know that our positions are secure and that the web will be ours to create on. And we'll treasure beautiful things like Blekko, whether the rest of the world does or not.
Discuss
Google's mobile operating system known as Android 
