Better graph algorithims
There are a few interesting postings I came across lately on graph parsing optimizations.
Google’s Mark Chu-Carroll discusses parsing strongly-connected digraphs on ScienceBlogs. His approach includes breaking up a complex digraph into an array of strongly-connected subgraphs and then throwing the power of parallelization at the problem. Each machine is responsible for processing its own subgraph, and when it’s done, it reports back, making this algorithm a special case of MapReduce. Such algorithms are useful for microforecasting, i.e. predicting weather pattern at very precise locations, or, if you choose, figuring out giant link farms established by large blog networks all inter-connecting to their sister blogs in an attempts to drive PageRank.
Caltech’s Yuri Lifshits discusses similarity-based searches. Similarity searches are employed by visual shopping engines, such as Like.com, recommendation engines and their varieties like personalized news recommendation engines, ad targeting engines, and algorithms that do any kind of fuzzy matching (job site matching your resume with potential employers, dating site, etc.) There’s also a paper attached to that presentation for heavier reading.