IIT Bombay to try its hand at managing search data

Trying to search for a topic on the web but don’t have time to go through all the pages that the search engine throws up? Or trying to look for a particular camera model but getting zero results for your search? Here is a solution.

Searching for information on the web is all set to get simpler and friendlier — thanks to a collaborative effort between IIT Bombay and Yahoo India Research and Development. It has provided IIT-B access to a cluster of servers running Yahoo’s open source Hadoop software to facilitate analysis and management of web data. A first-of-its-kind cluster for an Indian institute, it enables applications to work with enormous amount of data.

“We have an outstanding group of faculty and students working on web search and data mining. This facility will help take our research to the next level, leveraging web scale data,” said director Devang Khakhar.

The already operational Hadoop cluster lab was inaugurated on Wednesday with the objective of making web information easier, clutter free and accessible to more people.

“One gets several web pages while searching for information on the web. We are looking at a search system that can combine data from multiple pages and present it in a useful format,” said S Sudarshan, professor at department of Computer Science and Engineering (CSE).

Another important aspect which is being looked into is how search results on people, places and things can be made more effective so that the user get the exact information he or she is looking for. Professor Soumen Chakrabarti is leading the effort with the help of other faculty and students.

Senior vice-president and head of Yahoo Labs Prabhakar Raghavan said that the understanding of human behaviour on the web is primitive and lot of research is required on areas like language used to create content, ways in which people communicate on the internet and how websites are connected, among others. The cluster aims at providing raw computing power, technology and rich data. 

“We are excited to see what research and innovations emerge from the collaboration,” he said.

“We donated a cluster to the Carnegie Mellon University’s School of Computer Science two years ago and they were able to handle a range of problems. We want to replicate the success in India and IIT Bombay was a natural choice,” said Raghavan.