A new software may be on your next PC, allowing for the processing of data up to 70 times faster than ever possible. The technology is known as MapD and burns through large torrents of data by storing the data in an onboard memory of graphics processing units (GPUs) as opposed to the standard central processing units (CPUs).
Currently, the new software technology developed by MIT, is being demonstrated on Twitter, where for example, it can show where and how a meme is taking off in real time.
“The [existing Twitter visualizations] we know of are ‘canned’—based on some previous computation of a map or picture, rather than being truly interactive,” “We have built a new kind of database system. It will answer and also map every request by scanning through every tweet in the database, which can be done in just a few milliseconds.” – Samuel Madden, a computer science professor at MIT.
The potentially revolutionary technology was concocted last year by then Harvard grad student, Todd Mostak who was frustrated with the sluggish processing when attempting to crunch social media data sets from areas of the Middle East. Mostak hopes the technology will make it easier to pull from Twitter’s huge bank of information. For example, combining census data with the word “Pepsi” in geotagged tweets, could show how it relates to variables like income or education.
“By building a tool to explore data sets like this in a truly interactive fashion, with latencies measured in milliseconds rather than seconds or minutes, we hope to remove a computational bottleneck from the process of hypothesis formulation, testing, and refinement,” said Mostak.
Via TechnologyReview