It appears you have not yet registered with our community. To register please click here...

Go Back [M] > Madshrimps > WebNews
MIT researcher comes up with bargain super-database MIT researcher comes up with bargain super-database
FAQ Members List Calendar Search Today's Posts Mark Forums Read

MIT researcher comes up with bargain super-database
Thread Tools
Old 24th April 2013, 06:23   #1
[M] Reviewer
Stefan Mileschin's Avatar
Join Date: May 2010
Location: Romania
Posts: 110,375
Stefan Mileschin Freshly Registered
Default MIT researcher comes up with bargain super-database

MIT researcher Todd Mostak has invented a new parallel database that allows for crunching complex spatial and GIS data in milliseconds.

Dubbed MapD, the database uses off-the-shelf gaming GPUs in the same way that you would use a rack of mini supercomputers. Mostak reports performance gains upwards of 70 times faster than CPU-based systems.

According to Data Informed, it all started when he was at the Center of Middle Eastern Studies at Harvard in 2012 and trying to map tweets for his thesis project on Egyptian politics during the Arab Spring uprising.

He found that it was taking hours or even days to process the 40 million tweets he was analysing. While he saw the value of geolocated tweets for socio-economic research, he did not have access to a system that would allow him to map the large dataset quickly for interactive analysis.

MapD will be released to the great unwashed under an open source business model similar to 10gen and its company MongoDB.

Mostak said that while people had written little research pieces about algorithms no one had tried to build an end-to-end system.

What was strange for MIT was that Mostak was not really a techie and had no background in computer science.

Mostak wanted to test the theory that poorer neighbourhoods in Egypt are more likely to be Islamist. He looked at geocoded tweets from around Cairo during the Arab Spring upraising. He examined if the tweet writer followed known Islamist politicians or clerics.

He cross-referenced the language in the tweets with forums and message boards he knew to be Islamist to measure sentiment. He also checked the time stamps to see if Twitter activity stopped during the five daily prayers.

He then plotted the Islamist indicators from 40 million tweets, ranging from August 2011 through March 2012, against 5,000 political districts from the Egyptian census.

The system was based around a $200, mid-level consumer graphics card, with two GeForce Titan GPUs made by Nvidia.

It was able to crunch data at the same speed of the world's fastest supercomputer in the year 2000 and cost $5,000 to build. He said that it uses SQL queries to access the data, and with its brute force GPU approach, it could work not only geographic and mapping applications but machine learning, trend detection and analytics for graph databases.
Stefan Mileschin is offline   Reply With Quote

Similar Threads
Thread Thread Starter Forum Replies Last Post
Security appliances are riddled with serious vulnerabilities, researcher says jmke WebNews 0 15th March 2013 10:53
Researcher wants to kill QWERTY Stefan Mileschin WebNews 0 13th July 2012 06:52
Yale researcher warns of cloud complexity Stefan Mileschin WebNews 0 12th June 2012 06:10
Researcher develops NIC rootkit jmke WebNews 0 24th November 2010 12:54
comparison of the Asus Eee PC against six bargain laptops jmke WebNews 0 12th March 2008 15:45
Vista DRM Cracked by Security Researcher jmke WebNews 0 30th January 2007 10:49
CoolerMaster Centurion 543, a good bargain? jmke WebNews 0 16th June 2006 13:47
Half Of Corporate PCs Can't Run Vista, Researcher Says jmke WebNews 0 11th April 2006 10:04
Intel Pentium D 805 is a bargain basement processor jmke WebNews 0 9th April 2006 13:58
Maxtor 160gb bargain? jmke Mad Bargains 13 21st April 2003 00:49

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off

All times are GMT +1. The time now is 08:50.

Powered by vBulletin® - Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO