Viewing the hopes and fears of the world through the analysis of real-time Twitter data on the SGI® UV™ 2000 — #tweetbeat
Salt Lake City, UT — November 13, 2012 — SGI (NASDAQ: SGI), the trusted leader in technical computing has partnered with Kalev H. Leetaru of the University of Illinois and Dr. Shaowen Wang of the CyberInfrastructure and Geospatial Information (CIGI) Laboratory at the University of Illinois at Urbana-Champaign to create the Global Twitter Heartbeat, a first ever real-time combined population, tone and geographic analysis and visualization of Tweets on a SGI UV 2000 Big Brain computer. The ability to analyze Big Data in real-time, in sophisticated ways allows organizations from government to enterprises to gain immediate and actionable insights applicable to a multitude of applications. With this capability, researchers, scientists and engineers who are facing increasingly data-intensive problems, can move rapidly from data to insight.
The Global Twitter Heartbeat project performs real-time stream processing of ten percent of Twitter’s 500 million daily tweets as they are posted. The project analyzes every tweet to assign location (not just GPS-tagged tweets, but processing the text of the tweet itself), and tone values and then visualizes the conversation in a heat map infographic that puts tweet location, tweet density and tone into a unified geospatial perspective. With SGI UV, the entire process from ingest to data analysis to heat map runs at a speed that allows visualization of a map frame per second.
The project looked at two significant recent events – Hurricane Sandy and the U.S. Presidential Election. In both events, millions of tweets were sent, making Twitter a significant platform for communicating what was happening at every moment.
For the U.S. Presidential Election, Twitter reported 327,452 tweets per minute as news outlets began to predict an Obama victory. President Obama’s victory tweet quickly became the most-retweeted message in history, setting the record in just 22 minutes after the tweet was posted¹.
As seen through time-lapsed animations of the heat maps, viewers are able to re-live the events as they happened on Twitter. The project observed a massive increase in Twitter traffic as Hurricane Sandy approached, made landfall and took its course over the U.S. eastern seaboard. For the U.S. Presidential Election, the heat maps showed the dynamics of intensity and location of tweets favorable to either President Barack Obama or Governor Mitt Romney over the course of Election Day, from the first polls opening to after President Obama’s victory speech.
"This real-time data analysis approach is like having a new telescope in our hands. We are just seeing the Twittersphere in this way for the first time and we’re still not entirely sure how to make sense of it all and what it tells us, but it is allowing us for the first time to peer in the messy chaotic world that is the heartbeat of our society," said Leetaru. "SGI UV allows users to turn the traditional approach of decision making on its head with the ability to gain insights as they happen, to what I would call a ‘post-demographic’ world in which we can see people’s hopes and fears in real-time person-by-person, rather than just putting them into fixed demographic labels."
SGI UV: The Big Brain Computer
SGI UV 2000 is the world’s largest data-mining machine that can run up to 4096 cores and scale to 64 Terabytes of cache-coherent shared memory at a peak I/O rate of four terabytes per second (14 PB/hour). It could ingest the entire contents of the U.S. Library of Congress print collection in less than three seconds. With as little as 16 cores and 32 gigabytes of memory, SGI UV 2 can start small and seamlessly expand. SGI UV has been used for high performance data analysis in research fields such as cosmology and genomics to commercial applications such as fraud detection
"We are proud to bring the tools and technology to scientists, data analysts and engineers that enable them to find answers to their most challenging problems," said Jorge Titinger, president and CEO, SGI. "Understanding Big Data demands more than what the traditional data analysis approaches can provide and is moving into the realm of high performance data analytics where real-time data can be turned to real dollars and cents for any organization."
Please visit SGI at booth #2631 to see the live Global Twitter Heartbeat displayed on the 3D PufferSphere® display. During the course of the show, the live images of real-time Twitter analysis are available on the Global Twitter Heartbeat project page on facebook.com. Tweet your feedback on the project with hashtag #tweetbeat.
Learn more about the SGI UV and SGI Big Data solutions at sgi.com.
About CIGI
The CyberInfrastructure and Geospatial Information Laboratory (CIGI) at the University of Illinois at Urbana-Champaign researches and develops cutting-edge cyberinfrastructure to advance geospatial sciences and technologies. Dr. Shaowen Wang is CIGI founder and director, and Eric Shook is CIGI technical lead of this project. The project heat maps were generated using cutting-edge data analytics of CyberGIS – a new generation of geographic information systems (GIS) based on cyberinfrastructure and supercomputing environments. Contributions of CIGI to the project are supported in part by the National Science Foundation under grant numbers: BCS-0846655 and OCI-1047916. Any opinions, findings, and conclusions or recommendations expressed in the project are those of the CIGI members and do not necessarily reflect the views of the National Science Foundation.
About Gnip
Gnip is the world’s largest provider of public social media data, serving customers in a range of industries from social media monitoring and business intelligence to finance and government. Gnip delivers more than 100 billion social data activities each month, providing unparalleled access to data from dozens of sources including Twitter, Tumblr, WordPress, Disqus and more. For more information, email info@gnip.com or visit http://gnip.com.
About SGI
SGI, the trusted leader in technical computing, is focused on helping customers solve their most demanding business and technology challenges. Visit sgi.com for more information.
Connect with SGI on Twitter (@sgi_corp), YouTube (youtube.com/sgicorp), Facebook (facebook.com/sgiglobal) and LinkedIn.