XSEDE12

Science: High performance data mining of social media

Abstract: Difficulties in designing a system to mine social media lie in web service restrictions, legal permissions and security, as well as in network and execution engine latency. Our data mining algorithm tests on Twitter data on small scale at F .90 for accuracy at identification of streets, buildings, place names and place abbreviations. But for large scale, to maintain accuracy and efficiency, we have had to develop techniques to manage the real-time data load. Our contribution algorithm and architecture strategies for multi-core and parallel processing that exclude major program refactoring.