GISuser.com

GIS and Technology news for mapping professionals

  • PRESS
    • Submit PR
    • Top Press
    • Business
    • Software
    • Hardware
    • UAV News
    • Mobile Technology
  • FEATURES
    • Around the Web
    • Social Media Features
    • EXPERTS & Guests
    • Tips
    • Infographics
  • Events
  • CAREERS
  • Advertise
    • Advertise
  • Blog
  • Shop
  • Tradepubs
You are here: Home / * PRESS / Business / Tackling the GEOINT Big Data Mess

Tackling the GEOINT Big Data Mess

March 28, 2017 By GISuser

According to the PBS documentary, “The Human Face of Big Data,” we now create as much data every two days as the world generated from its inception through 2003

Naturally, organizations are struggling with the challenge of managing massive amounts of data that is constantly being created by new data sources, as well as traditional data sources that are often dumped into large, hard-to-manage repositories fragmented across multiple networks. The data is being further split and made inaccessible due to security half-measures. Big data is a big mess, and the Geospatial Intelligence (GEOINT) Community must clean it up in order to to meet their objectives.

Of course, there are plenty of technologies out there that tout the ability to make GEOINT data analysis efficient, simple and cost effective. They claim they have the ability to let GEOINT professionals collaborate in new and innovative ways using their data, providing never before seen insights. These claims all go out the window when they see the current state of the data we have to work with.

Much of GEOINT data is hard to access, oftentimes tucked away into numerous physical storage environments on multiple networks, with no regard for how many redundant copies have been made and continue to be created. This is further complicated by the fact that the full metadata sometimes required to solve analytic issues is not always available.
So how can the GEOINT Community start to clean up this big data mess?

1. Purchase the data

The National Geospatial-Intelligence Agency (NGA) has proposed the Commercial Initiative to Buy an Operationally Responsive GEOINT (CIBORG) vehicle for acquiring geospatial data. Perhaps CIBORG could provide transparency in regards to the terms under which NGA and National System for Geospatial-Intelligence (NSG) partners can acquire valuable spatiotemporal data. This dynamic acquisition vehicle could potentially provide data transparency between the U.S. national security community, international partners, humanitarian partners, the U.S. government and private citizens to begin tackling the big data mess.

2. Crowdsource the data

The desire for increased visibility into government data and the rising popularity of citizen science indicate a natural shift toward collaboration. It’s time to start thinking about how organizations like NGA can better leverage crowdsourcing tools to collect and create data. OpenStreetMap and similar initiatives have proven the value of user communities around the world for creating data sets in areas that have previously been underserved, too dangerous to study, or not a priority. What policy changes will need to be made in order to ensure crowdsourced data sets, like those of OpenStreetMap, are regarded as legitimate? Will NGA be willing to let citizen scientists verify and edit them as needed? With a growing number of autonomous data sensors and an increasingly capable community of citizen scientists, it will be essential to legitimize and leverage crowdsourced data sets as much as possible moving forward.

3. Migrate legacy data

Suppose NGA’s CIBORG initiative comes to fruition and they open their unclassified data sets to citizen scientists. We would then have the massive task of wrangling the legacy environments of said data across countless networks, file systems, databases and APIs. Many efforts have been made in the past two decades to patch these legacy environments together so that data can be accessed seamlessly; all with marginal to little success. However, the GEOINT Community has found hope in the the Intelligence Community Information Technology Enterprise (IC ITE) cloud. It offers the possibility, but not yet the reality, of migrating data into environments that will allow the GEOINT Community to get the most out of the modern technologies and strategies available to them. IC ITE could be the key to cleaning up the big data mess.

4. Address complex data challenges

A successful transition to the IC ITE cloud involves a four-step process that begins with taking a mission needs inventory by creating specific user stories that define the most common activities supporting GEOINT mission threads. Next, inventory will need to be taken for government, commercial and public GEOINT data sources. The data will then need to be decoupled from analytics by storing GEOINT content in IC ITE cloud-based open storage systems (e.g., Hadoop, HBase, Accumulo, Elasticsearch) that provide multiple ways of accessing content such as ArcMap, QGIS, full-text search, Google Earth, etc. Lastly, we’ll need to simplify data discovery by communicating to analysts, software engineers, data scientists and leaders how to access data for each GEOINT mission thread.

Once this occurs, the U.S. government and GEOINT Community will then have the ability to introduce big data and machine learning tools into their data sources. These powerful technologies will let data be stored at the level of its classification with cross-domain access for users on every network.

Micro-services and Open Geospatial Consortium (OGC) web services will be deployed on the elastic cloud, providing unprecedented flexibility and scalability. In turn, the data will be easily accessible for cataloging and a wide range of indexing schemes, greatly improving discovery and access. This will also undoubtedly open the floodgates for conversations regarding new ways individuals, teams and communities can collaboratively interact with each other among the data. At this point, the big data mess could be nearing its end. But what will be in store for the future?

A Bright Future

Although a daunting task, getting a handle on the big data mess gives us much to look forward to. By moving relevant data into an elastic cloud with basic standards for data structure and access, we will radically transform the face of GEOINT data. The GEOINT Community will at last be able to fully leverage the global wealth of data available to them to provide the U.S. with a time-dominant decision advantage when it comes to international affairs. Machine learning tools will sort through the data, identifying what’s known and queuing up the unknown for analysts to learn more. Invaluable geospatial data will be curated in collaborative efforts by the modern analytic workforce. The volume of data being dispatched will be massive and continue to grow, but the fidelity of the data will be unparalleled, and its update cycle will be much more rapid than it is today. When we solve the big data mess, the GEOINT Community will have the power to turn intelligent insights into meaningful actions like never before.
Authors

Anthony Calamito serves as the Chief Evangelist & VP of Product for Boundless and is responsible for product strategy and outreach, educational initiatives, and our ongoing commitment to the open source community.  Anthony is also a Steering Committee member at LocationTech, and and a Fellow of the American Geographical Society. His commitment to geography education and community outreach extends beyond Boundless — he is also an adjunct instructor at George Mason University.

Chris Tucker is the creator of Mapstory.org, a companion to Wikipedia, and Chairman of the Board of Trustees of the Mapstory Foundation. In addition to his work at Mapstory, Chris also manages Yale House Ventures, a portfolio of technology companies, social ventures and public entrepreneurship initiatives, and can also be found at a number of think tanks he supports including the Center for National Policy and the Institute for State Effectiveness.

Save

Related Articles on GISuser:

  • Big data companies roundupBig data companies roundup
  • The Top Jobs in 2021 Will all Be Related to Big DataThe Top Jobs in 2021 Will all Be Related to Big Data
  • Open Big Data to help Farmers weather climate changeOpen Big Data to help Farmers weather climate change
  • Big Data BasicsBig Data Basics
  • Big Data: Deep Learning for Financial Sentiment AnalysisBig Data: Deep Learning for Financial Sentiment Analysis
  • Glassdoor Reveals The 50 Best Jobs In America For 2017 –  Data Scientist Takes Top SpotGlassdoor Reveals The 50 Best Jobs In America For 2017 – Data Scientist Takes Top Spot

Filed Under: Business, EXPERTS, Top Features, Top Press Tagged With: Big Data






Editor’s Picks

Unique Hand Drawn Map of London

Unique Hand Drawn Map of London – Ten Years in the Making

DigitalGlobe Satellite Captures Dramatic Images of Alberta, Canada Oil Sands wildfire #fmmfire

AutoCAD for Mac 2015 and AutoCAD LT for Mac 2015 Now Available

Introducing the new, HP Z Series Workstation Portfolio – Event recap, images, video and key numbers

See More Editor's Picks...

Feed

  • RSS


GIS jobs


Recent Features

Digitalization and Simulation at the North Pole

Natural Gas District & GIS-Centric Asset Management

5 Ways Technology Is Affecting The Sharing Economy

Spotlight – Township Canada: Explore Canadian Legal Land Descriptions on a Map

GeoGeeksinCars – The one in “Tippy” the right-hand drive Mitsubishi, at DevSummit

More Posts from this Category

Categories

Recent Posts

  • Esri Announces Winners of the 2022 ArcGIS Online Competition
  • KBC WhatsApp Lottery Winner 2022 List – KBC Head Office Number +19188444477 – KBC Sim Card Lucky Draw 2022
  • Animation explainer for a trading application
  • Ordnance Survey supports the British Heart Foundation’s national database of defibrillators
  • Why are AI and Machine Learning Important?

RSS Career Tips

  • Four Characteristics of the Best Virtual Employees
  • 8 Questions Employers Should Ask About Coronavirus
  • Jobcase Launches Free Unemployment Resource Center for Workers Impacted by COVID-19
  • Must-Have Gadgets and Technology for the Remote Office Worker
  • Must-Have Gadgets and Technology for the Remote Office Worker


shop for geogeek swag

RSS Tech pubs

  • Breaking the Code: How Low Code Is Changing Development
  • The Essentials of IT Management - 2022 Kit
  • Achieve a 25-50% reduction in payment postings and charge entry expenses

RSS Computers

  • The Essentials of Information Security Kit: Includes a Free PC Security Handbook - 2nd Edition eBook
  • The Idiot's Ultimate Guide to Build Your Own PC
  • Windows... On Speed

RSS HR Tips

  • At Work: The 2022 Workplace Trends Report
  • The 2022 Guide To Creating A Great Visitor Experience
  • Enterprise Executives Say Hybrid Work is Here to Stay

RSS Gov Tech

  • The Bipartisan Infrastructure Bill: How you can win more bids
  • How Connected Vehicle Data Can Transform Public Sector Decision-Making
  • Getting started with personalization at scale

RSS UAV news

  • Draganfly Deploys Versatile Situational Assessment Drones to Ukraine
  • Ocean Specialists Inc. Launches “USVs as a Service” Business Line
  • UAS Drone Subsidiary, Duke Robotics’, Sniper Drone showcased at a leading International Airshow in the Asia Pacific Region

RSS Geojobs

  • IT Endpoint Lead
Copyright Spatial Media LLC 2003 - 2015