By 2020, a city of 1 million people will generate 200 million gigabytes of data every single day. Meanwhile, the volume of all data stored globally is predicted to rise from 1.4 zettabytes to 6.2 zettabytes over the next two years.
How will we store and access all of this data? Structured Query Language (SQL) is not going to cut it.
Looking at the big picture, we have to figure out what data is actually valuable versus what is just noise. We then have to find a way to process and use that data. Even the best mainframes will not suffice. That is where we need to start looking to distributed computing.
At its core, the Internet of Things (IoT) is all about the physical web. It is ubiquitous in all aspects of our lives, where everything is connected and sending data. How can we use the combined computing power of distributed devices – which might not be operating at full capacity – to find the value buried in mountains of data?
Once every device has connectivity built in, we will need to think in novel ways to use available data to make the world safer and more secure.
There will be many instances of unexplored data sets, compared to more heavily analyzed data sets such as financial services data. We must use that data so the combination of the machine and the service is greater than the sum of its parts.
Part of what will enable us to conquer this challenge will be the efficiencies we create in scaling data science models. Creating a data science model is one thing. But deploying it and maintaining it at scale is the key.
Even with all of this data, it still might not give us exactly what we need to accurately predict uncommon and oftentimes catastrophic failures – events like an airplane tire blowing out or a critical part of a nuclear power plant failing.
For scenarios where we can crash-test machines in controlled environments, it is not realistic to do such a thing frequently. For example, it is not cost-effective to test-crash train upon train to generate data that might alert us to problems.
We need to find new and creative ways to use the data that we do have to spot patterns that can give us a heads up on significant problems lurking on the horizon.
What else is in store for us on our journey to a fully connected world? Learn more in our report, Five Challenges We Must Solve on the Way to a Fully Connected World: