Team Uptake just returned from AWS re:Invent — Amazon Web Services’ global customer and partner conference that brought together 50,000 people in Las Vegas. Uptake was a speaker, exhibitor and sponsor of this year’s event. From hosting educational sessions on stage to giving demos of our platform and applications in our booth, it was an action-packed week.
Two Truths About Solving Hard Data Challenges from the AWS re:Invent Show Floor
From hosting educational sessions on stage to giving demos of our platform and applications in our booth — see our key takeaways from AWS re:Invent.
Amid all the AWS announcements and activities, two important truths stuck with us — both of which underpin the important work businesses are doing across the data engineering and industrial AI space.
Truth #1: Building data systems that scale for data science and IoT is hard work.
This shouldn’t come as a surprise, but it’s easy to forget. At conferences like re:Invent you can take a step back, get out of your daily grind and actually reflect on the work you’re doing. Spending a week with some of the world’s best engineering talent — talking about our common struggles and sharing how we’ve solved some of our toughest data challenges — reminded us that even when companies make it look easy, it is actually really hard work.
At Uptake, we face the added challenge of building data systems capable of scaling for industrial data science and IoT, which is incredibly difficult work. First, industrial data is dirty. It’s out of order, highly volatile and has a lot of room for error. Second, industrial data comes from extremely old and disparate systems, making data integration tricky. Third, the sheer volume of unordered, unfiltered data coming in from industrial systems is enormous.
Plus, we haven’t even gotten to AI and machine learning yet. With all of the work that needs to be done to prepare, clean and integrate data — in a repeatable way — it’s easy to understand why so many companies still lack a comprehensive data science strategy. Conference attendees we talked to said it’s a challenge just to get the data they need to do their work and reaffirmed what we already know. For many companies:
- Production deployments for data science are typically painful and infrequent.
- Data silos (and sometimes company politics) often get in the way of data accessibility.
- Lack of a data science strategy means data scientists often do all of the hard work — from business justification and data wrangling to data analysis and modeling.
It’s a good reminder that even with really great tooling, including many of the services launched at re:Invent each year, you still have to pull together talented engineers and data scientists that know how to attack the problems you’re solving. Advanced assembly lines are great , but you still need to bring together the technical expertise to know how all these systems work together and the domain expertise that ensures the final product that continually evolves while delivering outcomes that really have an impact on revenue, resiliency, and everything in between.
If you want to dive deeper on the issues above and how Uptake has solved our own data engineering challenges, check out the session we presented at the re:Invent Startup Stage, Data Engineering the Startup Way. You’ll also get to hear from my colleagues CJ Woolard and Brad Boven as well as more info on how Uptake has evolved as a startup and continually improved the Uptake Platform.
Truth #2: We’re all in this together.
Yes, you want your company to win. However, at the same time, technology that has the openness and flexibility to integrate with other systems is the future of our connected world. Additionally, there are many really smart engineers working on evolving the approaches we have today so we can solve the emerging hard problems of tomorrow. When we work together, whether that’s collaborating on open source or partnering with like-minded companies, it’s a win-win.
Uptake works with many technology partners to empower our customers to succeed. One of the areas we partner in is edge technology and data integration. We’ve found that industrial businesses are still in the exploratory phase when it comes to connectedness at the edge with many disparate SCADA and operations systems that integrate via wire protocols old and new this makes it even more important for us to provide our customers with options for seamless connectivity.
At re:Invent this year, AWS announced four new services that make it easier to build IoT applications and act on data at the edge. This is a challenging space to work in, which is why Uptake is an advanced partner of AWS IoT SiteWise — a new managed service that collects, structures, and searches data from industrial facility devices. This means customers integrating with AWS IoT SiteWise can also easily onboard their data to Uptake’s platform and put our industrial data science know-how to work.
Fellow Uptaker Alex Drake presented at the AWS Launchpad to show how AWS IoT SiteWise works with our customers’ machine data and can easily use the Uptake Platform. We even had a 3D printed wind turbine measuring blade acceleration connected through the SiteWise gateway for extra engineering points. Check out the Twitch archive of the demo here.
What would events like re:Invent be without a little fun?
We genuinely enjoyed every conversation we had with attendees at our booth. Our barista, beer cart, servers and swag certainly helped us bond over the challenges of building data systems that scale.
Even though some might call us crazy for spending our days solving incredibly hard data engineering problems, it’s what we at Uptake set out to do from day one. There is so much possibility in what we can do to make the world safer and more resilient using what we know about data engineering and data science. And if you ask us, we’d choose to jump in, solving the hard problems all over again, because that’s what we do, we solve the hard problems, every single time.
