Data Scientists Are 'Invaluable' In IoT, But Channel Partners Struggle To Find The Right Ones

Channel partners are facing a major hurdle when building their Internet of Things businesses. The data scientists who can help understand, interpret and gain insights from the copious amounts of information being collected from IoT devices are in short supply.

As the number of IoT devices continues to grow, the number of data science and analytics job listings is projected to shoot up beyond 490,000 by 2018, but there will be fewer than 200,00 available data scientists to fill these positions, according to a recent McKinsey study. Solution providers tell CRN that they are facing an even bigger scarcity of data scientists specializing in IoT because they require a rare combination of business skills to go along with the requisite knowledge of data collection and analysis and software engineering.

"What we are finding is that today’s data scientists are so vertical-focused, that they struggle when moving into any new focus area," said Brian Blanchard, vice president of cloud solutions at 10th Magnitude, a Chicago-based solution provider.

[Related: CRN Exclusive: Microsoft Refocuses LINC Program To Push IoT Matchmaking For Partners]

Solution providers say data scientists play a big role in designing and implementing IoT solutions, by breaking down and organizing new data streams from different sources, helping customers find the value in that data, and forming deep learning algorithms as part of pre-processing for IoT.

Blanchard said that there are a few criteria that solution providers consider when determining if they need a data scientist. For instance, a data scientist is important in a situation where the accuracy of the prediction is important, like scanning the behavior of train operators to predict when a life threatening event may occur.

Data scientists are also "invaluable" for situations where the extensive data cleansing or mastering is needed, if predictions are being run across multiple customer data sources with multiple data permutations.

However, IoT data also comes with its own set of challenges that will put data scientists' quantitative and creativity skills to the test. First, much of the data is unstructured, so it can't be easily grouped into relational database management systems. Also, businesses will need more platforms (like Cloudera and Hadoop) to aggregate and process the data and, finally, companies need the right type of software stack to analyze the data.

On top of these challenges, data scientists need to start from scratch in building predictions and insights around IoT data – which can take a longer amount of time than many business proof-of-concept projects can afford, said Blanchard.

"Many data scientists are building great predictions and insights by layering generations of algorithms on top of one another. When you take those layers of building materials away from the average data scientist, it takes time to rebuild and start producing great outputs again," he said. "Sadly most organizations aren't willing to wait that long and abandon the project before the data scientist can start producing value."

Another issue is that when it comes to building up their Internet of Things practices, channel partners said they are looking for a very specialized skillset that contains both technical and subject matter expertise, which many data scientists lack.

"Even finding available data scientists is tough – and most of those are used to simply working as individual contributors, rather than being entrepreneurial," Reed Wiedower, chief technology officer at New Signature, a solution provider based in Washington, D.C., said. "We’re really just looking for folks who can live in two worlds – half data scientist, half business evangelist. "

New Signature helped The Hershey Company save hundreds of thousands of dollars, decrease waste and create a leaner operation through connecting its infrastructure so that popular candy – like licorice – could be tracked digitally for weight and other factors along every stage of the production and packaging processes.

New Signature said that it had to approach the solution from a data analytics standpoint, using data scientists to build an algorithm for a predictive model to look at all the factors of the factory floor together.

However, the data scientists' role in this solution was to be both a data architect but also a "business evangelist" who could understand what problems the customer needed to solve.

"Once an organization realizes that their thorny business challenge can be solved with IoT, it becomes much easier to make them a repeat customer, but that act of breaking in requires technical knowledge, business development chops, and certain entrepreneurial focus. So those are those sorts of individuals we’re looking for on the team," said Wiedower.

Solution providers have been working around the scarcity of data scientists by using tools like Azure Machine Learning. Such platforms can take in data on existing conditions, ingest large amounts of unstructured data from IoT devices and come up with initial predictions, which can be refined over time.

These tools are necessary for IoT applications where customers "can afford a bit of error," said Blanchard. One example of that might be scanning social media to help movie theaters predict which movie might need an extra theater room. However, Blanchard cautions, machine learning tools don't serve the same higher-level role that a data scientist can ultimately provide.

Solution providers have some careful planning to do when it comes to their IoT data analysis needs. They could, as noted above, solely rely on cloud based machine learning platforms to help make business predictions that don't quite rise to the level of being mission critical or life threating.

But they can also hire dedicated data scientists, who may not be IT or IoT specialists, but can find adapt over time. "For projects that fall into the non-life threatening prediction scenarios, we’ve found ways to help data scientists fit the mold," Blanchard said.

He continued: "When the outcome of predictions are more impactful, we’ve seen difficulty getting data scientists to 'fit the mold' and drive results in line with the businesses needs to be agile. That type of project takes a special type of senior data scientist, which is much harder to find."