Chaos Sumo, a developer of object storage, has developed with it calls the first "intelligent," smart storage for Amazon Web Services S3.
The technology, Chaos Sumo Smart Object Storage service, provides data discovery, management, and analytics as a way to simplify storing of increasing amounts of data using the popular AWS S3 service, said Thomas Hazel, founder and chief technology officer for the Boston-based company.
Chaos Sumo embeds intelligence in S3 via its Data Edge platform, Hazel told CRN.
"So many companies are doing data analytics on object storage," he said. "I wanted to go beyond the noise. Everybody's dumping things into S3 object storage, but they face several issues."
Chief among those issues is the fact that customers are dealing with so many buckets of storage in S3, Hazel said. "Bucket" is the AWS term for a logical unit of object storage.
"Chaos Sumo funds all those buckets," he said. "There's just a click to discover them, which can then be dragged to a virtual bucket without the need to build a new service or ETL [extract, transform, and load] the data to normalize it. Those virtual buckets provide an aggregated correlation between the different physical buckets, which can then be queried with the S3 API."
Chaos Sumo could have created a new file store to handle those physical buckets, Hazel said. "But instead, we created a simple service to create virtual buckets without the need for coding or scaffolding," he said, referring to the process of building a new Hadoop cluster.
A typical use is for IoT, where customer might be looking to store data from multiple devices in AWS S3 because it is easy, elastic and cheap, Hazel said. A customer might want to discover what was stored and then refine it to present to apps for analysis before passing to data scientists to organize the information, he said.
"In a classic case, they would have to build a Hadoop cluster and hire a RedShift database administrator," he said. "Now they get self-service. Data scientists can now use Data Edge to log into S3, find the data, group it as needed, and get results."
Chaos Sumo increases the importance of the data scientist, Hazel said. "We let companies do more with their data scientists," he said. "It may actually decrease the importance of data engineers or data administrators who might be sitting in the middle to organize the data before sending it to the data scientists, business analysts, and business intelligence people."