Homepage This page's url is: -crn- Rankings and Research Companies Channelcast Marketing Matters CRNtv Events WOTC Jobs HPE Zone Intel Partner Connect Digital Newsroom Dell Technologies World Newsroom Dell Technologies Newsroom HP Reinvent 2020 Newsroom IBM Newsroom The IoT Integrator Lenovo Newsroom NetApp Data Fabric Intel Tech Provider Zone

Information Blizzard: Snowflake Computing Adds AWS-Based Data Loading Service To Its Cloud Data Warehouse

The new Snowpipe technology automates data loading jobs, reducing system latency and speeding decision-making tasks.

Snowflake Computing is making it easier for organizations to move data into its cloud data warehouse system with the launch of its new Snowpipe automated data loading service.

The development of Snowpipe is a response to growing demands for technology that can take structured and semi-structured data streaming from online and operational applications and move it into systems like Snowflake for near-real-time analysis.

"What we've been hearing from our customers is that continuous data ingestion is key," said Matt Glickman, Snowflake's product vice president, in an interview with CRN. "More data is coming in in a continuous form."

[Related: The 10 Coolest Big Data Products Of 2017 (So Far) ]

Snowflake Computing, based in San Mateo, Calif., provides a SQL-based data warehouse system running on Amazon Web Services.

The new Snowpipe service addresses the fact that more businesses are storing huge volumes of data in AWS Simple Storage Service (S3) – an accelerating trend given the rapidly falling cost of data storage – and are looking for ways to quickly tap into that data to support decision-making tasks.

Traditional data warehouse systems rely on batch-loading technology and processes to move data into a data warehouse. But the latency involved is proving to be too slow for today's fast pace of business.

Businesses and organizations are increasingly using S3 for data staging or "data lake" storage, Glickman said. And more of that data is machine-generated from operational systems and Internet of Things networks.

Data warehouse usage is also evolving from analysis of historical data to supporting customer-facing applications that require near-real-time responses and even predictive analysis.

The Snowpipe automated service "listens" asynchronously for data coming into S3 and loads it into the Snowflake data warehouse. Snowpipe is built using AWS Kinesis, the company's real-time stream processing "data pipeline-as-a-service," Amazon Elastic Compute Cloud and other AWS systems.

"We're further embracing S3 as a core cloud foundation that all Amazon customers are using," Glickman said.

Snowflake Computing works with a number of systems integration and solution provider partners including Slalom, Cognizant, Wipro, Intricity, DecisiveData, Cervello and Useready.

Partners who provide data integration and ETL (extract, transform and load) services can utilize Snowpipe as a way to manage their data streams and transformations without data staging, Glickman said. Systems integrators can use Snowpipe to lower data integration costs and focus more on high-level services for clients.

Snowflake plans to charge for Snowpipe use based on per-second compute utilization to load data.

Snowpipe is currently in a public preview stage, allowing customers to try out the system, and will be generally available in early 2018.

Back to Top

related stories



sponsored resources