Arctic Wolf Looks To Scale Its Aurora Platform Through Databricks Alliance

Arctic Wolf has integrated its Aurora XDR and security operations platform with the Databricks Data Intelligence Platform to help collect, manage and analyze huge volumes of security telemetry and other data to help identify and thwart security threats.

Arctic Wolf and Databricks today unveiled a strategic alliance under which Arctic Wolf’s Aurora security operations platform will be integrated with the Databricks Data Intelligence Platform, allowing Aurora to process what the company describes as massive volumes of security telemetry data in real time.

Executives of the two companies, in an interview with CRN, said developing the data pipeline link between the two platforms will boost the speed and scalability of Aurora’s threat detection capabilities.

The Arctic Wolf-Databricks integration is also a win for Databricks: It’s the latest in a growing number of cases where other vendors are tapping into the Databricks Data Intelligence Platform’s capabilities for large-scale data management and processing operations.

Currently the Aurora platform ingests and processes more than 300 petabytes of data annually.

“We collect a massive amount of data,” said Dan Schiappa, president of technology and services at Eden Prairie, Minn.-based Arctic Wolf, in the interview. “I think we’re approaching 9 trillion security observations a week now and it’s very hard for technology to keep up with that amount of [data] ingestion. We collect it now from over 100 data sources—third party and our own—and we analyze it.”

Along with the growing volumes of data, Aurora is also working with a greater variety of data that’s being collected at an increasing velocity.

Schiappa said limitations with the ability to effective collect and manage all that data was hindering Arctic Wolf’s ability to scale Aurora and the missing piece was a scalable, high-performance data lake system.

“And we looked far and wide for that capability and Databricks was the only [data platform] that could keep up with our scale,” Schiappa said. “What Databricks brings us is the enabling technology to store all of that information, that massive amount of data we have...in a data lake and then we can do our security magic on top of it.”

Aurora, previously known as Arctic Wolf Security Operations, is the company’s XDR platform that performs a number of services to protect against cyberthreats including managed detection and response, incident response, threat intelligence, managed risk and managed security awareness. Arctic Wolf has been expanding Aurora’s endpoint security capabilities this year with technologies acquired through its $160 million purchase of Cylance in February.

Databricks’ Capabilities

The Databricks Data Intelligence Platform is that company’s flagship AI and data system that provides organizations with a unified platform for managing structured, unstructured and semi-structured data. Built on a data lakehouse architecture, the platform performs a range of data management and processing functions including data engineering, data science, analytics, AI and machine learning, data quality, and data governance and security.

“The problem, at its core, is a data problem and an AI problem,” said Omar Khawaja, Field CISO and security vice president at San Francisco-based Databricks, in the interview. “At Databricks, our mission is to democratize the world of data and AI.”

At the Databricks Data + AI Summit in June the company debuted a number of new capabilities for the platform including data workflow and AI agent development tools, a new “Lakebase” transactional database, and the Databricks One interface and Lakeflow Designer tool targeting non-technical business users.

A key selling point of the Databricks platform is its ability to collect and process massive amounts of data from many sources. That fit the bill for Arctic Wolf and its need for a system to process and manage the telemetry data generated by its cybersecurity tools, network and endpoint sensors, and third-party products such as firewalls.

Network traffic data, cloud applications, identity information from DNS servers, email gateway data, and relevant data from other systems also can be added to the mix to enhance security analytics.

“Databricks provides that platform where the data can be ingested, the data can be cleaned and normalized and organized and governed. And then you can act on it using traditional analytics,” Khawaja said. “You can act on it by deploying AI models and then you can present that data and integrate it with the downstream tools and technologies that Arctic Wolf has.”

All that data is analyzed to identify and respond to security threats and attacks across multiple attack surfaces, generate intelligence for threat hunting efforts, and provide data to automate SOC (security operations center) processes.

“We can do so much more in the data lake,” Schiappa said. “It’s a massive enabling technology. We can look for security attacks across multiple attack surfaces. We can look at attacks that we call ‘low and slow.’ They take very short steps, very long periods of time, and if you're just looking in little snippets, you won't see it. But if you can look over longer periods of time, across a broader breadth of visibility, you'll find these things.”

The Critical Role Of AI

Schiappa said AI plays a big role in all this and the AI capabilities in the Databricks Data Intelligence Platform are key to sorting through and analyzing collected security telemetry data. The Databricks platform, for example, will augment the functions of Alpha AI, Arctic Wolf’s portfolio of predictive and generative AI technologies for security operations.

“The way they've designed the technology to work very strongly and heavily with AI is a key focus. Obviously, we can't operate at the scale we operate without AI being at the core of everything that we do, from SOC automation to threat intelligence, to orchestration or response—even to code development,” he said.

‘We’re not just looking at how to automate how a SOC works but reinventing how a SOC works with AI. And we're doing that with Databricks at our side,” Schiappa said.

The two executives said Arctic Wolf and Databricks have been working closely to link Aurora to the Databricks platform. The integration work is complete and the data management capabilities are now available for Arctic Wolf users who can license use of the Databricks platform as part of the Aurora platform.

Schiappa touted the advantages of the Databricks capabilities for its channel partners. “Our channel actually has a strong background in data. So they certainly understand the value that Databricks is bringing to our architecture. They're out talking to prospects about that. They're very attuned to that kind of ecosystem and the value that Databricks brings to the Aurora platform.”

The Arctic Wolf-Databricks integration is the latest in an increasing number of cases where other IT vendors, including Microsoft, AWS and Palantir, are developing links between their products and the Databricks Data Intelligence Platform to tap into its capabilities. Some IT vendors are even building the Databricks system into their own offerings: In February software giant SAP announced its Business Data Cloud that incorporates the Databricks platform through an OEM deal.