Databricks Targets Cybersecurity Tasks With New Data And AI Platform
Databricks new Data Intelligence for Cybersecurity, built on the same data lakehouse architecture as the company’s flagship data and AI platform, is designed to help security teams more efficiently collect, analyze and act on growing volumes of security-related log data.
With the new offering’s capabilities, security teams can identify and respond to security threats and incidents more quickly and even spot suspicious activity that previously might have gone unnoticed, said Omar Khawaja, Databricks vice president of security and field CISO, in an interview with CRN.
Khawaja said that some core aspects of cybersecurity today are “solved” in that there are established compliance frameworks and mature security tools and controls for preventing, detecting and responding to cyber threats.
[Related: Databricks Closes $1B Series K Funding Round, Exceeds $100B Market Cap]
“The big challenge now with cybersecurity really is how we do it,” he said. “The ‘how we do it’ when we’re in multiple clouds, the ‘how we do it’ when we have hundreds of SaaS providers, the ‘how we do it’ when, instead of having hundreds or thousands of users and service accounts, we are potentially going to have tens of thousands, hundreds of thousands.”
Security teams today face the challenge of collecting ever-growing volumes of security data but face a range of limitations including how much data they can collect and manage, the licensing costs associated with collecting log data from IT systems, the limited number of analysts on staff and the number of incidents security teams have the bandwidth to respond to, Khawaja said.
“As we start to think of the hardest problems from a technical perspective to solve in the cybersecurity space, we believe that these are the ‘how’ problems—and we believe that many of these are really about the data,” he said.
Cybersecurity is already one of the most common use cases for the Databricks Data Intelligence Platform, the company’s flagship product. The new Data Intelligence for Cybersecurity utilizes the same Databricks Lakehouse architecture as that platform with additional functionality and capabilities that make it easier to implement and operate specifically for cybersecurity tasks.
Data Intelligence for Cybersecurity provides the scalability and performance that security operations require today, but at lower cost, Khawaja said. He said that is significant given that cyberattacks are becoming more advanced and widespread as attackers adopt AI.
Data Intelligence for Cybersecurity leverages the data management and AI capabilities of the core platform. It also uses Databricks Agent Bricks, the unified workspace for building production-scale AI agents that Databricks launched in June, for building security-specific AI applications and agents to assist security teams with data analysis and security tasks.
Data Collection Capabilities
The new product can collect huge volumes of log data from firewalls, EDR and XDR tools, and other cybersecurity systems, as well as SaaS applications and other operational systems, for analysis to help detect suspicious activity. Khawaja said some customers already use Databricks to collect multiple terabytes to hundreds of terabytes of security data every day—a couple are even collecting a petabyte or more of security data on a daily basis.
By analyzing more data, Data Intelligence for Cybersecurity helps security teams spot suspect activity they might otherwise miss, see malicious activity and attacks in earlier stages, and better understand the full context of an attack. “The sooner you can identify [attacks], the sooner you can contain the damage,” Khawaja said.
The new Databricks offering also makes it easier to identify “false positive” cybersecurity warnings that are, in fact, not threats—a time-consuming chore for security teams, he said.
Data Intelligence for Cybersecurity provides intuitive dashboards, AI-powered natural language search and real-time analytics capabilities for identifying emerging threats, according to the company.
Working With Technology, Channel Partners
A number of leading IT and cybersecurity companies are already using Data Intelligence for Cybersecurity, which is now generally available, either as part of their internal security operations or to buttress the security they provide around the managed services they offer their customers. Those companies include Arctic Wolf, which announced a strategic alliance with Databricks in July, Palo Alto Networks, SAP (which also has a strategic alliance with Databricks) and Barracuda Networks.
“Cybersecurity is increasingly a data challenge, shaped by the scale, speed, and diversity of telemetry across modern environments. The [Arctic Wolf] Aurora Platform processes over 8 trillion security events each week, and Databricks is part of the foundation that allows us to unify and analyze this data in real time— enabling Arctic Wolf to scale the platform, accelerate AI innovation, and expand our AI-powered SOC to deliver faster threat detection, more reliable protection, and outcomes that security teams can trust,” said Dan Schiappa, president of technology and services at Arctic Wolf, in a statement.
Databricks also said a number of vendor and channel partners have developed integrations with the new product including Abnormal AI, Accenture Federal, ActiveFence, Alpha Level, Anvilogic, Arctic Wolf, BigID, DataBahn, DataNimbus, Deloitte, Entrada, HiddenLayer, Norma Security, Obsidian Security, Panther, PointGuard AI, Rearc, Securiti AI, SPLX, Theom, Varonis, and Ziggiz.
“It’s critical for businesses to integrate advanced data intelligence into cybersecurity strategies,” said Adnan Amjad, U.S. cyber leader at Deloitte, also in a statement. “Our alliance with Databricks helps enable organizations to fully utilize AI-driven insights, helping them transform their security operations to meet the challenges of today’s digital landscape. Together, we are paving the way for a more secure and resilient future."