Con: Accelerating Malware Development

Last week, researchers from security vendors including CyberArk and Deep Instinct posted technical explainers about using the ChatGPT writing automation tool to generate code for malware, including ransomware. CyberArk researchers Eran Shimony and Omer Tsarfati posted findings showing that the tool can in fact be used to create highly evasive malware, known as polymorphic malware. Based on the findings, it’s clear that ChatGPT can “easily be used to create polymorphic malware,” the researchers wrote.

In their report released Thursday, Recorded Future’s research team highlighted several ways that the tool could be used for malware creation in an advanced fashion. Those include training ChatGPT on malware code found in open-source repositories to generate “unique variations of that code which evade antivirus detections” and using “syntactical workarounds that ‘trick’ the model” into fulfilling a request to write code that exploits vulnerabilities.

Notably, ChatGPT can also be utilized to generate the malware payload itself that’s intended for distribution as part of a cyberattack, according to Recorded Future researchers. The research team has identified several malware payloads that ChatGPT is effective at generating, including infostealers, remote access trojans and cryptocurrency stealers.

The position of OpenAI appears to be that when it comes to user requests to ChatGPT for code, ChatGPT functions more like a search engine, and is not capable of doing the level of customization with code-writing that a human would be. “When it comes to code-related requests, I can provide examples of code, explain how to write the code and provide information related to the code, but I don‘t have the ability to generate new code or execute it,” the ChatGPT chatbot said in response to a question from CRN last week. “My approach with code-related requests is more similar to a search engine or a reference book, where I can provide information that I have seen during my training process, rather than generating original text.”

However, Deep Instinct threat intelligence researcher Bar Block told CRN that in her tryout of ChatGPT, she felt that the tool was functioning as something more than just a really good search engine. “When I started checking its limits to see what it can do — when I tried to make it write ransomware — I first started with telling it, ‘OK, write me code in Go that can iterate over directories and encrypt their content.’ And then I asked it to do more things like, ‘Do the same, but also go over subdirectories.’ And then, ‘Put .txt files in each subdirectory.’ And [ChatGPT changed] the code that it initially gave me,” Block said in an interview. “So it did have the capabilities to understand what it was writing, and to change it. It didn’t just search for another thing that it stumbled across during its training. It actually changed the initial input. So it has the capabilities to generate code.”