Cybercriminals are experiencing the same worries as many employees working in legitimate jobs: many are worried that the rise of AI tools and large language models (LLMs) could result in them losing their jobs.
That is according to analysis of chatter on cybercriminal discussion boards, dark web marketplaces and messaging apps by cybersecurity researchers at Sophos Counter Threat Unit (CTU), which has showcased some of the hopes and fears that hackers have around the rise of AI applications and tools.
The research detailed how AI-based hacking tools have become an increasingly common offering on underground marketplaces. Sellers, both established and new, claim to offer AI-powered kits which can aid attackers with generating phishing and social engineering campaigns, developing malware, performing actions within compromised networks and more.
For example, the Sophos blog post detailed how one user is exploring the use of generative AI to overcome language barriers, distribute content at scale, and rapidly respond to any victims who respond to the lure.
Another detailed how they have used AI to generate deepfake audio and video profiles to help build realistic profiles to use for romance fraud. Meanwhile, several sellers offer tools which claim to automate malware coding with AI.
According to Sophos, not everyone has welcomed the rise of AI toolkits for cybercrime with open arms. Users across cybercriminal underground forums and Telegram channels expressed concern about how the rise of AI could reshape roles, pricing and competitive advantage within the criminal economy.
Criminals Fear AI Will Cost Them Work
Some users have explicitly expressed concerns that those selling AI tools could actively take work away from those who manually write their malware scripts and codes. Like many legitimate coders and software engineers, some cybercriminals are worried that AI could be coming for the jobs.
However, cybercriminal forums are divided on the subject and not everyone is convinced of the potential of AI-assisted cybercrime.
Researchers note that there seemed to be a spike in discussions around AI on underground forums following the launch of Claude Mythos Preview, a powerful frontier AI tool which developers from Anthropic have claimed can rapidly identify security vulnerabilities.
Read More: What Fronter AI Models Like Mythos and GPT-Cyber Mean for Modern Cybersecurity
Many members of cybercriminal forums expressed cynicism about the power of Mythos and other frontier LLMs, with claims that corporate executives were “overreacting” to potential of the new models.
However, others continued to express concern about what the development of increasing powerful AI models would mean for them. One user complained that not only would the rise of AI-powered cybercrime take money away from manual malware developers, but they also warned that the ‘products’ would get worse as developers outsource their coding to AI.
Like legitimate workplaces, discussion around the advantages and disadvantages of AI-enabled tools within underground forums looks set to continue for the foreseeable future.
However, the research also noted how the discussion, on both sides of the argument has been led by the most vocal advocators and opponents, and that some threat actors may not participate in forum discussions, instead choosing to quietly explore the technology’s practical limits and tradecraft implications.
The Sophos blog post concluded with advice to legitimate organizations on how they can prepare to face the rise of AI-backed cyber-attacks.
“As AI tooling and capabilities evolve, organizations should continue to prioritize strong cyber hygiene such as timely patching, multifactor authentication (MFA), and passkey use to reduce exposure to established tradecraft and future AI-assisted acceleration,” Sophos CTU said.
“Defenders should also maintain visibility across their environment to identify and mitigate anomalous activity before attacks escalate.”
