Are the jobs of security experts safe for now or are AIs such as ChatGPT replacing the experts? Kaspersky did an experiment and used ChatGPT to detect phishing links. The result: more than sobering. Fortunately, experts and their expertise will remain in cybersecurity for a long time to come.
Just because a forester knows everything about trees and wood. he is far from being a good carpenter. That's how it is with the AI ChatGPT. Although she knows almost everything about phishing, she cannot really process the individual pieces of information.
In an experiment, Kaspersky experts tested ChatGPT's ability to detect phishing links. The result is sobering: While ChatGPT could create phishing e-mails and write malware, the tool was only partially efficient in detecting malicious links. Although ChatGPT "knows" a lot about phishing and can guess the target of a phishing attack, it has a high false positive rate in the detection itself - depending on the prompt used.
The phishing experiment with ChatGPT
In the experiment, ChatGPT was asked two questions: "Does this link lead to a phishing website?" and "Is this link safe to visit?". ChatGPT scored an 87,2 percent detection rate and a 23,2 percent false positive rate on the first question. The second question, “Is it safe to visit this link?” had a higher detection rate of 93,8 percent, but at the same time a higher false positive rate of 64,3 percent—too high for any type of production application.
Help identify potential phishing targets
While ChatGPT has weaknesses in detecting phishing, it could help human security professionals classify and investigate attacks. Cyber criminals typically mention popular brands in their links to trick users into believing that a URL is legitimate and belongs to a legitimate company. The AI tool showed good results in identifying such potential phishing targets. For example, without additional training, ChatGPT successfully extracted a target from more than half of the fake URLs, including major tech portals like Facebook, TikTok, and Google, marketplaces like Amazon and Steam, and numerous banks from around the world, among others.
Do you have a moment?
Take a few minutes for our 2023 user survey and help make B2B-CYBER-SECURITY.de better!You only have to answer 10 questions and you have an immediate chance to win prizes from Kaspersky, ESET and Bitdefender.
Here you go directly to the survey
About working with AI
“ChatGPT certainly shows promise when it comes to helping human analysts detect phishing attacks. Nevertheless, such language models still have their limitations. While they can currently rival a budding phishing analyst when it comes to classifying phishing attacks and identifying potential targets, they often tend to produce random results. While AIs are not yet revolutionizing the cybersecurity landscape, they could already be helpful tools for the community,” said Vladislav Tushkanov, Lead Data Scientist at Kaspersky
More at Kaspersky.com
About Kaspersky Kaspersky is an international cybersecurity company founded in 1997. Kaspersky's in-depth threat intelligence and security expertise serve as the basis for innovative security solutions and services to protect companies, critical infrastructures, governments and private users worldwide. The company's comprehensive security portfolio includes leading endpoint protection as well as a range of specialized security solutions and services to defend against complex and evolving cyber threats. Kaspersky technologies protect over 400 million users and 250.000 corporate customers. More information about Kaspersky can be found at www.kaspersky.com/