2026-10-30 –, Main Hall
Social engineers aim at one target: a human whose judgement has gone offline. They pull known levers, like authority, liking and urgency, to make their target compliant. Generative AI frequently pulls the same levers, whether by design or accident, while slowly eroding the very skills we would use to protect ourselves. Coming at this from an odd mix of social engineering, a year of psychology, some basic knowledge of tradecraft, and the AI work I do now, I will lay out the eerie overlap between long-game social engineering and everyday LLM use, the research on how it erodes human skill and what it does to our judgement. While cyber security’s usual defences were built for louder, one-shot attacks, tradecraft may have some useful techniques that could apply here.
Besides my daily work as a data and AI consultant I have a handful of eclectic interests: a hobby level interest in social engineering, a single course in tradecraft and a year of psychology. Lately I noticed an overlap I had not expected.
As cyber security experts know, humans are a great attack surface, and social engineering can be an incredibly effective attack. There are the corporate clichés of the classic phishing email, or the phone call from a very polite supposed customer with a really urgent issue and maybe a screaming infant in the background. Then there are the classic consumer facing attacks, like calls from someone impersonating a loved one who urgently needs money, or “Windows” calling your grandmother personally to help with that unknown danger on her computer.
What gets talked about far less is long-game social engineering. Slowly gaining someone's trust, building rapport over time. Bit by bit their opinions start to carry weight, they start to matter to you, and you find their input invaluable. In the corporate world this may be a new coworker playing a long game and has a surprising amount of overlap with some parts of tradecraft. So where is the overlap with AI? Strangely large language models have been pulling a lot of the same levers in their large customer base for a while now. While the goal is unclear there is the classic large scale gathering of information, the winning of trust through manufactured authority, the relationship building through sycophancy, and manufactured urgency (largely from 3rd parties) to “adopt AI now to boost your productivity”.
What we end up with are study after study on the effects of long-term AI use on human thinking, human skills, human isolation, and in the most extreme cases reported AI psychosis. I am not claiming there is some big conspiracy underfoot. I am pointing out that the disproportionate effects LLMs are having on people may lie in the disproportionate overlap their behaviours have with known techniques from social engineering and tradecraft for the explicit manipulation of humans. I will also touch on some old-school intelligence methods for keeping human judgement intact under surprisingly similar kinds of pressure.
Monika is a data and AI consultant in the Netherlands, with around 10 years across data quality, machine learning and engineering. Her current focus is AI safety and governance, including the EU AI Act, agentic AI risk and prompt injection.
On her blog ai.stewart.wtf she write about AI news and concerns about AI for business audiences, while keeping it grounded in science. You may have seen her as a visitor at the last EMF, WHY2025 or 39C3.