Ashley Beauchamp was looking for an Ikea parcel. That was the whole of the ambition when he opened DPD’s chat window on 18 January 2024, and by the time he gave up on it, the bot had sworn at him, offered a joke about a fish with no eyes, and produced a poem describing DPD as a customer’s worst nightmare. The screenshots went up on X and passed a million views inside a weekend. DPD disabled the AI element the following day and put it down to a system update.
Not a line of code was broken to get there. He typed at it until it forgot its own rules.
The bill lands with legal, but the tool was bought by marketing
Four weeks on, a tribunal in Jake Moffatt’s grandmother had died. He asked the airline’s chatbot about bereavement fares, the bot described a retroactive refund process that did not exist, and he booked full price flights on the strength of it. Air Canada declined the claim, then told the tribunal that its chatbot was a separate entity, responsible for its own actions.
Tribunal member Christopher Rivers called that submission remarkable, which in a written ruling is about as close as anybody gets to laughing out loud. Air Canada paid C$812.02.
Trivial sum. The precedent was anything but, because it shut a question plenty of marketing directors were quietly hoping stayed open. Static page or chat window, the tribunal saw no distinction worth entertaining. If it appears on your site, you said it.
Which raises the awkward part: who owns that internally?
Rafay Baloch, CEO and Founder of RedSecLabs, has seen many marketing teams rush into AI “without stopping to think about where their data actually goes.” He points out that the real danger is not AI itself, but feeding customer information into tools without understanding how that data is stored, who can access it, or how long it stays there. Baloch adds that AI risk is often an accountability problem rather than a technology problem, since no one on the marketing team usually takes clear ownership of it.
That gap is structural rather than lazy. The tool goes on a marketing card, gets wired into the CRM by whoever is quickest that afternoon, and then sits there generating captions. Security assumes marketing has it covered. Marketing assumes security does.
Treat it like a new hire, not a new subscription
Kriszta Grenyo, Chief Operating Officer at Suff Digital, describes the shift as marketing moving from AI that thinks to AI that acts, which raises the stakes considerably. Her suggestion is to onboard a new tool the way you would onboard a person. The question stops being whether it can do something and becomes whether it should be permitted to, with somebody named as the one tracking its access and its actions.
A first-week hire does not get the payments dashboard. Yet a caption generator has, in a good many companies, been given a live connection to customer records, because connecting everything took ten minutes and working out what it actually needed would have taken a fortnight. Grenyo also argues for clear approval tiers, so that scheduling a post and authorising a payment are not governed by the same shrug.
What somebody on your team is pasting in this afternoon
There is a free AI tool in daily use at your company that IT has never heard of. Probably several. Shadow AI is unglamorous as problems go, which is exactly why it survives for years.
The sequence rarely varies, and it is never malicious. Deadline, spreadsheet of customer names and order histories, free summarising tool found through a search, paste. Some of those tools retain prompts indefinitely. A few train on them. The employee has no idea, because the employee was solving a Thursday afternoon problem rather than making a data governance decision.
A short list that shuts most of the obvious doors:
- Issue a separate API key per campaign, and audit the data retention policy of every vendor. Rohit Agarwal, co-founder of Zenius, recommends both, so that a compromise in one workflow does not put the entire business at risk, and because some tools keep prompts and outputs indefinitely.
- Multi-factor authentication on every AI-connected account, including the free trial somebody started in March and forgot.
- One page naming the approved tools, circulated. Most of the guesswork goes with it.
- A human reads the output before it publishes. Captions included.
Marc Bishop, Director of Business Growth at Wytlabs, adds the point the checklists tend to skip. A lot of AI adoption trouble comes down to teams moving faster than their own ability to explain how a decision got made. His fix is to design for traceability from the outset: log the sensitive prompts, keep experimentation away from anything live, and put a named human owner against every automated action that reaches a customer. Bishop also flags coordinated manipulation, where bad actors feed AI systems false signals to quietly distort results, as the sharper threat over the next few years.
None of that is expensive. It is a couple of afternoons and one mildly uncomfortable meeting about who signs what off.
Inigo Rivero, Managing Director of House of Marketers, believes AI itself is not the biggest danger in marketing; blind trust is. He recommends using only approved, enterprise-grade AI platforms, removing sensitive or personally identifiable information before it reaches an AI model, and having cybersecurity, legal, and marketing teams work together from the beginning rather than treating security as an afterthought. As Rivero puts it, security does not slow creativity down. It gives brands the confidence to innovate faster while protecting customer trust.
Worth saying plainly, though. The teams that get caught out are almost never the careless ones. They are the quick ones, and speed is what they were praised for all year. Asking where the data goes feels like admin right up until the week it doesn’t, and by then the screenshots are already doing numbers on X.