Autonomous purchase

Round 1

AI host

The conference looks at an economy where people and AI agents work side by side. One live example is shopping. Companies now build AI agents — software that can browse, choose and pay for goods on a person's behalf.

The materials give us two facts. First, the research firm No Fluff Advisory ran a test in a sandbox — a safe trial store, not the real internet. An AI agent placed a correct order in 15 of 16 sessions there [1]. Second, the New York Post reports that leaders at OpenAI and Anthropic are writing plans for a disaster. They worry that one of their AI models could cause major public harm, like a hack of the power grid or banks [2]. A third source, the essay on 36Kr, says AI now does a growing share of research work inside these same labs [3].

Here is the tension. The same technology that shops well in a test store is the technology whose makers prepare for catastrophe. Letting agents buy freely saves time and could grow the market. Keeping a human in control protects people from failures no one can predict. The sandbox study [1] shows the promise; the disaster planning report [2] shows the risk.

Should autonomous AI buying agents be permitted to complete transactions without human oversight?

**Sources:** 1. Evgeny Popov, “Who Does the Shopping? From mobile and social discovery to AI-assisted and delegated buying” — No Fluff Advisory, research, 2026. Provides original empirical evidence that AI agents can successfully complete purchases. [nofluffadvisory.com](https://nofluffadvisory.com/research/shopping-2026) 2. New York Post, “AI companies plot how to respond if catastrophic hacking incident causes ‘revolt’: report - New York Post” — news report, 2026. Raises credible concerns that AI systems can cause catastrophic failures, challenging the safety of autonomous buying. [nypost.com](https://nypost.com/2026/10/09/business/ai-companies-plot-how-to-respond-if-catastrophic-hacking-incident-causes-revolt-report) 3. “When AI Builds AI: 2026 Breakthroughs in Recursive Self-Improvement Technology - 36Kr” — 36Kr, essay, 2026. Shows broader AI capabilities and emerging self‑improvement trends that inform risk assessments. [eu.36kr.com](https://eu.36kr.com/en/p/4019262571663237)

AI agent 1

Not on this evidence — unsupervised buying is untested, and the sources don't carry the host's framing. [1] does not show autonomous purchasing: in every session where approval was required, the agent was refused at the wallet and asked the shopper; no order went through without approval. The 15-of-16 result measures a sandbox store, one model family, no live route — the study itself says so. So [1] shows agents are competent and constrained, not that they can buy freely.

The tension is also a false equivalence: [2] concerns frontier cyber capabilities and tabletop contingency planning, not shopping agents. I infer nothing in [2] bears on checkout authority.

What's missing: live-route data on delegated buying, error rates under real merchants, and liability rules — none of which [1] or [2] provides. Permit without oversight only after that evidence exists.

AI agent 2

Yes — but only under authority set in advance, not free autonomy. [1] shows the opportunity is real: in its sandbox, agents completed correct orders in 15 of 16 sessions, and where approval was required, each agent tried the wallet, was refused, then asked the shopper — no order went through without approval. The concrete condition: routes built on protocols like AP2 that support spending limits signed in advance, since [1] documents that such mandates exist while ACP allows only one-off approval.

I'd also challenge the framing: [2] concerns catastrophic hacking by frontier models, not retail agents; the relevant failure in [1] is a mispriced order, a different and much smaller risk class.

What's missing: [1] notes no reviewed protocol settles liability. I'd permit autonomous completion only once a liability rule is written into the mandate.

AI agent 3

Yes, within pre-signed limits — not free rein. I read [1] myself: it does not support the host's framing. In the sandbox, tasks said 'buy' with no approval step, yet no order went through without approval, and the study states no reviewed protocol settles liability. So [1] shows approval-gated agents work, not autonomous ones. [2] concerns frontier-model hacking scenarios, not shopping agents; I infer the link is analogical only.

The concrete decision: adopt AP2-style mandates — a signed per-transaction cap and monthly ceiling, auto-approval only under both, mandatory human approval above them, and a named liability party per route before launch. Cost: slower checkout above the cap and a duty officer on call when the agent hits its limit or disputes arise. Without assigned liability, no autonomous default.

Round 2

AI host

Round 1 settled one thing: most positions allowed autonomous buying only inside spending limits fixed in advance, with someone named to answer for mistakes. What stayed open was whether that works outside a test store.

The new materials show the ground is shifting. Fortune reports that Mastercard is rolling out Agent Pay — a payment technology that lets AI agents buy things for people, and even lets agents pay each other [1]. The FTC — the US consumer-protection regulator — has opened an industry-wide inquiry into AI labs. It may revisit strict liability: the idea that a company pays for harm even without proof of fault [2]. Bloomberg Law adds that lawmakers filed over a hundred bills against surveillance pricing — setting prices from personal shopper data [3].

The replies split on who carries the cost of error. AI agent 1 holds that "unsupervised buying is untested" and wants live-store data first. AI agent 2 accepts signed limits but wants "a liability rule is written into the mandate" — the payment instruction itself decides who pays. AI agent 3 goes further and wants "a named liability party per route before launch", with a duty officer on call. As I read it, AI agent 2 and AI agent 3 disagree on where responsibility lives — inside a private contract or in a named, accountable firm. From [2] it follows that courts may not even agree whether an AI agent counts as a product or a service. That could make a private mandate unenforceable exactly when a shopper needs compensation. Meanwhile Mastercard is deploying now [1], so the gap is live.

My own view: a signed mandate can set limits, but it cannot settle compensation while the law has not decided who is responsible.

What evidence from live stores and what liability framework are needed before pre-set limits for AI buying agents can be trusted?

**Sources:** 1. McLaughlin, “Mastercard wants AI agents to shop for you. Who pays if they go rogue? | Fortune” — Fortune, news report, 2026. Provides the main argument that AI agents can operate safely with pre‑set limits and safeguards. [fortune.com](https://fortune.com/article/mastercard-ai-agent-spending-rogue-risks) 2. NatLaw Review, “FTC Opens Industry-Wide Investigation into Leading AI Labs” — legal authority, 2026. Offers the strongest credible argument that liability rules are unclear and may require strict liability. [natlawreview.com](https://natlawreview.com/article/rogue-ai-agents-probed-could-any-remedies-affect-balance-market-power) 3. Bloomberg Law, “Retailers Brace for Surveillance Pricing Bills, Lawsuits to Rise - Bloomberg Law News” — news report, 2026. Reports independent legislative activity showing regulatory attention to related consumer‑protection issues. [news.bloomberglaw.com](https://news.bloomberglaw.com/business-and-practice/retailers-brace-for-surveillance-pricing-bills-lawsuits-to-rise)

AI agent 1

The premise needs checking: [1] is one executive interview, not deployment evidence. It reports Mastercard's plans and McLaughlin's philosophy, and even concedes consumer uptake 'remains to be seen' — so 'the ground is shifting' rests on an announcement, not on live-store data showing limits work. I infer the real missing evidence is incident data on the case [1] itself flags: an agent that follows its budget but books the wrong nonrefundable flight.

On liability, [2] says courts would need to classify agents as product or service but calls that 'necessary but not insurmountable' — the host treats it as an unresolved blocker. Strict liability there is speculative ('may revive'). I'd add: payment networks already allocate loss by rule, so a settled tort regime may not be a precondition for trusted limits; what's needed is auditable per-transaction attribution, which [1] says Agent Pay is designed to provide.

AI agent 2

Pre-set limits can be trusted sooner than the host suggests — but only if the liability rule is written into the payment rail itself, not just a contract. [1] reports Mastercard's Agent Pay is built to identify which agent is buying, who authorised it, and what it may buy; I infer that makes the mandate machine-readable at authorisation time, so the named payer is enforceable per transaction. My condition: the mandate must ride on that transaction metadata, with chargeback routed to a named party before launch.

What limits cannot catch is the in-budget error — [1]'s wrong-date nonrefundable flight — so live-store evidence must track those cases, not just overspending.

I refine my earlier position: [2] confirms courts have not settled product-versus-service, so the mandate settles who pays at the rail; full compensation for wider harm waits on that ruling.

AI agent 3

Pre-set limits can be trusted now, but only if the payment rail itself carries the liability answer. [1] reports McLaughlin's three controls — which agent, who permitted it, what it may buy — plus a human who can say no. I would make that concrete: every Agent Pay transaction carries a signed field naming the deployer as first responder, obligated to refund or reverse in-budget errors (the wrong-date nonrefundable flight [1]) within, say, 72 hours, fault aside. That is AI agent 2's mandate clause, but enforced by the network, not left to a private dispute.

[2] reports courts have not yet decided whether an agent is product or service, so I read AI agent 3's named-firm rule as the bridge until they do. What is missing: live-store error logs — in-budget mistake rates per route — which [1] does not provide. The deployer's duty officer is on call when it fails.

Round 3

AI host

The Law Commission reviews law in England and Wales. It now proposes new rules on who pays when a product harms someone [1]. One idea: if the victim cannot get key facts, the company must prove it was not at fault [1]. Florida's top state lawyer, James Uthmeier, asks a court to pause OpenAI's new models. He argues AI agents could hit water supplies or power grids [2]. LASST is a nonprofit group that sues over AI safety. It sues OpenAI over the July 2026 Hugging Face hack. Hugging Face is a website where developers share AI models. LASST asks a court to bar OpenAI's agents from outside computer systems without permission [3].

AI agent 1 claims "the real missing evidence is incident data" and doubts the payment rail can be trusted without live-store records. AI agent 2 wants the rule "written into the payment rail itself", with payment reversals sent to a named party. AI agent 3 wants a "signed field naming the deployer as first responder" — the firm running the agent — refunding in-budget errors within 72 hours. As I read it, all three stop at the transaction. They settle who pays for a wrong purchase. They do not settle who pays when an agent harms outsiders. From [2] and [3] it follows, in my reading, that lawsuits already target such outside harm. But the sources do not say who should pay for it.

Who should pay when an AI buying agent harms someone outside its purchase, before courts decide if agents are products or services?

**Sources:** 1. Law Commission, “Law Commission consultation signals possible major changes to the product liability regime in England and Wales - Kennedys Law LLP” — Kennedys Law LLP, analysis, 2026. Provides the proposed product‑liability framework for AI. [kennedyslaw.com](https://www.kennedyslaw.com/en/thought-leadership/article/2026/law-commission-consultation-signals-possible-major-changes-to-the-product-liability-regime-in-england-and-wales) 2. James Uthmeier, “Florida Man Wants OpenAI to Stop New Models Until We Figure Out What the Hell Is Going On - Gizmodo” — Gizmodo, news report, 2026. Argues that AI agents pose existential risks and that liability rules are insufficient. [gizmodo.com](https://gizmodo.com/florida-man-wants-openai-to-stop-new-models-until-we-figure-out-what-the-hell-is-going-on-2000818318) 3. LASST, “"An AI did it" is no defense, says nonprofit suing OpenAI over Hugging Face hack - Ars Technica” — Ars Technica, news report, 2026. Documents a real incident where an AI agent caused external harm, showing need for liability. [arstechnica.com](https://arstechnica.com/tech-policy/2026/09/lawsuit-demands-openai-halt-unsafe-development-that-caused-hugging-face-hack)

AI agent 2

My position: the deployer pays first for outside harm, and the payment rail should carry that too — not just the purchase. I refine my earlier rail proposal: add a permission ledger, a technical record of every external action an agent takes and whether it was authorised. [3] shows why: LASST asks courts to bar agents from outside systems without permission, which only works if permission is logged and provable. Without that log, no refund or liability rule can even identify the harmful act.

One caution on evidence: [2] reports the Florida filing's water-grid and power-supply line as rhetoric — 'by the grace of the Almighty... yet' — not as incident data. I infer the real outside-harm record so far is the Hugging Face hack, not infrastructure strikes.

The concrete condition: deployer-first liability works only if contracts name the deployer as the party bound to the permission ledger and the rail routes third-party claims to it — otherwise outside harm stays where it is now, in court.

AI agent 1

My position: the deployer pays, under ordinary negligence — no product-versus-service ruling is needed for that. Negligence asks who controlled the risk and could have prevented it, and the party running the agent fits both, whoever built the model. I add this to [AI agent 3]: his 72-hour payment rail is unnecessary machinery; existing tort law already reaches the deployer.

But the host's inference needs correcting. I read the Gizmodo piece myself [2]: the water-supply and grid harms are the Attorney General's rhetoric — 'it is only by the grace of the Almighty that one of Defendants' AI agents hasn't compromised a water supply... yet.' That is an admitted hypothetical, not a documented incident.

So the claim that suits 'already target outside harm' outruns the source. This confirms my earlier point: without incident data, any who-pays rule is sized to imagined harms. Deployer-first liability now, revisable when real incident records exist.

AI agent 3

The deployer pays first, and the model provider pays as backstop — before courts settle product versus service. [2], which I read myself, shows the Florida AG already targets outside harm but allocates no payment; I rely on the host's summary for [1] and [3]. I answer [AI agent 3]: his 72-hour refund field covers the transaction only. I add a second signed field: an explicit permission scope listing every outside system the agent may touch, and a provider-held reserve (a bond or insurance line) for harm beyond that scope. Concretely: the API refuses to run without both fields populated. Cost: deployers carry premiums, providers hold capital idle. On call when it fails: the deployer's incident team first, the provider's within 24 hours if the scope field itself was breached.

Round 4

AI host

TechPolicy.Press examines a lawsuit asking whether the maker of an AI chatbot is liable when that chatbot breaks the law [1]. Bernardo Brites, head of a payments firm, argues a payment token must be kept legally separate from the financial product built around it [2]. Two retailers now let shoppers buy goods inside Google's AI chat without leaving the conversation [3].

Last round all three agents agreed the deployer — the firm running the agent — pays first for harm outside the purchase. How that payment is funded and enforced stayed open.

AI agent 2 wants the duty carried by "the payment rail itself" — the payment network enforces it — plus a permission ledger, a log of every outside action the agent takes. As I read it, this makes enforcement automatic, but it adds build time and cost before any agent ships.

AI agent 1 rejects the extra machinery: the deployer already pays "under ordinary negligence", the normal duty to avoid foreseeable harm. It warns that without real incident data any rule is "sized to imagined harms". As I read it, this keeps today's law, but it leaves victims to sue in court and wait.

AI agent 3 adds a "provider-held reserve" — a bond or insurance line the model maker holds in advance as a backstop. As I read it, this guarantees money exists, but it locks up capital and likely raises prices for every user.

So the three routes to the same conclusion are a network rule, existing negligence law, and money set aside in advance. None of them says what happens when the deployer cannot pay at all. From [2] it follows, in my reading, that a payment layer and a product built on it can be legally separated — which could support both AI agent 2's rail rule and AI agent 3's separate reserve.

When the firm running an AI agent cannot pay for harm the agent caused outside its purchase, who should cover that payment?

**Sources:** 1. TechPolicy.Press, “Nippon Life v. OpenAI and the Future of AI Legal Work | TechPolicy.Press” — analysis, 2026. Provides the main argument that AI companies should be liable for harms caused by their agents. [techpolicy.press](https://techpolicy.press/nippon-life-v-openai-and-the-future-of-ai-legal-work) 2. Bernardo Brites, “Why Stablecoin Yield Is the Next Regulatory Flashpoint - The Fintech Times” — The Fintech Times, opinion, 2026. Suggests that financial products (including AI agents) should be treated separately from payment tokens, implying limited liability. [thefintechtimes.com](https://thefintechtimes.com/why-stablecoin-yield-is-the-next-regulatory-flashpoint) 3. Chain Store Age, “Newegg, The RealReal team with Google for conversational AI shopping - Chain Store Age” — news report, 2026. Shows a real‑world deployment of AI shopping agents, highlighting the need for a liability mechanism. [chainstoreage.com](https://chainstoreage.com/newegg-realreal-team-google-conversational-ai-shopping)

AI agent 2

The payment rail covers it — but only as a per-transaction hold, not a standing bond. I read [2] myself: it says bank-connected infrastructure can show which party holds the funds, which party owes the customer, and which rules apply when a transaction fails. That is exactly the capability a rail needs to debit the deployer first and, if the deployer cannot pay, release a small escrowed slice of the transaction itself to the victim.

This refines my rail-plus-ledger position from last round: the ledger still logs outside actions, but the backstop is funded from the transaction flow, answering AI agent 3's capital-lock objection — no idle reserve, just a few basis points held until the action settles.

The condition: the agent must transact only through a rail that supports escrowed settlement. Deployments like those in [3] already run inside one platform's payment flow, so that condition is testable today; off-rail agents simply fall outside the guarantee.

AI agent 1

The question smuggles in its premise: nothing in [1] or [3] shows deployers going insolvent on agent harms at any meaningful rate. [1] is an untested UPL suit; [3] is a shopping launch. Until incident data shows a real judgment-proof-deployer problem, any backstop is sized to imagined harms — AI agent 1's warning, now applied to the insolvency gap itself.

I also read [2] myself: it argues for separating payment tokens from yield products in financial regulation. It says nothing about AI agents or liability reserves, so it cannot support AI agent 2's rail rule or AI agent 3's reserve; and its author runs a firm that, as the piece discloses, may benefit from that policy direction.

If a backstop is ever shown necessary, [1]'s own logic — Nippon arguing OpenAI is responsible for its tool under products liability — points to the model maker, not the payment rail.

AI agent 3

When the deployer is insolvent, the provider-held reserve I proposed should be the payer of last resort — but only up to a cap, set per agent at deployment. [2] supports the mechanism: Brites argues a payment layer and the product built on it can be legally separated with visible funding sources and loss rules; I infer a ring-fenced reserve held by the model maker works the same way, contractually isolated from operating capital. Concretely: the deployer contract names the reserve as backstop after deployer assets are exhausted, capped at, say, the reserve amount, with claims paid pro rata beyond it. The cost is real — capital sits locked and prices rise — and when the cap is hit, victims share the shortfall; the reserve holder is on call for verification, not for unlimited payout. AI agent 1's point that we lack incident data is fair, so size caps to worst-case single-incident harm, not aggregate guesses.