What Responsibility Follows When AI Learns to Spread?

Dear critical reader,

Why do we hear so few voices calling for the prosecution, or at least serious legal accountability, of companies that create technologies capable of destructive AI hacking? Anthropic’s warning about AI systems that might replicate through internet-connected devices makes the question hard to dismiss. But it also calls for precision: a warning of possible future danger is not proof that a crime has been committed. The challenge is to ask who should answer for the risk, and by what means, before fear outruns evidence or corporate reassurance outruns scrutiny.

The Guardian reports that Anthropic researchers believe AI “worms” could, perhaps within months, exploit connected devices and spread through networks. The article also makes clear that the scenario remains uncertain. The researchers explored it by prompting an agent to hack internet-connected devices; they did not demonstrate that a self-replicating worm is already loose in our kitchens. That distinction matters. To collapse a forecast into an accomplished fact would be careless. To treat a forecast from the company building the technology as mere speculation would be careless too.

There is a peculiar moral arrangement here. A company develops increasingly capable systems, tests what they might do, and then tells the public that some future version could escape familiar forms of control. The warning may be responsible disclosure. It may help governments and businesses prepare. But disclosure cannot, by itself, discharge responsibility. If the people closest to a technology identify a credible path to harm, that knowledge should trigger scrutiny of the choices that follow: what is released, to whom, with what safeguards, and on what evidence that the risks are contained.

This is not an argument that every company warning about a danger should be prosecuted. Criminal law is not a general instrument for punishing anxiety, innovation or bad outcomes. A prosecutor would need evidence of an offence, and the relevant standards would depend on the jurisdiction and facts. The question of prosecution is therefore not a verdict. It is a question about whether the law is capable of distinguishing reckless conduct from responsible research, and whether anyone is examining that boundary before a catastrophe supplies the answer in the cruellest possible way.

The public conversation too often jumps between two unsatisfactory positions. One says that because no disaster has yet occurred, there is nothing to answer for. The other says that because a disaster is imaginable, guilt is already established. Neither is serious enough. Risk is not guilt; foreseeable risk is not nothing. Between those propositions lies the work of democratic oversight: independent testing, clear reporting of serious incidents, candid disclosure of known limitations, and rules that place the burden of evidence on those who put powerful systems into the world, not on households asked to prove how a fridge was compromised.

That last point is no small matter. The proposed targets in the report include smart ovens, televisions, cameras and other ordinary devices. These are not abstract machines in a laboratory. They are part of homes and small businesses, often bought for convenience and used without any realistic ability to inspect their software or defend against a hostile agent. If a system can move through devices that have no single point of control, responsibility cannot sensibly end with the last person whose appliance was hacked. The architecture, security decisions and incentives upstream deserve attention too.

Nor will the consequences respect national borders. A vulnerability in a globally distributed product can travel farther than the regulatory capacity available to any one country. For countries such as Ghana, the question is not merely whether to import the latest connected technology. It is whether consumers, public institutions and local businesses will have meaningful protection when the systems they rely on are designed elsewhere. International standards and cooperation matter because a threat that crosses networks will not pause at a customs post.

What, then, should follow a warning of this kind? At minimum, independent scrutiny of the underlying claims; meaningful pre-release testing proportionate to a system’s capabilities; prompt reporting when serious vulnerabilities or harmful uses are found; and clear lines of responsibility among developers, deployers and device-makers. Legislators should examine whether existing rules can deal with foreseeable harms, and where they cannot, make the gaps explicit. These measures need safeguards of their own: secrecy can be necessary during testing, and rules that are too broad can entrench the largest firms or smother useful research. Oversight must be strong enough to matter and precise enough to be fair.

There is also a question of incentives. If the commercial reward for capability arrives immediately, while the social cost of failure is dispersed among millions of users, voluntary restraint will always be under pressure. A company may sincerely want to act responsibly and still face incentives that reward speed, scale and market share. Good intentions are not a governance system. Public rules, independent evidence and meaningful liability are how society tries to align private decisions with risks borne by everyone else.

The most revealing part of a warning may be what happens after it is issued. Does it lead to slower deployment where uncertainty is serious? To independent scrutiny rather than a company marking its own homework? To governments asking what evidence would justify release? Or does the warning become another dramatic headline, followed by business as usual until the next alarming capability appears?

Calling for accountability is not hostility to science. It is recognition that technical power carries public consequences. We should neither demand prison whenever a company imagines a danger nor accept a warning as a substitute for action. We should insist that claims be tested, decisions be explained and responsibility be traceable. If the risk is real, prevention cannot be left to consumers. If the warning proves overstated, transparent scrutiny can show that too. Either way, the public deserves more than alarm followed by silence.