Recent Items

UK Home Secretary’s Shocking Admission About the Emerging AI Surveillance State

Posted on by

“When I was in justice, my ultimate vision… was to achieve, by means of AI and technology, what Jeremy Bentham tried to do with his Panopticon.”

UK Home Secretary, Shabana Mahmood, the person nominally in charge of the UK Police, justice system and MI5, sat down for an in conversation event with the former Prime Minister (and current Labour government’s eminence grise) Tony Blair, organised by the Tony Blair Institute for Global Change. In that conversation, made a chilling admission about the government’s ultimate goal of AI surveillance.

From the Daily Telegraph, the only mainstream newspaper besides Scotland’s The National to deign to cover the story:

“AI and technology can be transformative to the whole of the law and order space.

“When I was in justice, my ultimate vision for that part of the criminal justice system was to achieve, by means of AI and technology, what Jeremy Bentham tried to do with his Panopticon. That is that the eyes of the state can be on you at all times.

“Similarly, in the world of policing, in particular, we’ve already been rolling out live facial recognition technology, but I think there’s big space here for being able to harness the power of AI and tech to get ahead of the criminals, frankly, which is what we’re trying to do.”

Bentham, an 18th-century philosopher and social theorist, promoted the Panopticon as a circular prison with a central inspection tower from which a single guard could observe all inmates all the time while unseen.[1]

In the case of the original conception of the panopticon, prisoners caught doing something wrong in their cells would be duly punished. Over time, they would learn that total compliance was their only option.

In much the same vein, the digital public infrastructure being rolled out in countries across the world, including the UK — in particular digital identity and central bank digital currencies (CBDCs) — could be used to discipline members of the public who are caught diverging from the prescribed norms of behaviour and thought.

The ultimate goal, as we’ve been arguing for a number of years now, is to close off people’s access to basic goods or services, including Internet websites (as we’re already been seeing with the age verification legislation passed in Australia and the UK), telecommunications services, public benefits and services, and even one’s own bank account. [2]

As we’ve seen with the recent debanking scandals, governments have already shown a perfect willingness to do that. The introduction of CBDCs, coupled with the removal of cash, would simply automate this process, taking away the need to go through the courts or political bodies. It would also mean that people would have even less recourse to appeal.

Mahmood’s admission is shocking, not so much in terms of its actual message but rather the candidness with which it was conveyed. Seasoned NC readers will not be surprised that the British government, like many other governments, is trying to build an AI-enabled panopticon. Over the years we’ve covered in some depth the totalitarian thinking behind Bentham’s Panopticon, as well as the threats posed by the emergence of a digital panopticon. [3]

That said, this is almost certainly the first time that a senior government official of a major Western nation has come out and openly admitted that they are effectively building a digital panopticon in order to maintain total, constant surveillance of criminals. Such a system could be quickly expanded to the broader population — indeed, in the case of the UK it already is, through the nationwide rollout of live facial recognition systems, reports The Telegraph:

As justice secretary, Ms Mahmood proposed a major expansion of GPS tagging of criminals to create “virtual prisons” for offenders punished in the community. Since moving to the Home Office, she has announced a planned nationwide rollout of police-operated live facial recognition cameras.

Most senior government officials tend not to boast, in public at least, about building a digital panopticon, even if that’s exactly what they’re doing, for an obvious reason: it’s totally dystopian — the Stasi on steroids. But as we’ve been warning, the Starmer government has turbocharged the UK’s slide into dystopia since coming to office, including by scaling back trial by jury, escalating its attacks on lawful speech and rolling out digital identity despite public opposition.

Like most governments in the West, the Starmer administration is also pushing for online age verification, which is essentially a Trojan Horse for digital identity. As the member of the House of Lords Claire Fox warns in the clip below, “this is a threat, potentially at least, to adult civil liberties and the right to privacy, and effectively means we’ll have to digitally verify to participate in the public square.”

Yet there is precious little reporting in the legacy media on these sweeping changes (quelle surprise!). As Jonathan Cook notes in the tweet below, Mahmood is effectively telling the British public that they are prisoners and that the government quite literally plans to become Big Brother using AI. Yet there is nary a word about it in the media (with the notable exceptions of The Telegraph and The National). This, of course, is also a feature, not a bug.

The panopticon, in its original conception, is so controversial that it has barely been properly tried (though elements of it have been incorporated into the design of some prisons). The closest thing, the Presidio Modelo complex in Cuba, built in 1920, was so plagued by corruption and cruelty that it was abandoned. As Collingwood explains in the above tweet, the psychological effects of such living were considered too cruel for even prisoners to endure:

Many indeed consider the entire concept of the Panopticon the foundation of the theory of totalitarian regimes in operation: if it could seek to arrange society so [that] every citizen may be watched at any time but cannot know whether they are being watched or not (e.g. the telescreens in Nineteen Eighty-Four’s Oceania) a regime could force all citizens to act as though they were being watched at any given time.

And this is what our Home Secretary—the office in charge of the police and MI5 and the justice system—wants to impose on us. This is her dream society. Not even joking or embellishing.

It is as if people like Mahmood read Orwell’s 1984, Huxley’s Brave New World, Phillip K Dick’s Minority Report and a host of other dystopian novels and came away with an instruction manual. Meanwhile, their big tech paymasters came away with new business models.

Here’s Oracle’s Larry Ellison, the Tony Blair Institute’s largest donor, bragging about how an AI surveillance state will ensure that people are always on their best behaviour:

What often gets ignored in what little public discussion that is allowed on this topic is the fact that AI surveillance systems are as prone to making errors as most AI systems. The biases in facial recognition systems are already well documented. Just last month, the UK Home Office admitted that facial recognition technology is more likely to incorrectly identify black and Asian people than their white counterparts on some settings.

Yet the systems are still being rolled out nationwide, inviting the question: feature or bug?

There was also the recent controversy over West Midlands Police’s handling of its AI search for data about alleged Israeli football fan violence, which generated false claims of a non-existent match. The ensuing scandal undermined public confidence in officers’ use of the technology, reports The Telegraph.

Perhaps more disturbing of all was Mahmood’s response to the scandal: she told MPs that AI was an “incredibly powerful tool that can and should be used by our police forces” but said it needed to be regulated in a way that was “always accurate”. In other words, in Mahmood’s mind, for AI systems to function correctly, all that is needed is the right sort of regulation. It is a masterclass in why bureaucrats should be kept as far away as possible from these technologies.

Meanwhile, the energy and water demands of the data centres needed to run these AI systems are growing exponentially, as BlackRock Chairman and CEO Larry Fink laid out at Davos two years ago. Fink, who is now co-interim Chair of the World Economic Forum, said that data centres would need “massive power”, which he described as a huge “investment opportunity”.

If anything, Fink appears to have been underestimating the scale of the problem by orders of magnitude. A January 2025 paper by RAND Corporation forecasts that “global AI data center power demand could reach 68 GW by 2027 alone and 327 GW by 2030, compared with total global data center capacity of just 88 GW in 2022.” Other estimates have placed it somewhat lower but still well into triple figures.

And it won’t just be big tech companies and financial giants that will be footing the bill; it will be all of us, as Yves noted in her pre-amble to a recent cross-posted article by Jomo Kwame Sundaram on the growing global revolt against data centres:

I find it remarkable that data center, as in AI, new energy demands are being foisted onto all customers. Those demanding incremental and higher cost energy should pay more. But instead the population at large is subsidizing AI, both through said energy costs plus also bearing the cost of additional pollution.

And this is the really insulting part: not only do governments and the corporations whom they serve and represent plan to use AI to digitally imprison us; they want us to pick up the tab along the way.

 

[1] According to a 2015 article in The Guardian, it wasn’t actually Bentham himself who came up with the idea:

“The panopticon wasn’t originally Bentham’s idea. It was his brother’s,” says Philip Schofield, professor of the History of Legal and Political Thought and Director of the Bentham Project at UCL.

“His brother Samuel was working in Russia on the estate in Krichev and he had a relatively unskilled workforce, so he sat himself in the middle of this factory and arranged his workforce in a circle around his central desk so he could keep an eye on what everyone was doing.”

Bentham went to visit his brother in the late 1780s, saw what he was doing, and decided the centralised arrangement could be applied to all sorts of different situations – not just prisons but factories, schools and hospitals.

Bentham managed to persuade the prime minister, William Pitt the Younger, to fund a panopticon National Penitentiary, but a stream of problems eventually meant the project was abandoned. Bentham never saw a panopticon built during his lifetime.

[2] As I document in my 2022 book Scanned, the World Economic Forum, one of the most important purveyors of digital public infrastructure, admitted in a 2018 paper on digital identity that while verifiable digital identities “create new markets and business lines for companies”, they (emphasis my own) “open up (or close off) the digital world for individuals.” Which is precisely what we are beginning to see play out.

[3] In 2011, former NC stalwart Phillip Pinkington wrote in an article on Jeremy Bentham’s theory of utilitarianism:

The original theorist of utility was, of course, Jeremy Bentham who, as we have noted, also came up with the Panopticon. The Panopticon, as noted, was a totalitarian prison system wherein every prisoner was to be watched constantly by a central observer who monitors their behaviour. Bentham thought that this model could be extended to a variety of social institutions, giving rise to a terrifying vision of a totalitarian hell which was later to be captured in 20th century novels such as Orwell’s ‘1984’ and Huxley’s ‘Brave New World’.

Bentham’s vision was downright paranoid, of course. But it says a lot about the psychology behind his theorising. This man was not a prophet of human freedom and actualisation. No, he was the harbinger of a dark vision of totalitarian control. And his theory of utility was but another manifestation of his own slightly villainous technocratic tendencies.

In a January 2025 article on the Swiss government’s decision to table a second referendum on digital identity adoption, four years after two-thirds of Swiss voters rejected the government’s proposed e-ID system, we wrote:

One of the main reasons why there is (seemingly) no alternative to digital identity is that there are simply too many powerful interests aligned behind it. It is the keystone of the new panopticon of digital public infrastructure (DPI) being constructed around us.

For governments and national security agencies, the benefits are clear: expanded power and control at a time when economic conditions are about to get significantly worse for the vast majority of the population. For big tech companies, it will mean new opportunities to amass even more data over our lives, which they will then be able to transform into even more revenues and profits. For central banks and the TBTF banks whose interests they predominantly serve, it will mean even more centralised financial power in their hands.

In May of the same year, we cross-posted a piece by John P. Ruehl, a world affairs correspondent for the Independent Media Institute, primarily on China’s evolving social credit system but which also explored the “sprawling and mostly unregulated personal scoring” systems that have been quietly built by US companies. The article finished with this stark warning:

Though promoted as tools to encourage good behavior and deter bad conduct, these [social credit] systems amplify social pressure and push societies toward a digital panopticon — a state of constant surveillance driven by government and commercial incentives. These models will continue to mature and become more dangerous in the U.S. and other countries that lack adequate data protection. Without strict limits on surveillance by both governments and corporations, fears of AI misuse, algorithmic bias, false correlations, and harmful feedback loops will only grow as these scoring systems govern more of everyday life.

 

Print Friendly, PDF & Email

Subscribe to Post Comments
24 comments

    1. Stephen

      Maybe. And I hope so too.

      But if the elite “consensus” embraces these approaches then judges will surely find ways to reinterpret “human rights” law.

      When the profit motive, ideology and the urge to control align then the forces to drive this will be hard to stop. The only silver lining is that the modern UK state is so incompetent that it likely will struggle to do this.

      However, the 1984 state was similarly incompetent, except at exercising control. So perhaps incompetence and control do go together.

    3. Carolinian

      You beat me to it. Of course governments used to use religion to control the lowers with God watching over all of us and meting out ultimate punishment for misbehavior. In this system even the powerful had to worry about their behavior and buy indulgences, make confessions to priests, death bed convert to monks if you were a warlord who spent his life slaughtering people.

      Tech is our new religion if you are a tech bro or a weak minded leader like Starmer or Trump. Real leaders have talent and lead by example.

  4. thoughtfulperson

    Orwell and others works mentioned certainly rhyme with what we are seeing develop. Globally. I guess it is the shared goals of total control and domination.

    The proof is in the suppression of political views (i.e. opposition to mass murder! Why is this controversial lol). This is very far from “protecting society” from what we used to think of as “criminals”. Now our countries police and military forces are being used to suppress the excersise of political rights. The definition of what is criminal and who is a criminal is paramount and we all know how that is decided (not by a democratic vote! Or even debate! ).

    For those not yet concerned, imagine your actions surveilled by someone like Trump 2.0 (anyone who disagrees with me is a “far left terrorist”). Still not worried? How about when your life is surveilled by Trump 3.0?

  5. Frank Dean

    Nitpicking item: in footnote 2, Bentham is responsible for Utilitarianism, not Marginal Utility, which is a different concept developed by Jevons and Marshall, among others.

  6. Stephen

    The descent of the west into authoritarianism seems to be general. More evidence too that Trump is less an aberration than a particularly direct (or even honest?) manifestation of what is happening more generally.

    Hard to see that the Democrats will put things back in the bottle if they win the mid terms and in 2028. They will likely continue the lurch to authoritarianism in a more subtle way than having armed agents wandering around cities. After all, Starmer, Blair et al are hardly Trumpers.

    1. Ignacio

      Thank you Nick. Now to Stephen.

      Authoritarianism might have some justification if and when the authoritarian leadership would merit some cognitive trust which is obviously not the case with most if not all the professional and managerial castes of today, starting with this Mahmood and her candid an at the same time chilling declarations. When you can say things like those quoted here by Nick without having to face any consequence, backlash, correction, whatever, it is when the thing becomes really, really chilling. There is a near consensus amongst the leaderships.

      It would be nice if some whistleblower, let’s say from the BBC, would come up and say that she or he proposed to report on this but was cancelled by the directorate because…, because …, explain the reasons! Of course, someone not unwilling to risk the salary, being de-banked etc… Difficult.

      1. Stephen

        Right. When there is true control, of course, journalists know not even to propose to report on things that will not get approved. They have careers, after all. My recollection of 1930s German history is that very little overt censorship was needed, precisely for these reasons.

        Trump is overtly aggressive because his brand of authoritarianism lacks elite consensus. What follows will be far more pernicious, precisely because it will be supported by the elites.

  7. Grumpy Engineer

    Larry Fink’s comments about data center power needs and intermittent renewable energy lacks context. To quote:

    “By 2030 [data centers] need 30 gigawatts.. Where’s that power gonna come from? To power these data companies you can’t have intermittent power like wind & solar“

    30 GW sounds like a lot, but average electricity consumption worldwide adds up to ~3500 GW. If the world can successfully provide 3500 GW of power with renewable energy, then the extra 1% imposed by data centers shouldn’t be any problem. And conversely, if the intermittency of wind and solar makes it too difficult to power a mere 30 GW of data centers with renewable energy, then there is no hope of powering the rest of the world that way.

    Now please understand… I find these proposals for AI-enhanced mass surveillance to be appalling, but the concerns about power requirements aren’t key. The utter loss of privacy and the inability to go on about your business (without somebody passing judgment as to whether or not it is politically approved) are much larger concerns.

    1. Nick Corbishley Post author

      Thanks for your comment, Grumpy. I see your point, and I broadly agree with it: the worst aspects of this AI-enhanced mass surveillance are the threats it poses to privacy and the near-total loss of autonomy and personal agency, and with that in mind I have slightly reworded my somewhat glib ending. That said, the fact that we’re being asked to pony up some of the funding for this “digital panopticon” is still, I believe, pertinent.

      FWIW, it seems that Fink may have been underestimating the scale of the problem, by orders of magnitude. A January 2025 paper by RAND Corporation warns that global AI data center power demand could reach 68 GW by 2027 alone and 327 GW by 2030, compared with total global data center capacity of just 88 GW in 2022. Other estimates place it somewhat lower than that but still well into three figures.

      1. Grumpy Engineer

        “The fact that we’re being asked to pony up some of the funding for this “digital panopticon” is still, I believe, pertinent.“

        Oh, yes. With this I have no argument whatsoever. If new power stations are required to meet the electricity demands of new data centers, then the owners of those new data centers should pay for those power stations. Not other electricity customers who won’t even be using that power.

        And if this makes the costs for AI prohibitive, so be it. [And with DDR5 and SSD prices quadrupling, DDR4 prices tripling, and even old DDR3 and HD prices doubling, they’re already facing significant cost headwinds. I suspect a lot of the initial proposals for AI data centers didn’t account for computer HW getting 3X as expensive overall.]

        And if the new power requirements are closer to 327 GW rather than 30 GW, then financing costs become problematic, as it’s going to take many years to get all of that power deployed. Particularly if it has to be at least somewhat “green”.

        [That’s about the only potential silver lining I see in all of this. If somebody out there figures out how to do cost-effective “modular” nuclear for purposes of powering data centers, then we can use that technology to replace fossil-fueled power stations even if the AI bubble pops. And that may be possible. The last nuclear station in the US cost over $15000/kW, whereas the Chinese, Russians, and South Koreans have all built stations for less than $2500/kW. And in half the time.]

    2. Ignacio

      Even if AI would be a mere 1% of current power energy demand, adding more demand when we need to fight tooth and nail to make all and every GW of power and every GWh of energy demand “sustainable” is not helping at all. On the contrary. Well, unless we believe those snake-oil salespeople insisting that AI and digitalization will result in net energy savings. I, for instance, do not believe in any of that.

  8. The Rev Kev

    The idea of an Panopticon contained one huge flaw. Yes, a guard would be able to inspect hundreds of people in their cells but by the same token, those hundreds of prisoners were also watching any guards. But with digital technology, people can be spied on 24-7 and they will never know who is doing all this surveillance-

    https://en.wikipedia.org/wiki/Panopticon

  9. lyman alpha blob

    Implementing a technology known to be extremely error prone in order to surveil and punish the populace – the ethos here from the would-be ruling class is quite clearly “Kill them all and let God sort them out”, but with Skynet taking the blame rather than the Starmers of the world.

  10. Nicholas

    And lest we forget, Bentham lent his eloquence to the defense of pederasty in his essay titled “Paederasty”. Ironically the digital panopticon will no doubt be justified with the appeal to protecting children from….

  11. icancho

    Nick C. mentions “the Presidio Modelo complex in Cuba, built in 1920,” but it might be of interest to note that, back in ~1840, the Huron County Gaol, in Goderich, Ontario, Canada, was constructed on explicitly Benthamian ‘panopticon’ principles. It still stands. Incidentally, it was the site of Canada’s last public hanging, in 1869.

    1. Nick Corbishley Post author

      Thanks for that little nugget, icancho. Interesting to see that the 19th century Ontarians made a better go of it than the early 20th century Cubans.

Comments are closed.