There is a company, Conduit, which is trying to develop telepathy. This is a terrible idea. The best that can be said for it is that it probably won’t actually work. If it actually does work, it would be the greatest aid of totalitarianism ever invented.

The technology is simple enough, conceptually. You need some way of detecting impulses inside the brain, and you need to map these onto outputs. At the moment, they (Conduit) are assembling a dataset by having people go to an office in San Francisco and have a conversation with AI. They’re a bit vague in how they are collecting the data, but it’s non-invasive and probably isn’t an MRI. I am skeptical it is going to work perfectly. There is a lot of noise in what people are thinking about, and a lot of noise in what we observe in people’s brains. However, just as typos in training text have not prevented AI from existing, I expect that we will have improving technology so long as there is money spent on it.

Has nobody considered what the world looks like where they succeed? Suppose that they are right, and we are able to detect and interpret people’s inner monologues, conjured images, and even as yet unexpressed thoughts. They think it is going to be a world of people using it to control their AI agents with extraordinary efficiency. I need them to stop thinking about what people could do with it voluntarily. Start considering instead what people will want to do with it involuntarily!

There are two categories of problems. First, it will enable authoritarianism. Second, even if we avoid this, it could remove our ability to think independently. The first case seems, to me, to be much stronger than the second, although both are real.

This technology need not be used to monitor the thoughts of the public at large. You simply do not need to do this. Instead, all you need to monitor are the security forces. Authoritarian states are already perfectly capable of suppressing the masses. What they are unable to do, and what is the real reason that authoritarian states fall, is to perfectly monitor those in positions of power who could replace the current leaders. Popular protests lead to the fall of regimes only when the security forces are unwilling to shoot. An armed security force that is willing to shoot an unarmed population without limit will always win.

Authoritarian states are normally unable to monitor everyone personally. They invest a lot in coup proofing, which is having everyone be monitored by everyone. No single entity holds the power to replace the leader. Now, you don’t need that. You can select people who are perfectly loyal, with a device that serves anyone.

Consider this. Do you want President Trump to have a technology that can detect, with real accuracy, whether you would support him in a coup? Do you want the President to be able to scan everyone for loyalty to him personally, above the Constitution? Do you want every government in the world to have this? It is already standard to screen people in positions dealing with classified intelligence for their honesty and loyalty, through polygraphs. We have been saved due to them not working. Why should we make them work?

Beyond the use in authoritarian regimes, which will definitely happen, I am also concerned that it will narrow the ranges of thoughts which are acceptable to hold. Suppose that their world comes to pass, and everybody interacts with their technology as a matter of course. In the ideal scenario, by 2027, AI agents will be able to interpret our thoughts, and react to them. Her specific vision of her daily life involves an AI agent interpreting her scarcely voiced thoughts about an interview at lunch, and acting upon them. Any agent capable of doing that is perfectly capable of acting upon scarcely voiced dissatisfaction with the present administration, with a desire to run away, or even just opinions that differ from the community at large.

I fear society will tear itself apart over intrusive thoughts. Did you know that, in Britain, 12,000 people are arrested and questioned every year over online communications that could cause “annoyance”, “inconvenience”, or “anxiety” to others? Britain is, despite the online jokes, a rich, liberal democracy. Suppose that we have a window into everyone’s thoughts. Are you so confident we will restrain ourselves from arresting everyone who has had a flash of attraction to children or to their siblings or to their mother? Do you want to be known as the sort of person who doesn’t want to protect the children?

Perhaps you endorse this, because this is our taboo, and our taboos are right. But do you want an Ayatollah to detect traces of apostasy? Do you want a counterinsurgency campaign to detect who supports their cause, and who does not? Do you want the Nazis to know exactly where the Jews are hiding?

The downsides must be fairly considered against the upsides. Many technologies are not unconditional goods. But what are the upsides? If all goes well, it allows us to interface quicker with a laptop. Where is the improvement? I type. I type quite fast. Lots of people type fast. If the problem is that people cannot move their fingers fast enough, then you can record a voice statement – transcription software is plenty good enough for that.

If anything, as AI agents get smarter, human interaction is going to matter less and less. We direct them to do a task, and they go work for a long while. The present workflow skews toward things happening which you cannot keep up with thinking in live time. This sort of technology is read only; it cannot write to the brain.

The best use case is a physical extension of the body. Plausibly, we could control a large machine with our mind. This is, incidentally, the plot of Pacific Rim. I think such scenarios are a bit far out.

I think the people involved with Conduit should stop what they are doing. Go back to work at OpenAI! I’d far rather you take your chances with AI that might kill us all, than with an AI that will very definitely enslave us.

This post is brought to you by Mechanize, Inc. They are hiring for a variety of positions, including software engineers. I encourage you to apply here.