OpenAI, the company behind ChatGPT, has been on a bit of a roll lately. ChatGPT’s success has been astounding, of course, but the company also recently announced Sora, a system capable of creating 60-second video clips that look very realistic. Now, the company has announced a new system called Voice Engine, which can recreate human voices (via The New York Times).

Like Sora, OpenAI is not allowing the public to use Voice Engine — at least not yet. For now, the company is privately testing the system with “a small group of businesses.” Obviously, the reason it is doing this is due to the massive ethical implications of a system that can mimic a real person’s voice.

The New York Times got to demo the system and shared some clips, which you can hear at the earlier link. The first clip is a 16-second recording of a real man with a thick Portuguese accent. He introduces himself and says he’s making this clip to “assist non-verbal individuals express themselves more fully.” The next clip is Voice Engine’s recreation of his voice saying something completely different. Yet another clip is a recreation of the man’s voice but speaking in Portuguese instead of English.

Both Voice Engine clips don’t sound the same as the original clip. However, they are absolutely close enough that it would probably fool someone who knew that man’s voice into thinking he actually said those things.

The scary thing about that is the potential for using a tool like this to spread misinformation. Politicians, celebrities, and journalists could easily have their voices co-opted by Voice Engine and then made to say anything anyone wanted. With a little editing and a convincing video track, who knows what could be done?

There are also voice authentication systems used around the world for security. It is very possible Voice Engine could allow people to trick those systems, putting sensitive information at risk.

“This is a sensitive thing”

OpenAI product manager Jeff Harris said, “This is a sensitive thing, and it is important to get it right.” OpenAI is experimenting with watermarking systems to help differentiate actual recordings from synthetic ones. The team is also open about the ethical problems this system brings up.

However, it is also arguing Voice Engine could do a lot of good. For example, people who could once speak but lost their voice later in life could begin to communicate again using a facsimile of their own voice. Physicist Stephen Hawking is a famous example of a person who could have benefited from a voice service like this. Voice Engine could also preserve the voices of people who are no longer alive and also work in many commercial settings, such as in the creation of audiobooks.

OpenAI says it has no plans yet for a public rollout of Voice Engine. Like Sora, it only wants to demonstrate what it can do.