Disabled Woman Banned From AI Voice Clone for Making It Say ‘Arse’

Motor Neuron Disease, or MND, is a condition that leads to the gradual loss of muscle control, including speech and mobility. South Floridian Joyce Esser suffers from a form of MND and has lost her voice.

An AI tool developed by a company called ElevenLabs brought her old voice back—before taking it away again by banning the voice after she incidentally trained the voice to say the word “arse.”

ElevenLabs used old recordings to digitally re-create her voice, allowing her to “speak” in her old voice by typing into a device either by hand or with her eyes and allowing the device to repeat the text aloud. The device uses a voice that sounds a whole lot like what she sounded like pre-MND. Think Stephen Hawking but with a more human voice.

These voices are called “voice clones.” Joyce’s voice clone was slapped with a ban after some remarks she made to her husband were flagged as inappropriate. Did she go on a racist tirade? Did she drop a series of swears so profane that even the AI was offended? Not quite. She typed a couple of tame sentences filled with tame curse words.

She says sentences like “Come on, Hunnie, get your arse in gear!!” and “I’d better get my knickers on too!!!” were deemed too risqué by the “prudish American computer.” ElevenLabs sent her a warning, flagging her language as inappropriate, and told her not to speak in such a filthy tongue again.

Disabled Woman’s AI Voice Clone Was Banned After She Made It Say Tame British Curse Words Like “Arse”

Joyce was told by a company representative that the team had assessed the warning and that it would be dismissed. She was then banned from using the technology the very next day. For saying arse and knickers, the company punished her by quite literally taking away her voice.

ElevenLabs has since rescinded the ban and reinstated her account, but Joyce says she has no idea why she was banned in the first place. When MIT Technology Review asked a company representative for some details on why Joyce was banned, they were referred to the company’s “prohibited use policy” page, which provides no clarity on the reason for the ban.

What’s strange is that another sufferer of MND who also uses a voice clone provided by ElevenLabs is a standup comedian named Jules Rodriguez. Jules regularly fires off curse words much harsher than anything Joyce has said during his sets and he’s never received a ban.