AI may be a buzzword on Wall Street, but on the West Coast itâs at the center of Hollywoodâs biggest labor dispute in more than 50 years. Among those warning about the technologyâs potential to cause harm is British actor and author Stephen Fry, who told an audience at the CogX Festival in London on Thursday about his personal experience of having his identity digitally cloned without his permission.
âIâm a proud member of [actorsâ union SAG-AFTRA], as you know weâve been on strike for three months now. And one of the burning issues is AI,â he said.
Actorsâ union SAG-AFTRA, which has around 160,000 members, went on strike last month over pay, working conditions, and concerns related to the use of AI in the film industry. It joined the Writers Guild of Americaâa union representing thousands of Hollywood writersâwhich went on strike in early May, marking the industryâs biggest shutdown in more than six decades.
A key sticking point for actors on strike is the possibility that studios could use AI to make digitally replicate their image without compensating them fairly for using their likeness.
Speaking at a news conference as the strike was announced, union president Fran Drescher said AI âposes an existential threatâ to creative industries, and said actors needed protection from having âtheir identity and talent exploited without consent and pay.â
During his speech at CogX Festival on Thursday, Fry played a clip to the audience of an AI system mimicking his voice to narrate a historical documentary.
âI said not one word of thatâit was a machine. Yes, it shocked me,â he said. âThey used my reading of the seven volumes of the Harry Potter books, and from that dataset an AI of my voice was created and it made that new narration.â
Fryâwho has appeared in movies including Gosford Park, V for Vendetta, and The Hitchhikerâs Guide to the Galaxyâis the narrator of the British Harry Potter audiobooks, while actor Jim Dale narrated the American version of the series.
âWhat you heard was not the result of a mash up, this is from a flexible artificial voice, where the words are modulated to fit the meaning of each sentence,â Fry told the audience at CogX Festival on Thursday.
âIt could therefore have me read anything from a call to storm parliament to hard porn, all without my knowledge and without my permission. And this, what you just heard, was done without my knowledge. So I heard about this, I sent it to my agents on both sides of the Atlantic, and they went ballisticâthey had no idea such a thing was possible.â
Fry added that when he discovered his voice was being used in projects without his consent, he saw it as just the beginning of an emerging threat to creative talent, warning his angry agents: âYou ainât seen nothing yet.â âThis is audio,â he said he told them. âIt wonât be long until full deepfake videos are just as convincing.â
As AI technology has advanced, doctored footage of celebrities and world leadersâknown as deepfakesâhas been circulating with increasing frequency, prompting warnings from experts about artificial intelligence risks. Fry warned on Thursday that those technologies only had further to go.
âWe have to think about [AI] like the first automobile: impressive but not the finished article,â he said, noting that when cars were invented no one could have envisioned how widespread they are today.
âTech is not a noun, it is a verb, it is always moving,â he said. âWhat we have now is not what will be. When it comes to AI models, what we have now will advance at a faster rate than any technology we have ever seen. One thing we can all agree on: itâs a f***ing weird time to be alive.â
Not the first
Fry isnât the only famous actor to publicly vocalize their concerns about AI and its place in the film industry.
At a U.K. rally held in support of the SAG-AFTRA strike over the summer, Emmy-winning Succession star Brian Cox shared an anecdote about a friend in the industry who had been told âin no uncertain termsâ that a studio would keep his image and do what they liked with it.
âThat is a completely unacceptable position,â Cox said. âAnd that is the position that we should be really fighting against, because that is the worst aspect. The wages are one thing, but the worst aspect is the whole idea of AI and what AI can do to us.â
Oscar winner Matthew McConaughey told Salesforce CEO Marc Benioff during a panel event at this yearâs Dreamforce conference that he had concerns about the rise of AI in Hollywood.
âWe have a real chance, if we are irresponsible, of cannibalizing ourselves and creating this digital god that weâll bow to, and weâll all of a sudden become tools of this tool,â he said.
Meanwhile, Star Trek and Mission Impossible star Simon Pegg has called AI âworryingâ for actors.
âWeâre looking at being replaced in some ways,â he said at the rally in London in July. âWe have to be compensated and we have to have some say in how [our image is] used. I donât want to turn up in an advert for something I disagree with⊠I want to be able to hang on to my image, and voice, and know where itâs going.â
A spokesperson for the Alliance of Motion Picture and Television Producers (AMPTP), the entertainment industryâs official collective bargaining representative, was not available for comment when contacted by Fortune.
The much, much, much more concerning aspect of voice cloning technology is that it will be used to scam people on a massive scale.
Imagine you get a call at 4am from a loved one who tells you that they are in an emergency situation and had to borrow a phone to call you. The beg you to venmo some money to a strangerâs account so that they can get their car fixed/get a plane ticket/pay someone back for giving them a lift/etc.
You recognize your loved oneâs voice. They can respond to your questions (because chatbot AI). They know details about your life (because social media). Itâs the middle of the night. Youâre scared and not thinking clearly.
This technology all exists TODAY. In 10 or 20 years itâll be so terrifyingly sophisticated, even the most wary people will be vulnerable to it.
Easy solution, donât have any loved ones. Checkmate scam artists
âHey itâs ur son Iâve been arrested in Mexicoâ
âWell good thenâ.
My uncle was recently arrested in another state. We had a similar reaction.
Someone tried to scam my grandpa with that. He told âmeâ to enjoy rotting in jail then called me up to ask how jail was.
âItâs about time they caught you! Oh wait, they donât want the reward money, do they? Ah fuck it, they canât unarrest you, tell them to get fucked!â
Or if you do, make sure none of 'em are dumb enough to rely on âcash appsâ like venmo. Even Zelle, through our bank is suspicious as shit.
Thatâs why I do like Gilbert Gottfried and do two voices: one in public and one for friends and family.
It gets confusing when we dine outside.
đ Wtf does that guy sound like at home? Posh mid Atlantic accent or some shit? Iâm so curious now.
Nothing, heâs long dead.
He passed away last year. I personally wouldnât call that long dead.
Thereâs a Howard Stern clip of Gilberts ânormalâ voice on his voicemail.
They canât get me if I live in a hole. Not a nasty, dirty, wet hole, filled with the ends of worms and an oozy smell, nor yet a dry, bare, sandy hole with nothing in it to sit down on or to eat: but a hobbit-hole, and that means comfort.
The solution that EVERYBODY needs to learn for something like that is to hang up and call them back using the contact you have in your phone. They can afford 10 seconds while you do that if theyâre calling you for money. And if it isnât them calling for money, well sorry for waking you up Frank, but an AI was posing as you asking for cash.
That and a family or per person verification word or protocol or something.
âClumsyâŠâ
âDraconiquist!â
Thank you, your suggestion has been added to the training data.
/s
âOh my god, agent_flouder, I was just in a car accident and they need the bank info to process the co-insurance so I can get the organ transplant that is expiring in minutes! Iâve lost a lot of blood, have a concussion and have forgotten our code word. PLEASE donât do this right now or this might be the last time you hear from meâŠâ
In the capitalist hellscape that is the US, that isnât that far fetched and with emotions high, I doubt itâs unlikely. On the other hand, I can see a news article that reads, âMan lets daughter die by refusing hospital critical information needed for transplant.â
Lawl they donât do insurance shit for emergencies like needing an organ or blood immediately, they deal with that shit after the operation
Yeah but this is America. And do you think the average person knows that, will remember it when their loved one calls them crying, and will have the temerity to actually refuse when thereâs a time constraint?
Good point.
Unless they are calling from the hospital, police station, borrowed a cell phone after a car accident, etc.
Then you call THAT number. Stationâs non emergency number, hospital, etc.
yet another reason to never answer the phone
Ah, thatâll be the equivalent scam for our age that spam emails are for the age before.
Scam them out of what?
90% of the world will be unemployed and fighting for whatever scraps of food are grown between the constant flooding and fires we had 100 years to prevent but it wasnât profitable to.
We will have to learn to live entirely without factual information as every form of communication becomes hopelessly compromised by corporations, governments and billionaire extremists.
You wonât even be able to trust the people you meet in meatspace. You think Fox News addicts are fucked up? Wait until every piece of entertainment is propaganda thatâs been personalised just for you.
And when it inevitably turns to war and youâre put in charge of the big red button, will you even care if the order to press it has been deep faked by a death cult?
Unemployment and petty scams are small fry. With this technology, we can end the world.
Easy peasy. Tell them okay. Then hang up, proceed to call that loved one using your record of his /her number. Confirm.
In the referenced scenario they had to borrow a phone to call you.
Presumably their phone is out of battery, broken, stolen, or theyâre in another country without service.
So that means the ârealâ person definitely wonât answer their phone right? That all is useful for trying to confirm someone is who they say they are.
Yeah it has the flaw that at that hour the real person might not answer tough⊠(If they shutdown the phone or mute it or whatever). But yeah that is the common approach.
Itâs not perfect, but itâs something.
Deviant had this fascinating/awful video about this kind of situation: https://www.youtube.com/watch?v=6ihrGNGesfI
Here is a (really) top infosec expert saying that when someone you know is in jail, you absolutely have to turn off all your call filtering and spam filtering, because who knows what shitty system the facility theyâve been moved to today is using to route calls.
Well since itâs a scam, it means that the phone is really not broken or out of commission.
Not readily believing anything you hear from something that is unusual or out of the ordinary will save you in the future.
Call just to check. If the phone is unreachable, call someone close to that person, a wife, a son /daughter. Just think. If they have a wife or kids, why call you in the first place.
Sometimes itâs good to be not overly trusting.
deleted by creator
Or even better, conference them in to that call. Then get them to debate each other about which one is real.
Unfortunately, that is already happening⊠https://www.independent.co.uk/tech/ai-voice-clone-scam-kidnapping-b2319083.html
Iâve had friends fall for email scams. Weâre from Canada and I remember one being that another friend of our was stuck in Wales and needed a bank transfer. I knew it was a scam but a few of my friends were worried. I said that I live in the UK now and can take a train to Wales if you REALLY want. They were still panicking and saying they should do it in case. Iâm like, you canât be serious!
So yeah, it can already be bad and with ChatGPT they can pass the Turing test. All of our friends will probably test each other on our memories. âTell me the name of your ex-gf and which year and how you broke again?â
deleted by creator
This type of scam exists and, unfortunately, works without voice cloning and social media in East Europe for years.
Most people donât know what their loved ones sound like on the phone. This is already a scam and you should never believe someone calling you like that. You can ask them something only you two know or just tell them to call the police and that youâll meet them at the police department or hospital or whatever. Never give out credit card info ect over the phone. Nobody would ever do that in a legit situation.
Easy enough to teach people to just hang up and call their friend.
deleted by creator
Physical 2 factor authentication. Have a code word for your kids to tell you or for you to tell them.
Iâm not going to lie, the ability to have Stephen Fry narrate my daily schedule every morning is one of the things I am most excited for. Like, I understand why heâs upset about a movie using his voice without his permission, but I didnât expect to get his permission for my thing, either.
I mean if he wants to make a fortune and also make people giddily happy⊠Iâd totally pay for that. Or like Andy Serkis (whose narration of The Hobbit is a fucking masterpiece, by the way).
Probably couldnât afford it though.
Andy Serkis is such a good audiobook narrator. I loved his version of Terry Pratchettâs Small Gods!
Ooh! Adding that to my want list.
Yeah, this is where the union demands are heading: if you use the AI replica of an actorâs voice for a project, youâll need to pay for a license.
I already have Siri be an English man so it kind of sounds like a butler. I would pay for it to sound like Stephen Fry. Or Michael Gough.
The Fry version of the audio books are fantastic. I too would like Stephen Fry narrate my day.
Yes, but theyâll will make money with that movie. For you itâs just your own entertainment
Sure, thatâs why I donât feel that bad about it, but I bet he would still prefer to consent to it.
Fry could just license a company to use AI for a narrow purpose in this instance. He could or have his people quality control check it and get royalties on sales. He could even record new bits of voice for stuff not already covered adequately. This would be a way for AI to benefit both parties. Instead AI companies are stealing prior work and copying it wholesale.
deleted by creator
Like the few actual celebrity TomTom nav voices.
I havenât used a TomTom in like a decade. Did they do that for the original ones? I just assumed they had them record the things.
My friend had the Mr. T one. If it wasnât Mr. T, it was a dead-on impersonation. I mean I know a lot of people can sound like Mr. T, but this was exact.
Usually, itâs an impersonator, last I saw.
There really needs to be a âright to identityâ that companies canât pretend to be you without express permission on a per instance basis and roll it into fraud protection/identity theft laws.
Arguably, enumerating such rights in the first place mainly benefits the wealthy and corporations because once encapsulated, such ârightsâ can be bought and sold.
An established actor like Fry has a lot of leverage, and the union may win the recognition of those rights, but then thatâs just putting them on the negotiating table. What studio, trying to launch a film franchise, wouldnât get the exclusive Digital Identity Use Rights, in-perpetuity, solely for use in that character? Sure, RDJ is free to go make other movies and control how his image is otherwise used, but Marvel-Fox-Disney gets to keep making Iron Man content (starting the AI-replicated likeness) for all time. And if they want Iron Man to sell Big Macs, too fucking bad, shouldnât have sold your rights so cheap. Leverage he didnât have when Marvel was rebooting his career.
âin perpetuityâ rights to use somebody elseâs identity seems like it should be illegal tbh
Actors need to be able to trademark their image and audio likeness or corporations will puppeteer them for free. Forever.
I honestly think this is the way forward. Trademark the likeness, and studios can use the likeness while you get an upfront fee and royalties.
Everyone wins.
Expansion of copyright that way could set a very dangerous precedent and have unintended consequences.
For example, does that mean Trump, who was once part of SAG, can claim royalties every time SNL/late night talk show shows a clip of him saying dumb things? (Iâm pretty sure one of these shows used a combination of impressionist/deepfake to mock him too, and you KNOW Trump is petty enough to actually do it)
Clips being used in such a way would fall under Fair Use and Parody, no?
You do raise a good point about knock-off effects, but things like âclip showsâ and similar segments are generally protected by existing law.
Again, not a bad point overall, and definitely thought-provoking, but I think this one specific example has an âout.â
Great debate and I wish I could add something to the discussion, but Iâm only on my first cup of tea and the caffeine doesnât hit like coffeeâŠ
Can we all do that then, for all that encompasses us. So that all these internet business also have to stop selling our info since itâs ours.
Your data cannot be trademarked. Itâs a usage pattern.
Gâdangit
deleted by creator
It wonât happen because complex models need beefier hardware. And I donât mean âmy gaming PC is real good!â beefy, but more like âonly companies/institutes/rich people can afford and maintain this kind of hardwareâ beefy.
The majority of people will be dependent on companies offering access to their models and we know how that goes. Even when people are trying to train open models, this will be a power only already privileged people can wield. Further increasing the gap between rich school kid and poor school kid, rich school and poor school, rich country and poor country.
Because, even though the current AI is not the Sci-Fi âactual AIâ some people seem to think it is, it can be quite the powerful tool and potentially change how (rich) people work.
âonly companies/institutes/rich people can afford and maintain this kind of hardwareâ beefy.
So go rent some space on AWS for the weekend, it maybe costs you $500 to have a voice model. Itâs nowhere near as cost prohibitive as you think. Heck, iOS is going to have it on-device. Maybe not as good as this voice model, but free on consumer level hardware.
deleted by creator
Once the tech of generative AI is out of the box, you canât really put it back in the box again.
(Gotta sneak in my movie reference today. đ)
What are you smoking that you think thatâs even remotely the future here?
deleted by creator
David Attenborough documentaries for eternity!
Or for at least thirty eight thousand years
Fucking magnificent! Iâll be playing this in the background while working.
That will be AI Attenboro promoting fossil fuel interests camouflaged as environmentalism.
Iâve seen at least one youtube channel use his voice to narrate warhammer 40kvideos
Removed by mod
Itâs the low hanging fruit. Voice acting is easy. You just sit and talk into a microphone. It was always going to be first cab of the rank to replace.
Any job that requires subjective analysis and opinion will be harder to replace. But it will go eventually as well.
One day they will match up AI with autonomous machines and then manual labour will be completely replaced as well
Iâve no idea whatâs going to happen to most people when that happens. A few rich people controlling every job in the world. There will probably have to be a shift to proper communism so the state owns all the robots and AI and the people just exist in what the state provides.
Itâs going to be bad except for the very few at the top.
Voice acting is easy. You just sit and talk into a microphone
There is more to voice acting than just talking into a mic. Can you honestly say youâve never seen an animated show, or played a video game, and noticed that one character is particularly good? Or particularly bad?
Itâs easy to replace. Not necessarily easy to do yourself. Voice simulation is trivial compared to things like replicating a full person or doing complex tasks that arenât the same every time. Or making subjective judgements.
Itâs why voice acting will be gone in the near future.
Donât piss off Stephen Fry. You donât know what itâs like to have a Cambridge-educated wit fighting against you.
Iâd say itâs not just his intelligence and presentation. Thereâs a high level of bravery there. You never know how someone is going to react when you tell them their beliefs are suspect (or outright bullshit) and just saying this opens him up to derision or worse from any rando with a soapbox to stand on.
Voice actor here. Thereâs an ad on the radio that sent in an audition tape for. I keep hearing it and Iâm pretty sure itâs an AI voice. Thankfully itâs awful. Unfortunately, only a few people will notice or even care.
It astounds me that people tolerate bad computer voices, iâm rather picky which real actual human voices i can tolerate listening toâŠ
I hear AI voices on YT videos all the fân time. Itâs monotonous, super consistent pauses, and not obvious until you listen for 30 secs or so.
⊠And they pause for hard carriage returns mid sentence when the copy / paste goes wrong.
Well, thatâs depressing - to have it on the radio is an intrusion in real life in a way - outside the digital space.
I will never pay for an AI audio book. Never. Why would I pay a book company when I can train my own AI voice clone?
Book publishers donât realize how hard theyâll fuck themselves over if people realize how easy this is.
At that point, why not just pirate the audiobook? If itâs going to be AI anywayâŠ
We would never!
deleted by creator
Can you give a short tutorial how I exactly I can replicate your process?
deleted by creator
Legality I guess. But good point
Oh hell yeah itâs just the beginning. Go check out some popular twitch streamers, they have TTS of tons of celebs using their voices. Those idiots are gonna get sued so fucking fast once celebrities figure out computers
So never then?
Really hard to sue someone if youâre just using sound bites, I doubt actors give zero shits about twitch streamers.
Oh I definitely think theyâll care. Itâs using their likeness for their own personal gain, the actors will want some of that money
Unless some streamer is making millions using a single actors likeness. I donât think thats who Fry is talking about here
Some of these streamers are making millions off the donations they receive to use TTS on their stream
You canât copyright or trademark a voice. Itâs 100% legal to clone someoneâs voice and use it however your like. Basically, a personâs voice is public domain (according to the law as written right now).
You may be thinking, âwell, theyâre just going to change the laws now that rich and powerful people are concerned about it.â Itâs certainly possible that theyâll try to do that but in reality itâs not workable. Hereâs why: What makes your voice your voice?
A personâs voice (and accent) is unique for sure but using technological means to detect this is nearly impossible because thereâs only so many possible variations of the human voice. Chances are thereâs at least 80 million people with the exact same vocal chord sound as you. They all probably wonât have the same accent but thatâs incredibly opinionated.
If a law gets passed giving people the right to copyright or trademark their voice (i.e. the algorithmic signature of it) there will instantly be millions of completely Innocent âviolatorsâ who are suddenly having their videos and music removed from everything because their voice matches the signature. Itâs not workable. At least not technologically.
Another problem is that peopleâs voices change over time. Imagine if a kidâs voice was copyrighted⊠how many new children would suddenly be caught in that law enforcement net just by speaking? The voices of children are even less unique than adults!
They arenât sound bites of anything the celebrities have said. They are original clips generated from text submitted by the viewer who has paid a fee for that to happen. Some celebrities actually sell their services this way and the streamer is taking that revenue from them.
So itâs even more a nothing burgerâŠ
Sue them how? You canât copyright a voice.
Well, you probably can copyright aspects of your likeness that are sufficiently distinct. Just your voice might be too generic, but your voice and accent and mannerisms etc. presented in context (e.g. Trump-bot used in a political ad) might be specific enough to be convincing in court. I donât think streamers are getting sued (successfully) at any point though.
I think/hope we eventually set some sane legal ground rules for the protection of individualsâ likenesses, causing (or stemming from) some high profile court case(s) involving serious offenders (e.g. publishing an AI-narrated audiobook without the original VAâs consent). Once thatâs in the media for a bit (again I think/hope) we see companies like Twitch and Youtube ban the use of AI imitations on their platform to protect themselves. Just like DMCA.
You can not copyright aspects of your likeness. There are specific things that can be copyrighted in the US, and that is not one of them.
There are laws against having your likeness used for commercial purposes but that gets pretty murky when it comes to voices.
less than 20 second clips get a fair use exemption.
âfair useâ varies by jurisdiction and usually has a bit more than just sample size or length as a criteria.
Not really. You get the occasional loon that thinks they can copyright it unless it is a critique, but Courts have ruled the use transformative when it is a sound byte in a performance and not like a soundboard application or other such. And if it is a soundboard clip then the rights would be owed and paid by the company making the soundbyte and not the performer. The folks on twitch or whatever are on fairly safe ground.
What about celebrities who sell personalized messages? They have a right to make money off their likeness, not anyone else unless theyâve been given permission. These streamers are making money off their likeness with no cut going to the celebrity.
Not remotely the same thing. A streamer playing a clip of the original Artist saying the famous line isnât going to cause said Artist to want to attack the use. Playing Mark Hamill yelling, âYou are not my Father!â, isnât a personalized message nor is that what the streamer is selling.
Thatâs not what we are arguing about here. The streamer is using AI to mimic the celebrities.
That is what was stated above. Sorry you missed that.
Fair use is a legal defense, not a law. It doesnât mean you canât be sued and it doesnât guarentee youâll win if you are.
You need a Lawyer willing to take the case and a Judge that wonât laugh while sanctioning them. I did production work for Years and this is how their Lawyers explained it to me.
Oh shit I didnât know that. Do you think thatâll apply to AI generated TTS?
Are the clips just soundbytes? then yes.
What weâre talking about is not so much AI itself but who owns the data set that the AI is created and trained with?
Individuals should own the right to themselves, but if they want to sell it as a data set, then so be it.
They can have restrictions as to how the AI that uses their data set to be created can be used and for what purposes.
I disagree with this because companies will start enforcing a âif you wish to work for us you must give the ability to use your data set including after termination and etc with no further compensationâ. This needs to be strictly a per individual per instance basis preferably requiring the person who owns it be the primary sale person included in the transaction.
Yeah itâll definately take legislature to prevent scenarios like you described, but at the end of the day, a person should own their own identity, and nobody else should be able to make a 100% copy of it.
There are going to be some laws hastily passed for this that is going to put impressionists out of the job. If it is Rich Little or and AI impersonating Howard Cosell, how is it any different?
Impersonators are humans who also need to eat. Impersonating requires practice and talent. Impersonating doesnât put the impersonated out of a job.
Be real, Stephen. Youâre voice wasnât stolen from the audiobooks.
It was stolen from every recording of your voice :D
Fry ⊠is the narrator of the British Harry Potter audiobooks.
Well, thereâs your problem. Should have got Hatsune Miku herself to narrate the books she wrote.
