Security Confidential S12 E9 Dmytro Bielievtsov

Manoj (00:01.183)
Hello everyone, this is your host Manoj Tandon welcome to another episode of Dark Rhino Security, Security Confidential. Today joining us from half a world away in the Ukraine is our honored guest, Dmytro Bielievtsov. He is the CTO and co-founder of Respeacher. Very cool company. I mean, these guys have been doing some very, very cool things. They were the first to be adopted by Hollywood production studios in 2019. They have worked on many video games, animation studio projects. Some of the things that you would have heard about would have been Mark Hamill’s voice in The Mandalorian. They are the team that was responsible for de-aging it. James Earl Jones’ voice as Obi-Wan Kenobi. You know, those are some of their credits and many more.

They’ve been featured in Forbes, The Guardian, TechCrunch, VentureBeat, to name a few. Dmitro, thank you for joining us and we’re really glad you’re able to do this all the way from over in the Ukraine.

Dmytro Bielievtsov (01:11.638)
Yeah, hi, Manoj. Thanks a lot for this amazing intro and really happy to be here. Thanks for having me.

Manoj (01:19.359)
Yeah, how, I have to ask you, being in the Ukraine, how have you managed to run such a high tech company in a time of crisis over there?

Dmytro Bielievtsov (01:32.354)
Right. Yeah. So I guess before the time of crisis, it was, it was, uh, it is still like a really good place for, for having a high tech company because of, you know, the, uh, Engineering talent that we have in Ukraine and the kind of the deep tech culture that, that has been developing, uh, in Ukraine since, I don’t know when.

So yeah, it’s a pretty good atmosphere and environment for starting a high tech startup like ours. But it is true that it was a super challenging time when the full scale invasion began, you know, started by Russia. And so…

I think the sort of the way we managed it is by a preparing to this as much as we could for some things it’s hard to prepare, but in general, like we were aware, well, we had a guess that probably something’s going to happen soon. We’re like hugely underestimated, I guess, what happened. But anyway, we prepared a lot. We moved.

some people around Ukraine just to make sure that in case the internet breaks down or something, we’ll still be online and we’ll be able to talk to our clients and, you know, be staying in touch. Another thing is the company’s culture that we have, the culture of like ownership and communication. Like from the management side, we’re upfront with everyone about what’s going on. We kind of kept everyone on.

in updated about what we’re doing as a company, who’s going where and who needs help. We’re kind of helping as much as possible the team members. And from the team’s side, it was like tremendous. It was amazing to see how everyone kind of took ownership of the projects that we’re working on and kept communicating with the clients, albeit, you know, even in these super challenging times.

Dmytro Bielievtsov (03:50.378)
And I guess the third thing is maybe

Dmytro Bielievtsov (03:56.918)
Having an ongoing purpose, it kind of helps people distract from really tough things that are happening. And we’re kind of, we’re generating, we’re helping economy in Ukraine. So that kind of gives us motivation to keep going with this.

Manoj (04:16.415)
Wow, it’s, see, you guys have something special. You guys are actually unified in a cause. That’s what it sounds like from what you just described. And that’s not easy to get. You’re unified as a company, as a team, as a country. There’s a lot of power in that.

Dmytro Bielievtsov (04:27.147)
Right.

Dmytro Bielievtsov (04:38.91)
Yeah, a huge cost, unfortunately, but it’s good.

Manoj (04:46.059)
So, you know, you started, you talked a little bit about the company’s culture that has helped you sustain, but what’s the story behind Respeacher? How did you guys get started? What was the vision? What was this, your idea? How did you do it?

Dmytro Bielievtsov (05:04.298)
Yeah, it wasn’t actually that romantic. It was just, so essentially we, there was a hackathon that was organized by Grammarly, which is now like a unicorn Ukrainian startup. And they’re actually like really active in community. They try to help, yeah, they’re very active in community. They hold various events. They try to…

Manoj (05:17.847)
Thank you.

Dmytro Bielievtsov (05:33.99)
facilitate as much as possible. So they were holding this, organizing this hackathon on data science and anyone could come with any idea. So somewhat randomly, we thought that it was gonna be cool to try something in audio because back then the computer vision was, you know, super like more popular than anything else in machine learning. So we kinda dropped in with the idea of like, oh, hey, can we…

Manoj (05:57.344)
Yeah.

Dmytro Bielievtsov (06:00.966)
Sounds pretty simple. Can we just take me saying hello and I don’t know Barack Obama saying hello and Just you know say a thousand different words and train a network to map these things, you know Maybe it will be able to say something new So with this naive assumption, we kind of went to this hackathon and it was a lot of fun took the first place but further results were very bad, but at that point we kind of started thinking that oh

realizing that the potential of the technology and we also talked to a bunch of studios here in Ukraine realized that there’s really there’s a lot there’s a big potential there’s lots of use cases with this technology and at some point we like didn’t do much it was you know just having this as a pet project but then in 2016-ish like

I guess in the end of 2016, beginning of 2017, Google published this WaveNet paper, which was like a little bit of the puzzle that was missing. No one could do super high quality speech parametrically, but now they published this little bit and just something clicked and we thought, oh, it’s the right moment. The field’s going to explode. We better start now.

Manoj (07:11.587)
Okay.

Manoj (07:27.087)
That’s amazing. So you guys came out of an hackathon with a little bit of support from Google. I try to explain to a lay person here, how does this technology work?

Dmytro Bielievtsov (07:40.046)
Uh, yeah. So, um, essentially it is in some sense, like similar to, to how sort of like brain works, it just needs to be exposed to a ton of examples of what you want from it, and then it can learn to perform a task and to generalize to new scenarios. So, you know, like computers can simulate physics, right? You can use, you can use a computer.

Manoj (08:08.982)
Yes.

Dmytro Bielievtsov (08:10.294)
to make, you know, realistic water movement or something. And in a similar fashion, you can use computer to simulate how neurons learn. And so in that sense, we use these artificial neural networks that are simulated by a computer. We expose them to thousands of examples of real speech by different people. And then computers,

try to figure out what it is that, you know, unites all these people. So there’s like a ton of different people, they pronounce, they make different sounds, but there’s something similar about all of it, because all of this is speech, all of this is comprised of small building blocks, that phonetic units. And then this is something that neural networks get kind of deduced from being exposed to thousands of hours of audio. And that’s how our system works. So we showed a ton of audio.

And then by looking at it, it kind of learns that, oh, maybe me and Barack Obama have a lot in common. It’s just the timbres are different or something. And then I can pronounce a new thing once it’s trained and then it will kind of try to make it sound as Barack Obama.

Manoj (09:27.955)
But do you have to identify the attributes that define speech unique? So things like timbre, inflections, tone of voice or rate of speech, and then where in the audio spectrum someone’s voice falls, are these all part of the neural net and…

Dmytro Bielievtsov (09:47.287)
Mm-hmm.

Dmytro Bielievtsov (09:53.834)
Well, the cool thing about neural nets is that you don’t, and it’s kind of the whole point of them is that you don’t need to specify much at all, like in terms of rules. So much like, again, like if you don’t need to explain to a kid what it is that makes a cat, they will kind of figure it out on their own, right? And that’s similar sense when you show, when you expose neural nets to lots of audio.

they will learn their kind of internal idea about, oh, phonetic units are important. You don’t need to tell it that phonetic units are important. So we like, our networks are what’s called end to end. So, which means that we don’t use any intermediate representation, like we don’t use text or we don’t use phonetic transcriptions. We show it audio, and then once it’s trained, you give it

It’s a speech-to-speech system. Once it’s trained, you give it a sample of audio, and then it turns that sample of audio into an impression of a different speaker saying the same content, and performing the same emotions, like same intonations. We do have some controls, so we do supply it with, we do have explicit…

intonation pattern. So we can change pitch, we can maybe make it a little bit less emotional if we want to, we can make it a bit more emotional by kind of exaggerating the highs and lows of your pitch. But other than that, it’s all up to whatever the network has learned.

Manoj (11:21.609)
Okay.

Manoj (11:41.607)
That’s incredible. And have you like overlaid a the generated speech pattern over the real person’s speech pattern and seen it on a spectrum analyzer? Are they do they overlay really well? I mean, can you or can you spot it that okay, this one is the fake? This is the real

Dmytro Bielievtsov (12:03.447)
Oh yeah, that’s a good one.

Dmytro Bielievtsov (12:07.666)
It really depends. I mean, if you just take a rendered, like right out of the furnace, if you just take it like a raw rendered sample from a synthetic network, in most cases it will be possible to, it is still possible to identify it using some clues, some like small nuances, but like by just looking at it,

at a spectrogram, I don’t think it will be pretty hard to tell which one is true and which one is synthetic unless you have like, unless you’ve been trained to tell them apart. And by listening as well, it’s pretty hard these days. And the moment you add a little bit of like reverb or like background noise or some sort of like realistic sound effects, then it becomes really hard, even like both for humans and for deep fake detection algorithms.

Manoj (13:07.531)
Wow. So in a sense, you’re pretty much at the point where you could replicate somebody vision from a speech pattern wise, you could bring somebody to life that doesn’t exist anymore just based on what you might have seen historically even.

Dmytro Bielievtsov (13:27.314)
Yeah, yeah, exactly. So if a person has left some recordings, some real recordings, it is possible to kind of make them say new things if it’s appropriate, of course.

Manoj (13:46.891)
Oh, how did, I mean, it’s pretty obvious that Hollywood would be interested, Hollywood would be very interested in something like this, but how did they find you guys?

Dmytro Bielievtsov (13:58.862)
Oh, it’s a, I guess it’s a funny story. So I mean, of course we, from the very beginning of the company, our main, initial goal was to, Hey, I mean, synthetic speech is fun, but can we make it, you know, as good and as high quality that it will actually at some point in time, see it on, on the big screen. That was kind of our, our beam. And, so we, we did put a lot of effort into.

reach out and things. But what actually got us our first kind of really cool Hollywood contract was just one of our like a super indirect thing like one of our advisors told about to his friend about our technology and then the friend worked at a sound effects or the visual effects studio and then they kind of just you know send it out on a

on internal chat for technical people and then someone picked it up and then it just so happened that people at

Dmytro Bielievtsov (15:07.926)
people at Lucasfilm really needed the tech and they kind of reached out so we started talking and we got our first project out.

Manoj (15:20.139)
So was this somebody at ILM?

Dmytro Bielievtsov (15:24.017)
Um…

Manoj (15:26.571)
The Industrial Light and Magic, the…

Dmytro Bielievtsov (15:28.089)
Yeah, yeah, I think so, but I don’t remember exactly what it was.

Manoj (15:35.539)
That’s pretty cool, man, because those guys are so secretive. And they’ve done some pretty amazing visual effects works, right? I mean, they were the original creators from Star Wars back in the day. So you guys must have really blown them away with the tech. So if you’re doing…

Dmytro Bielievtsov (15:45.696)
Yeah, yeah.

Dmytro Bielievtsov (15:50.978)
Right.

Dmytro Bielievtsov (15:57.858)
Now that I know when I listen to the samples, we kind of used to show off our tech. It’s it’s like hilarious. You know how bad it sounds to me now, but sort of like back then it was. It was kind of, it showed a lot of potential. Of course, the final result was way better, but the sample that we used to kind of show off our tech is quite hilarious now.

Manoj (16:27.863)
So, let me ask you this, like when, for Mark Hamill’s voice or for Obi-Wan Kenobi’s voice with James Earl Jones, how much effort do you have to put into it to create that voice? Does it involve the actual actors coming into a studio and saying a bunch of phrases or how does this all work? How does the process work then?

Dmytro Bielievtsov (16:52.978)
Right, so for like, we do a lot of studio, we call these studio projects. It means that we have, part of it is a service. We have a delivery team that has synthetic speech artists. There are people who have like a lot of experience in professional sound engineering, sound design, but they’re also technical. They can train models and evaluate them and do things like this. So.

within these for these studio projects and we’ve delivered more than 100 of them over the years so these studio projects the pattern is like this um initially a client sends us um a sample whatever they have in terms of data for the target character um and then the people look at this uh people in the delivery team look at the target data

and see if it’s suitable for training. Like if there’s, because it could be like really bad historical recordings that are corrupt and we can’t use them. So sometimes it would refuse. But assuming it works, now the delivery team kind of just takes that and starts training the network essentially. And once the network, it takes a week or two, depending on how urgent the project is and how good the data is to train the network. And then once the network is trained,

we send back drafts, what we call drafts, like initial conversions, to whoever is responsible for this, for the stream at the studio. So they look at it and they maybe use it in their draft mixes. They show it to the director and the director says, oh, I see, let’s change this line, or maybe let’s add some more emotion there. And then they

they go back to the studio, an actor records a bunch of new lines, gets sent to us, gets converted to the target character, and then these iterations go on for some time, depending on the projects. For urgent projects, it could take maybe a week or something, but for big projects, some of the ones that we did for Lucasfilm, it could take months.

Manoj (19:14.251)
And at the end of the project, then, do you does the studio? Do you deliver the studio then an entire model? So then they don’t even need that actor’s voice anymore. They can just all future episodes. They can just generate the voices or.

Dmytro Bielievtsov (19:29.954)
Oh, no, no. Yeah. So our deliverable is rendered audio files, you know, much like a VFX studio, with their deliverable as a rendered part of a particular project with a particular character or particular scene. So in the same way, we generate a bunch of lines that are required for a certain project that we’re working on.

Manoj (19:38.653)
Okay.

Dmytro Bielievtsov (19:57.134)
And that’s our delivery. We never give our models out and we never own anyone’s identity because we believe that the owner of the voice must own their identity. And the permissions they give for studio projects are usually per project.

Manoj (20:21.507)
And that’s a very noble thing to do. I can see though in the future state, there could be less ethical companies that come along that.

Dmytro Bielievtsov (20:25.419)
Right.

Dmytro Bielievtsov (20:34.382)
Um, I don’t, I’m not sure. I mean, of course, yeah, this would be less ethical companies, but then there’s also going to be more regulation in the space and, uh, there are. There’s lots of, um, like actors strikes and, uh, there’s lots of activity in the space and a lot of like awareness about the dangers of AI. So even though there’s going to be more unethical companies, I feel like.

will be able to, the community will be able to restrain them much like, I don’t know, there’s, we’re just sort of, we don’t have too many unethical companies doing, um,

Dmytro Bielievtsov (21:19.27)
I don’t know. So in any other sphere, you can have unethical companies. So I feel like, and those are controlled. So, yeah.

Manoj (21:27.095)
Well, you know, I, we look at in the, in the world of cybersecurity, what you guys have built has huge potential for disruption, right? I mean, if, if we were to play the, the dark side of this out, when, when we look at people trying to influence an election, and let’s just pick on that one. Cause in the U S that’s a paranoid topic that’s been going on for

time for the last four years or so, or God knows how much longer it’s going to go on. But, you know, there was all the fakes on Facebook that were attributed to China. There were, or most of them were, not all of them, of course. There’s been all kinds of social media peddling out there. But now with what you have and what’s available visually, the marriage of the two

you could create a high profile political figure that’s totally fake and have them put out a social media message and it will, there’ll be a chunk of the population that will absolutely believe it, whether you tell them it’s a lie or not, right? And that’s a real danger.

Dmytro Bielievtsov (22:41.442)
Yeah.

Yeah. So that is true. I mean, misinformation is a huge thing, has been a real danger in general. Like think about all the articles that are written by Russian media, for example, about, you know, and a huge problem, even in Ukraine, like a big portion of the population just reads the article. They believe it. And it is a problem.

Manoj (23:02.826)
Oh yeah.

Dmytro Bielievtsov (23:14.742)
With regards to specifically media, I think what we’ve been doing with this content authenticity initiative by Adobe or with XC2PA is a good, is like a strong step forward towards sort of combating potential misuse of deepfakes. So, Artek, specifically Artek can’t…

really be used for this because we require permission, we keep our tech to ourselves. But of course there’s other tech out there, some of it is open source and it could be misused. But what content provenance is trying to do is instead of trying to make algorithms that detect deepfakes, which is, as I mentioned before, getting harder and harder, instead of that they kind of flip the problem

upside down and they say, hey, let’s introduce content credentials, which is a cryptographic way of signing, cryptographically signing a piece of media content. And if someone modifies it, then it will be possible to see that modification. It’s not going to be, you’re not going to be able to verify the certificate. So the idea is, of course, it’s reliant on adoption.

But the idea is for all the big media platforms like Facebook, Twitter, YouTube, whatever, to adopt this technology and flag content that doesn’t have content credentials as potentially dangerous or something like this. In the same sense as we have with HTTPS, right now if you go to an HTTP web page, even if it’s harmless.

you’ll still, like most browsers will prevent you from seeing it and like warn you that, hey, this is potentially dangerous because it’s not encrypted. So in the same sense, if you submit a fake video of a politician saying something to YouTube and you want to conceal your identity and you’re not going to sign it with anything, then the YouTube will notify users that, hey, this is potentially fake.

Dmytro Bielievtsov (25:40.074)
because all the good players are kind of interested in signing their media with their credentials and Respeecher’s Marketplace, for example, signs all the outgoing audio with our content credentials.

Manoj (25:40.196)
And

Manoj (25:59.099)
And that, I guess, for major media, that makes a lot of sense, and it’s a great step in the right direction. But what about the cases where someone uses a similar technology to, in cybersecurity, now there’s been an ongoing, if the target is rich enough, if there’s enough value in this target, whoever you’re going after, then,

to compromise their identity, you know, use a technology like yours to create their voice and then you can call their bank, you can call or call their company and say, hey, this is Minoji, I need you to wire half a million dollars to Demetro right now. And the person on the other side, I know Minoji, yet this sounds like him, it’s the right voice.

Dmytro Bielievtsov (26:48.203)
Yeah.

Manoj (26:57.067)
Oh, what’s your data? You know, all the stupid, easy questions that everybody can hack and get access to. You know, how old you are, what’s your social security number, your address, all the easy things.

Dmytro Bielievtsov (27:07.996)
Yeah.

Right.

Manoj (27:11.703)
the other end of the line, that’s better than hacking. Cause now I can get someone to send money out without having to do anything. They’ll do all the hard work for me.

Dmytro Bielievtsov (27:23.818)
Yeah, definitely. So this is called vishing when someone

Manoj (27:29.008)
I didn’t know that. I learned a new word today. Voice fishing.

Dmytro Bielievtsov (27:32.19)
Yeah, it’s like voice phishing, which is, yeah. Yeah, and like voice cloning is, there has been several cases now, which do exactly what you just described. And yeah, again, it is an issue. But the good thing is that it follows like a similar pattern to phishing, and you could use similar tools to…

to fight against it. Like for instance, we’re talking to a bunch of security consulting firms. And in particular, we had like a joint blog post by Richie May about this. And the idea is to use voice cloning together with security consulting firms to perform social engineering attacks on companies.

to try to expose whether employees are susceptible to these kinds of attacks. And then sort of, and then remediate by introducing appropriate training and appropriate like internal processes in the company that would make sure that people are aware of this. Because I mean, I think it’s gonna be pretty hard to…

to fool me with vishing, for example, because I’m aware about these things. In a similar sense, a well-trained personnel in a firm and a bunch of well-established processes would kind of most probably help with most of these attacks.

Manoj (29:15.007)
Yeah, you know, there’s been so many examples of like a sim swapping, and I’m sure you might’ve run across that, where you call up the telephone provider and you convince them that you are the person and you wanna swap the swim over to another phone. And of course you’re a total fake and now you just have their phone. And now you have your second authentication factor and you became them. And now you can do all kinds of wonderful things.

Dmytro Bielievtsov (29:44.93)
Right.

Manoj (29:46.927)
I don’t know what the solution is. I think there’s going to be enough things that are gonna come to light that there will be a solution for this. Maybe there’s a, in the future, there’ll be a way to digitally identify us.

Dmytro Bielievtsov (30:04.456)
Mm-hmm.

Manoj (30:06.751)
That’s wishful thinking, but.

Dmytro Bielievtsov (30:06.998)
I mean, my personal view… Oh yeah, sorry.

Manoj (30:11.719)
Now go ahead. Love to hear your view.

Dmytro Bielievtsov (30:13.694)
Yeah, my personal view is that there’s not going to be a solution. The solution is in kind of raising awareness about this and attacking the issue from different angles, like content provenance is an important thing that I mentioned, awareness and personnel training, social engineering attacks.

or print pen tests, let’s call it penetration tests, are important thing. Then deepfake detection algorithms are important thing. They’re not gonna catch the most advanced technology, but they’re gonna catch all the cheap fakes. Then fingerprinting is when you introduce or like a watermark in audio. So, and then white adoption with media distribution channels. So I feel like,

Given that there’s a lot of talk about this in this space and a lot of awareness, there’s a good chance that most of these tools will be implemented and it will be relatively safe.

But maybe that’s wishful thinking. I don’t know.

Manoj (31:32.934)
I think it’s, you know, just like we couldn’t have imagined this, there are solutions that we haven’t imagined yet that are absolutely going to come to light. I think you’re spot on. You know, nothing’s going to be 100%, but as long as we put the right controls and processes in, hopefully we’re doing the right thing. And there’s enough ethical people

that the majority of stuff out there isn’t garbage and it’s real. Now you’ve done so many really cool studio projects. Is there one that was a favorite of yours personally?

Dmytro Bielievtsov (32:11.222)
If I had to pick one, probably the Luke Skywalker’s voice for Boba Fett. And maybe it was, yeah, I guess it was mostly a personal thing because it was a lot of material, like just not a single or two lines. It was like a bunch of, it was like almost, you know, a big portion of an episode. And it was…

really sort of the highest quality that we could do by that time. And there was like, they didn’t apply, you know, a huge amount of posts because the quality was pretty good. And then there was no, you know, much, there wasn’t much music. The speech was like really kind of out there, that really exposed and sort of intricate and

It was kind of a cool moment for me to realize, oh, hey, like here we are. Here’s the synthetic speech on a big screen. It sounds great. It sounds natural. And it’s something I was like really frankly, like skeptical about when we started the company, I was like, I guess there’s, we should take a shot, but it’s pretty like unlikely that it’s going to be, you know, that good. Um, so it was, yeah, it was kind of a, a nice personal moment for me.

Manoj (33:35.535)
Now in that one, you actually de-aged Mark Hamill’s voice, right?

Dmytro Bielievtsov (33:41.63)
Yeah, we brought back his young voice by training the network on his young recordings.

Manoj (33:53.875)
I was going to ask how you did that, because in my mind, it’s like now a computer is actually creating information where it didn’t exist before. You know, a lot of it’s

Dmytro Bielievtsov (34:06.246)
Oh, but it kind of did, right? So we took the… They provided us with a bunch of archival recordings of Mark Hamill’s young voice. So we used that as the target. It is true that computer, you can, in the absence of young voice, and we’re working on that, on one of the…

upcoming projects. So it is true that you could DH sometimes make the network imagine how someone would have looked or would have sounded like when they’re young when they when they were young. And that’s going to be a guess. I mean, there’s nothing I don’t think there’s really anything magical about this in that sense that I don’t know, you can imagine how someone could like would look when they’re old. And it might be

a good guess, but it’s not gonna be the truth.

Manoj (35:08.903)
Right. It would be remarkable. You know what? I actually think we could probably get there. You guys are smart enough. You’re gonna figure this out. It’s just like today when you see the iPhone apps, you can remove a person or an object out of the picture and it fills it in, in the background. It’s making up some data. It has to be, because once you remove the object,

it has to make up some data to stick in place of that object. If that’s possible, why can’t that same kind of, why doesn’t the same sort of mathematics apply to speech? And who knows?

Dmytro (35:38.764)
Yeah.

Dmytro Bielievtsov (35:48.906)
Yeah, it’s possible to the same extent. What I’m rather saying is that when you remove an object from a scene and then in the actual scene there was, I don’t know, like a rabbit behind that object, there’s no way a network would have guessed that. And in the similar sense, we can have a pretty good guess about what the young voice would sound like, would have sounded like, but then…

There’s no way to know that for like 100%. That’s what I’m saying.

Manoj (36:19.959)
You know, I guarantee you, Demetrio, I bet you there’s a university out there that has researched voice aging. And they probably have graphs and charts and tons of data that shows how the timbre pitch. Everything changes with age. You might be able to build a forecasting model. There’s somebody out there. There’s you go looking hard enough. You know, it’s surprising what you find.

Dmytro Bielievtsov (36:49.63)
Maybe you’re right. We’ll see.

Manoj (36:53.036)
You guys also have done work with animal voices. Have you tried playing them in like a real animal environment and seeing if the real animals actually buy it? Do they?

Dmytro (36:56.852)
Oh yeah.

Dmytro Bielievtsov (37:08.418)
Nah, we tried it with dogs, we had like a small video about it, but unfortunately because the driver is still a human, so in order to make barking you need to kind of bark yourself, it’s just, it’s going to make it sound like a real dog bark, it’s not going to sound like a human trying to bark. But you still, I guess, in order to imitate…

Manoj (37:15.732)
I’m sorry.

Dmytro Bielievtsov (37:36.01)
animal language, you need to bark the right way with right intervals. So we, firstly, we didn’t have a professional barker on the team. Let me try that. Yeah. That’s right.

Manoj (37:40.299)
Yeah.

Manoj (37:45.342)
That would be a tough resume. I mean, go look for that on.

Dmytro Bielievtsov (37:53.846)
Yeah, but it’s I don’t know. I actually I really like that. That Yeah.

Manoj (37:58.207)
You know, there’s so many cool applications if you guys do that, DeVito. In the world of, you know, in field biology, you know, if you want to pull in some strange kind of critter and you can mimic the actual critter that you want to attract, well…

Dmytro Bielievtsov (38:16.418)
Or like, we’re for hunters, I don’t know. You know, like they do the duck sounds. Hahaha.

Manoj (38:18.611)
Oh yeah, I mean, well, that’s big in North America. Yeah, right now you’re sitting there, you know, with a little gadget in your mouth or whatever, you know, and trying to mimic a duck or a moose or whatever it is that you’re trying to do, but be a lot easier if you just play it on your iPhone or your Android and sit here.

Dmytro Bielievtsov (38:31.479)
Yeah.

Dmytro Bielievtsov (38:41.962)
Yeah, it’ll be a vishing, vishing attack against the moose.

Manoj (38:42.103)
stream the voice, fishing attack against animals. So what’s next for you? Yeah, you’ve done this, this is a really cool work. Do you, what are your plans for re-speech? What are you guys planning on doing here in the future?

Dmytro Bielievtsov (38:58.674)
Yeah, totally. What we’re, what we’re up to is we do want to keep going with our studio directions because it’s, it’s a lot of fun to make. And we also take pride in making these voices for cool pictures out there and to, you know, participating in movies and creating nice movies and films. So

So we’ll keep doing that, but we’re also exploring the ways of democratizing the technology that we use in studios for like wider audience and smaller content creators that might want to scale voices or they might need more diversity in their small game or movie. So for that direction, we’ve…

built this, we’ve released this B2C platform called Marketplace. So we’re kind of actively developing that and bringing like adding new features, adding more voices to the library. And on that platform, you can speak with one of the voices, with several voices from our library and have multiple characters, voiceover multiple characters in your game. So we’re gonna keep working on that, but…

Manoj (39:59.767)
Okay.

Dmytro Bielievtsov (40:22.726)
What’s also cool is the real-time voice conversion, where we can speak with different voices in real time on Google Meets or something like that. And also in call centers, an agent might pick a voice that is the most suitable for the client they’re serving. So the real-time direction is something really exciting and we have cool

We have good results there. We have several, it is B2B for now. We have, we’re serving it for several clients. We also did a bunch of, couple of shows with like on-prem real-time voice conversion. And also what’s really cool about it is that it could also be used for medical purposes. Like for example, we’re working with this,

Dmytro Bielievtsov (41:23.646)
with a medical device company that makes these electrolarynxes. Electrolarynxes is this device that people, that laryngectomy patients are using. Those are people who had throat cancer and they had their vocal cords removed. So they need some source of pitch.

Manoj (41:42.068)
Yeah. Yep.

Dmytro Bielievtsov (41:51.378)
And so what they do is they use electric layering sets. It’s like a vibration vibrating device that you put to your, to your, to where your vocal cords were. And then it kind of generates the vibrating sound and a person could speak. But then the speech is sounds, you know, somewhat robotic and what you could do with real time voice conversion is you can try to turn that robotic speech into

natural speech that sounds similar to what they sounded like before they underwent the surgery. And yes, so this is something that really inspires us to keep peep-waring on the real-time technology.

Manoj (42:35.991)
And Dimitro, is the compute power, the DSP technology is advanced enough and miniaturizable enough where you see this becoming reality in the not too distant future?

Dmytro Bielievtsov (42:49.694)
Yeah, actually, like cell phones have really powerful GPUs on them. And our networks are not that huge. They’re so like these models like chat, JPT or something, these large language models are the, they need like supercomputers or like data centers to run on. Uh, but phonetic models or speech models that we have, they’re actually relatively lightweight and we had, uh, we’ve been able to run.

them on small devices like portable Nvidia graphics chips that consume 3 watts of power, you can still run it. The latency is going to be a little bit bigger than on a gaming laptop, but it’s quite close. It’s actually just a matter of optimizing the software.

Manoj (43:31.123)
Oh.

Manoj (43:43.571)
That’s amazing. I could, you know, it would be really cool as if you could come up with a real time language translation model. So I could speak in English and then you can hear it in Ukrainian. And then and it sounds like my voice. So that would be that would be very impressive.

Dmytro Bielievtsov (44:01.982)
Yeah, you can cook something up even with current technology, like stack APIs together and have some like this. It is, however, maybe one technological obstacle is that it’s gonna be probably hard to be better than, in terms of latency than these, I’m sure what the right name is, the translators, the translator people who work in real time.

Manoj (44:04.951)
that

Manoj (44:29.19)
Yeah. The language translating people. Yeah.

Dmytro Bielievtsov (44:32.382)
Yeah, and imagine like if you’re speaking German, like the last part of the sentence that could change the whole meaning of what you said before. And then, so these are kind of challenging things that need to be accounted for in these scenarios.

Manoj (44:47.511)
Well, yeah, they are challenging. It could also be quite funny, but I, you know, you guys are going in a remarkable direction. It’s fascinating.

Dmytro Bielievtsov (44:58.782)
Oh yeah, we’ll be there for sure. The industry is pretty close. I mean, there’s the tech is… You just need to connect the dots. I feel like pretty much the tech is there.

Manoj (45:10.248)
So, you know, we’re at the hour here. I wanted to give you the last couple of minutes to plug anything you wanted to any anything you want to let our audience know about.

Dmytro Bielievtsov (45:21.902)
Oh, yeah, maybe one thing that I want to mention is, I really appreciate the support for Ukraine. And then especially what you mentioned personally, and all the support that comes from the community. So probably what I would ask is to

to keep doing that, to keep spreading the word about the issues that are happening here so that what’s happening in Ukraine doesn’t go out of radars, doesn’t get forgotten. And we have this share UA program that we’ve launched to let…

actors and celebrities who want to support, to say words in support of Ukraine, to be able to speak in Ukrainian with their own voice. So we kind of encourage people to join the program and spread the word about our company and Ukraine.

Manoj (46:38.503)
If you could give us the links for that, then we’ll put them in the show notes so they can go onto your site and reach out to you on that topic, if anybody chooses to.

Dmytro Bielievtsov (46:53.738)
Yeah, that’s cool. Well, I guess a research should be able to provide all the information that’s needed.

Manoj (47:02.007)
Fantastic. Well, Demetro, it’s been an honor to talk to you. I don’t know, what is the time right now over there in your?

Dmytro Bielievtsov (47:11.228)
Oh, it’s 9.30 PM.

Manoj (47:13.023)
Okay, well thank you for spending a little bit of your precious evening with us. And we’re honored and you know as things progress don’t be a stranger. Feel free to reach out if you would ever like to come back and keep us abreast. It’s a fascinating technology and congratulations on all the great work and your success.

Dmytro Bielievtsov (47:35.134)
Yeah, thanks a lot. I really enjoyed the conversation. And yeah, thanks for having me.

Manoj (47:41.067)
So did I. Well, thank you. Well, hey, then take care and have a good evening.

Dmytro Bielievtsov (47:46.546)
Awesome. Thanks, Manoj. Take care.

Manoj (47:49.131)
Thank you.