Google’s AI model will potentially listen in on all your phone calls — or at least ones it suspects are coming from a fraudster.
To protect the user’s privacy, the company says Gemini Nano operates locally, without connecting to the internet. “This protection all happens on-device, so your conversation stays private to you. We’ll share more about this opt-in feature later this year,” the company says.
“This is incredibly dangerous,” says Meredith Whittaker, the president of a foundation for the end-to-end encrypted messaging app Signal.
Whittaker —a former Google employee— argues that the entire premise of the anti-scam call feature poses a potential threat. That’s because Google could potentially program the same technology to scan for other keywords, like asking for access to abortion services.
“It lays the path for centralized, device-level client-side scanning,” she said in a post on Twitter/X. “From detecting ‘scams’ it’s a short step to ‘detecting patterns commonly associated w/ seeking reproductive care’ or ‘commonly associated w/ providing LGBTQ resources’ or ‘commonly associated with tech worker whistleblowing.’”
“…locally on device without connecting to the internet”
How would it then report such behavior to Google, without internet?
If it notifies the end user, what good does that do? My phone is at my ear, I don’t stop a conversation when another app sends a notification while I’m on a call.
This will 100% report things in the background to Google.
You’re putting a very large amount of trust on something which may simply require the flip of a switch to add the specified information to be sent back to Google along with all the regular heavy telemetry already feeding back…
Mega hot take on this site: I have no trust in Google
There are a few ways this could work, but it hardly seems worth the effort if it’s not phoning home.
They could have an on-device database of red flags and use on-device voice recognition against that database. But then what? Pop up a “scam likely” screen while you’re already mid-call? Maybe include an option to report scams back to Google with a transcript? I guess that could be useful.
Any more more than that would be a privacy nightmare. I don’t want Google’s AI deciding which of my conversations are private and which get sent back to Google. Any non-zero false positive rate would simply be unacceptable.
Maybe this is the first look at a new cat and mouse game: AI to detect AI-generated voices? AI-generated voice scams are already out there in the wild and will only become more common as time goes on.
My bet is it will work like their federated text prediction in gboard.
I assume it means the “AI” bit is running locally (for cost/efficiency reasons and so your actual voice isn’t uploaded) the results are then uploaded wherever (which is theoretically better but still hugely open to abuse)
How would it then report such behavior to Google, without internet?
It doesn’t
In a demo, the tech giant simulated a scam call involving a fraudster impersonating a bank. A pop-up message appeared, encouraging the user to hang up.
If it notifies the end user, what good does that do?
You can’t see why it might be helpful for a user to know that they’re speaking to a scammer?
Blasting is all fine and good but they should’ve slammed them really.
Or spanked
Let’s talk about wiretapping laws and states where two-party consent is required to record a call.
Where I live, I must notify the other party that I am recording. If not, it’s illegal. Also, any audio recorded without consent is not admissible in court.
“Not admissible because it was illegally captured” didn’t give me the warm-and-fuzzies this comment sounds like it should’ve.
Complete tangent but what is two-party consent even for? I can imagine it gets in the way of getting a lot of evidence in cases of domestic abuse or organized crime.
Caller: [threatens me for 45 seconds] Me: “Could you call me again and repeat all that, for the recording?” Caller: [hangs]
Means both parties have to agree to be recorded (usually at the onset of a call).
“Be advised, this call is recorded for quality assurance purposes” at which point you could hang up. The 4th Amendment still applies in America, regardless of what local cops and prosecutors believe. You have a right to privacy in two-party conversations.
Oh, that makes more sense!
not admissible in court
I get what you’re saying there but that sounds very much like the kind of detail that s possible future Trump administration wouldn’t much care about
Does not matter who is president of the US.
That is probably the dumbest thing you said all week. At least, I hope it is because i don’t want to know how much dumber than that you can get.
You said a Trump presidency. Do you honestly believe that no other person is capable of being an authoritarian? Guess you don’t remember Obama.
Second reply as the first was just … Trying to understand your mental state.
Trump claimed multiple times that he loves and admires dictators. Not my claim, his. Trump said he wants to be a dictator. He said he wants to be a dictator for one day. This week he said he would like to add a third term for himself (against your precious constitution that you always love to ignore when it’s inconvenient)
No other president in the history of (mostly rather shitty) presidents has ever done any of those things.
And you want to steer away from Trump to another president when talking about authoritarian?
The guys literally told you he wants to be a dictator, why won’t you believe him?
Why has the US collectively been taking crazy pills making conversations like these even necessary in the first place? What is wrong with you people…
… Wut?
Obama was authoritarian? I’m intrigued; Exactly what mental gymnastics are you using to get that idea in your head?
Few simple facts. Obama served twice. First time he won, and he had to campaign again to be elected a second yime.Second time he couldn’t run again for the third time, so he didn’t.
Where exactly do you see authoritarian? Fox news quotes not allowed for answers here, I don’t need sad lies. Simple facts, what did Obama do to be authoritarian?
I can’t wait…
Ask how many civilians Obama killed without congressional approval under the War Powers Act.
Google the IRS targeting conservatives under Lois Lerner and sanctioned by the Administration.
And you call that authoritarian?
Man you need to leave the house more often.
For one, he supports democracy, which kind of already takes the foundation away from under your arguments.
I’m not saying that any of those things are good (though the latter seems to not be something he directed anyways) but a lot of that is what the US does, or have you forgotten Iran, Iraq, Chile, Nicaragua, el Salvador, Afghanistan, Vietnam… I can go on for a while, but you get the idea.
None of it is good, but if you want to compare Obama to any Republican president, I can quite easily tell you he’ll come out on top. I don’t particularly like US presidents, most are really rather dumb, egocentric, and so on, but he really wasn’t that bad.
There’s no recording taking place…
So that data then goes nowhere? I don’t believe that.
I mean that’s fair, but the answer from Google is yes.
One of the things they glided around was whether a lot of this on-device stuff needs a special processor chip with AI+security to work?
The Pixel phones (especially newer ones) made by Google have them, but the vast majority of Android phones don’t.
So either these features only work on latest Google phones (which will piss off licensees and partners), or they’re using plain old CPU/GPUs to do this sort of detection, in which case it will be sniffable by malicious third-parties.
And let’s not forget that if the phone can listen to your conversation to detect malicious intent, any country can legally compel Google to provide them with the data by claiming it is part of a law-enforcement investigation.
Things are going to get spicy in Android-land.
It’s not just google who have AI stuff built into their phones. All recent SoCs I have seen have had NPUs going back the last couple generations. A lot of older or cheap phones won’t have one, but the new devices will.
I don’t see the problem with using the phones normal GPU. This shouldn’t be more insecure than making a call currently is. I am pretty sure android phones don’t have a secure enclave just for making calls as you can give different apps access to calling features, and most calls I make are through third party apps anyway, not via POTS. That being said android is pretty secure anyway provided you don’t give permissions to the wrong app. It’s more secure than your average Linux system, as each app has its own user and is only allowed to access things it has explicit permissions to access. Secure enclaves aren’t all that in my opinion.
And let’s not forget that if the phone can listen to your conversation to detect malicious intent, any country can legally compel Google to provide them with the data by claiming it is part of a law-enforcement investigation.
The point of doing it locally is the audio never gets sent to google directly. That being said they could definitely do some dodgy things by training the ML model to search for words like abortion, drugs, transgender, etc depending on what the laws are in the country the phone is being used in.
As a OpenSource app with no need for centralized server it would be great. I want that. As spyware configured out of my control absolutely not.
I think you do need some central server, and then check if a lot of users report certain number blocks for spam in a short amount of time. No need for AI on this one. Isn’t that how most phone spam blockers work?
I have that now, its OK. But spammers cycle through numbers so quick that good amount get through
That is how it already works. I don’t think most people have as much of a problem with that as complete client side screening.
It might be a good feature for the elderly as long as it’s local and optionally enabled (especially if it can be enabled only for unknown callers).
Yes, I understand you would never really know if it’s not always enabled. But then again, you currently don’t know if anything similar isn’t already enabled.
For other users, again potentially useful if it’s opt in. However, many people (myself included) simply don’t answer the phone anymore unless it’s a caller we already know. I use Google’s call screening feature for any other caller not in my contact list already, and I would estimate about 1 in 20 or 5% of such calls I receive aren’t spam (marketing or fraud). Of those non-spam calls, the majority are appointment reminders I don’t need.
So would I turn this feature on? No, I don’t have a need. Could it be beneficial for the elderly? Yes, but probably not implemented in a way where it would actually be effective.
Honestly, if it was transparent enough, I’d be fine with a service you could turn on that would listen to the first x seconds/minutes of a phone call to try and detect an AI generated voice. I have family that would 100% wire thousands of dollars if someone with their kid’s voice asked them to.
deleted by creator
Think of the
childrenelderly!I mean, yeah, exactly. Keep in mind scammers are targeting vulnerable people. Granted I don’t see how such a feature will work on my grandmother’s flip phone.
Did the blast with cum??
Top 10 Cum Blasts You WONT BELIEVE (subscrab)
#1: 120 mm Penetration Cum Blast (PCB) - https://www.drdo.gov.in/drdo/120-mm-penetration-cum-blast-pcb-and-thermobaric-tb-ammunition-mbt-arjun
Checkmate
I feel like this would require cooperation from the manufacturer, as Google doesn’t actually provide the Phone app (except when they are the manufacturer).
except that it does. some OEMs like Samsung serve their own (or at least Samsung used to; not sure if it’s still true), but it’s definitely available on non-Google-branded phones.
It’s available on all phones, but they all have their own version, forked from long ago. Even the standard AOSP Phone app has long split from Google (who have ceased open source development of the app).
I’m well aware that it’s available to install on all phones, but I’m also fairly sure that other OEMs do use it as the default dialer, too. I saw it preinstalled on one of my mom’s phones, either her current midrange Samsung, or her previous Xiaomi.
Google doesn’t actually provide the Phone app
Yes they do. Anyone can download the Google Phone app.
But downloading an app won’t install it as a system app.
What difference does that make?
They might be taking about Google Fi.
They seem to be offering a lot of this sort of thing as a value-add for buying a Pixel
Can’t really get behind her on this one. Assuming it works the way Google says it does, it couldn’t be used for any of those things she’s concerned about.
If it doesn’t work that way, well then your entire device is compromised to begin with, because Google controls the entire operating system.
Assuming it works the way Google says it does,
There was a time when I trusted Google to be open and honest.
But that time is long in the past.
If it doesn’t work that way, well then your entire device is compromised to begin with, because Google controls the entire operating system.
