Privacy experts are sounding the alarm over Google’s demonstration of a call-scanning AI showcased at its recent I/O conference. The feature, designed to detect financial scams in real-time during voice calls, has raised concerns about potential censorship and invasion of privacy.
Powered by Google’s Gemini Nano AI model, the technology enables client-side scanning, a controversial practice previously associated with detecting child sexual abuse material (CSAM) and grooming activity on messaging platforms.
While Apple abandoned a similar plan in 2021 due to privacy concerns, pressure from policymakers persists for tech companies to address illegal activities on their platforms. The introduction of on-device scanning infrastructure, such as Google’s call-scanning feature, could lead to broader content scanning by default, potentially infringing on users’ rights and freedoms.
Meredith Whittaker, president of encrypted messaging app Signal, warned of the dangers of centralized, device-level scanning, which could extend beyond scam detection to monitoring other sensitive topics like reproductive care or LGBTQ resources.
Cryptography expert Matthew Green from Johns Hopkins University expressed concerns about AI models analyzing text and voice calls for illicit behavior in the future. He suggested a scenario where users may need to provide proof that scanning was conducted to ensure their data passes through service providers, potentially limiting open client options.
