Gemini Gains control: Google’s ‘Screen Automation’ Feature Promises AI-Powered Android Tasks
Table of Contents
Google is poised to significantly expand the capabilities of its Gemini AI model, granting it the ability to directly control functions within Android phones. Recent analysis of the Google app’s latest beta release reveals details about a forthcoming “screen automation” feature,codenamed “bonobo,” designed to allow Gemini to complete tasks such as ordering goods and booking transportation directly within compatible applications.
The facts comes from a deep dive into the latest version of the Google app uploaded to the Play store,a process known as “APK Insight” where developers decompile application files to uncover hints of future features. While Google may ultimately choose not to release these features, the revelation offers a compelling glimpse into the company’s AI-driven future for mobile.
Gemini’s “Bonobo” Project: Automating Tasks on Your Phone
the “Get tasks done with Gemini” feature, as it’s officially labeled, leverages what Google refers to as “screen automation.” This technology, built upon the foundation laid by Android 16 QPR3, will initially be available within “certain apps,” suggesting a phased rollout.According to internal strings, Gemini will be able to assist users with a variety of tasks, including placing orders and arranging rides, all through automated interaction with the phone’s interface.
Though, Google is proactively addressing potential concerns surrounding the reliability of an AI controlling a user’s device. A warning within the beta emphasizes that “Gemini can make mistakes,” and users will retain full obligation for the actions taken on their behalf. The system is designed to allow users to halt the process and regain manual control at any time.
Privacy Considerations with AI-driven Automation
The integration of Gemini’s screen automation also raises vital privacy questions. Google acknowledges this,stating that “When Gemini interacts with an app,screenshots are reviewed by trained reviewers and used to improve Google services if Keep Activity is on.” This data collection is intended to refine the AI’s performance, but users should be aware of the implications.
To mitigate risks, Google strongly advises against entering sensitive information, such as login credentials or payment details, into Gemini chats or relying on screen automation for critical situations. “avoid using screen automation for emergencies or tasks involving sensitive information,” the company cautions.
Beyond Automation: Exploring Digital Likeness with “Wasabi”
The beta also hints at another intriguing advancement: a “Likeness” feature, internally codenamed “wasabi.” This appears to be connected to the 3D avatars already utilized in Google meet calls, part of the Android XR ecosystem. One string suggests the ability to access and potentially utilize this digital likeness through a prompt, opening possibilities for personalized AI interactions. The strings indicate users will have control over their likeness, with options to “Retake” and assurances that “Your likeness can only be used by you.”
Google’s ongoing efforts to integrate Gemini more deeply into the Android experience signal a significant shift towards AI-powered mobile computing. While the “bonobo”
