Apple’s Siri Gets Major Upgrade with Screen Sensing and Deeper App Integration


Apple has recently announced a significant upgrade to its Siri virtual assistant, which promises to bring a more natural and smooth interactive experience and stronger practicality. This upgrade is made possible by the integration of the “Apple Intelligence” generative artificial intelligence model, which enhances Siri’s capabilities in various ways.

New Human-Computer Interface (UI)

One of the most noticeable changes in the new Siri is its new human-computer interface (UI). The assistant now presents responses to user commands in the form of content-rich cards, and soft light effects appear around the device frame. This new UI design provides a more intuitive and engaging experience for users.

Contextual Understanding and Screen Perception

The new Siri can also pay more attention to understanding the context of the conversation. This means that users do not need to repeat information in subsequent instructions, making the interaction more seamless and efficient. Additionally, Siri’s “screen perception” function allows it to understand the content that users are browsing and provide operation suggestions for related apps. For example, when a user needs to enter their driver’s license number when filling out a form, Siri can automatically find their electronic driver’s license and extract the relevant number to help them complete the form.

Deeper App Integration

The new Siri also supports direct input of commands, allowing users to interact with the assistant more naturally. Furthermore, the addition of the “App Commands” feature enables Siri to interact more deeply with Apple’s own and third-party apps. For instance, Apple demonstrated that the “Photos” app can edit the pictures the user is browsing, and Siri can directly complete the operation on behalf of the user.

Siri upgrade

Richer Product Knowledge Base

The new Siri also has a richer product knowledge base to help users solve device usage problems. For example, when a user asks “How do I write a message now and set a delayed delivery?”, Siri can directly provide a guide to the new “Scheduled Delivery” feature added to the iOS 18 Messages App. This feature allows users to schedule messages and the system will send the message later. This makes it easier to manage their communication.

Enhanced Siri Capabilities

One of the most notable upgrades to Siri is its ability to control individual app functions through voice commands. This feature is made possible by integrating large language models, a key technology powering generative AI. This means users can now manage specific features within various apps without having to physically interact with their devices. For example, users can ask Siri to open ebooks, set camera timers, manage slide titles, and even unsubscribe from email newsletters, making it easier to organize folders, tags, albums, and filters in the Notes and Photos apps.

Gizchina News of the week

Improved AI-Driven Tasks

Apple also enhances Siri’s AI-driven capabilities to perform a variety of tasks. These include summarizing emails and threads, generating smart replies, sorting emails into categories, transcribing text notes, editing photos, and summarizing both websites and notifications. This will make it easier for users to manage their digital lives, from organizing their emails to editing photos and creating summaries of online content.

Camera and Photos Enhancements

The Camera app has also seen significant upgrades, allowing users to control camera operations using voice commands. This includes starting or stopping video recording, opening the camera in various modes (such as photo, portrait, or video), beginning a shutter timer, and switching between the front and rear cameras. Additionally, the Photos app will introduce advanced photo editing tools using generative AI, similar to those on Google’s Pixel devices. Users will be able to remove objects from photos, apply AI-based filters, and locate specific photos of people or animals.

Siri upgrade

Siri will also see deeper integration with other apps, making it easier to adjust display brightness, home screens, and the app library. The main Settings app has a new design to provide more intuitive access to Siri’s features. This integration will enable users to manage their devices more efficiently, from adjusting display settings to organizing their apps and files.

Launch Timeline

Apple emphasized that new features will be launched in stages, and more powerful features will be added to Siri during the life cycle of iOS 18. While the new Siri will be gradually pushed to all users in the next year, it is expected to bring Siri’s capabilities closer to those of other advanced digital assistants like Gemini.

Future Developments

While the current release of iOS 18 focuses on enhancing Siri’s capabilities and user experience, future updates should further expand Siri’s functionality. For example, Apple is planning to use large language models to make Siri smarter, enabling it to perform multi-step tasks such as summarizing a recorded meeting and then texting it to a colleague in a single request.


In conclusion, Apple’s recent upgrade to Siri significantly enhances Siri’s capabilities. The upgrade makes it more natural, intuitive, and practical. The new Siri upgrade promises to bring a more natural and smooth interactive experience and stronger practicality to users. With its enhanced contextual understanding, screen perception, and deeper app integration, Siri is poised to become a more powerful and versatile virtual assistant.

The new UI, contextual understanding, and deeper app integration have streamlined user interactions, while the richer product knowledge base and enhanced AI-driven tasks have made it easier for users to manage their digital lives. The camera and photo enhancements have also opened up new possibilities for users to control camera operations and edit photos using voice commands. As the new Siri gradually rolls out to all users over the next year, it should bring Siri’s capabilities closer to those of other top digital assistants like Gemini.

Disclaimer: We may be compensated by some of the companies whose products we talk about, but our articles and reviews are always our honest opinions. For more details, you can check out our editorial guidelines and learn about how we use affiliate links.

Read more

Local News