What’s new in App Intents

Hi, my name is Kenny and I’m an engineer on the App Intents team.

App Intents has enabled you to bring amazing experiences to your apps with Shortcuts, Spotlight, Widgets and many other features including the Action Button and the new Apple Pencil squeeze.

And this year, App Intents is core to building experiences for Apple Intelligence and for Controls.

And the App Intents framework is providing new ways to help the system understand and use your app entities and making AppIntents even easier to use.

We have a lot to cover today, so let's dive in. We’ll cover how you can index your app entities to Spotlight, providing Siri with a rich understanding of your content.

Being able to convert your app entities to specific types of content or directly expose your documents as app entities.

Exposing your App Intent types as universal links so the device can take you right to where you need to be in your app.

And some developer experience improvements to make it even easier to work with App Intents. If you’re new to App Intents I’d recommend checking out the "Bring your app's core features to users with App Intents" video first.

Spotlight is a great place to find content that is relevant to you. Spotlight knows the apps I use frequently and offers them as suggestions. And based on those apps it offers quick access to common actions It might also show the current weather conditions or recent search results.

Spotlight also has powerful search functionality to help you find what you are looking for, like quickly finding a specific Note.

Or App Shortcuts in Top Hits, which enable you to easily expose useful features of your App through AppIntents.

This year we’re adding new APIs that allow you to index your app entities in Spotlight, bringing it’s new, powerful semantic search to app entities. Lets start with the new IndexedEntity protocol. CSSearchableIndex allows you to index information about your app to Spotlight using CSSearchableItem. Any CSSearchableItem has an attribute set to extend the information you provide.

The new IndexedEntity provides an easy way to index your app entities to CSSearchableIndex, while still giving you the ability to customize the attribute set. This allows your entities to show in Spotlight search results and helps Siri understand and find them.

My family and I love hiking, so I’ve been working on an app to keep track of the trails we like. It would be great if I could search for a trail in Spotlight without having to even open my app. To do this, I’m going to add support for IndexedEntity to my TrailEntity.

First, I’ll add conformance to the new IndexedEntity protocol to my TrailEntity.

Then in my app’s init method I’ll index all the Trail entities from my data manager by calling indexAppEntities on CSSearchableIndex. And that’s it. I’m now donating my TrailEntity content to Spotlight. Lets see it in action. First, I'll launch my app so my new entity indexing code has a chance to run.

Now I'll search for one of my trails in Spotlight. We can see that my trail content is showing up as a search result.

And because I have an OpenTrailIntent that accepts our TrailEntity as a parameter, I can tap on the result to go straight to it in the app. We’ll talk more about this later The default implementation of IndexedEntity only uses the DisplayRepresentation to populate the attribute set, but the attribute set allows you to specify all kinds of information that could be useful to Spotlight like location information or keywords. I’ll provide some more information to Spotlight by implementing attributeSet myself.

First I’ll provide an implementation of attributeSet on my TrailEntity.

I’ll customize the attributeSet to include some properties that might be relevant to my entity, like the city and state from the location of my trail.

And setting the keywords to the activities the trail supports.

Providing more information to Spotlight can help search and understanding return even more relevant results. And if your application is already indexing content in Spotlight via the CSSearchableItem API, you can use the new associateAppEntity method to associate an AppEntity with your searchable item before you index it. This allows the new semantic search to find information about your AppEntity.

You can also set a priority with any of the IndexedEntity indexing APIs. The larger the value, the more important the item. For example, you could use this to give a favorite item a higher priority than a non favorite.

On it's own, CSSearchableItem provides powerful search but it can’t take action on the content. By indexing your app entities, you can take advantage of your existing work on the entity and you will be able to help Siri find your entities and take action with them, like finding an email with your flight details and forwarding it to a friend Now let’s talk about how you can make entities meaningful to the device.

An AppEntity allows you to define and expose concepts from your app like a Trail.

But other apps can’t understand these concepts. Instead, I could represent my entity in a standardized format that any app can understand, like a PDF.

And use Uniform Type Identifiers to label the file or data with a type.

And there’s already an API available to help us do this. Transferable is a declarative way to describe how your models can be serialized and deserialized for sharing and data transfer. And now you can make your app entities Transferable.

For example, you could use Transferable to convert and export your AppEntity as a PDF, an image, and rich text. With Transferable, Siri and Shortcuts can convert your AppEntity and pass the new value to other AppIntents.

This could allow your content to be passed to Mail as an attachment, or converted to an image and imported into the Photo library.

My trails app allows me to track activities I’ve completed and represents them as an Activity Statistic Summary entity. I’d like to make it meaningful to other apps so I’ll add support for Transferable.

I’ll extend my entity with Transferable. And implement a static transferRepresentation method.

I already have a method to convert Activity Summary to rich text, so I’ll add a DataRepresentation, and specify RTF, or rich text, as the exportedContentType.

I also have a method to export my summary as a PNG file, so I’ll also add a FileRepresentation exported as a PNG.

The order that your transferable representations are defined is important. Provide them from highest fidelity to the lowest. For example, my type’s private Codable representation is first, followed by more lossy formats like rich text and then plain text. Let’s see this in action in Shortcuts. I already have a Shortcut that takes the output of my Summarize Activities action and appends the Calories Burned property to my Activity Log note.

Now that I’ve implemented Transferable, lets see what happens if I change the output to be the entity itself. After running the Shortcut, you can see that it converted the result of Summarize Activities to text and appended that to the note.

And if I change the type to Image and run the Shortcut again, we can see that it used my PNG representation instead. Look at that, I think I have a future on our design team. At this time there are some limitations to how you can use the Transferable API in combination with AppEntity, because Xcode needs to understand your Transferable representation at compile time. And will provide feedback if it is unable to do so. Additionally, ProxyRepresentation can only be used if it references properties of your AppEntity with the @Property attribute. So in this example I can use the name property because it has an @Property attribute.

But not description, because it does not have the attribute.

If you want to learn more about Transferable, I highly recommend Julia’s amazing “Meet Transferable” talk from WWDC22.

Now let’s talk about how to access content with improvements to IntentFile. We discussed how to convert an Entity to various content types with Transferable. Lets see how this works on the receiving side. Earlier, I showed appending my Activity Summary to a note. When an AppIntent like AppendToNote receives an IntentFile parameter, it can check what content types are available and request whichever types it needs. When it does this, App Intents will use the Transferable representation to convert the Entity into the requested content type. Here’s an example of what the AppendToNote might look like. Notice the attachment parameter declares what types of content it supports. This allows Siri and Shortcuts to automatically convert content when possible.

To access the content of the IntentFile you can use one of the new APIs for extracting content. You can check what type of content the IntentFile represents. And get access to the URL to do conversion. Before we move on, let’s talk about one more way to represent your content. The FileEntity API is great for document based apps or apps that manage files.

We discussed how Transferable allows you to convert your AppEntity to a file or data, which works great for cases where the Entity is representing an object from your database or server. What if your entity is itself a file, like a text document or an image? In these cases, the file is the canonical version of the entity.

I have PhotoEntity in another app. The PhotoEntity represents an image file on disk. While my app can understand and use the Entity in my AppIntents, like my SetFavoritePhoto intent, other apps can’t understand my entity or use it in their AppIntents. But as we just discussed, other apps can understand files and access them through IntentFile.

By using FileEntity, Siri and Shortcuts can facilitate secure access to your file in other apps, allowing them to directly access the file. For example, a RotateImageIntent in another app could securely access the file backing my PhotoEntity and rotate it in place.

Here’s an example of how you would implement FileEntity. It works like any other AppEntity with the additional requirement that you provide a list of supported content types.

And that the ID be a FileEntityIdentifier.

FileEntityIdentifier can be created with a URL, or if the file doesn’t exist yet as a draft identifier. The FileEntityIdentifier uses the URL’s bookmark data, so if the file is moved or renamed the entity will still be valid. My app displays my PhotoEntity items in a grid. I just noticed that the most recent two items I added, showing at the bottom of the grid, are oriented incorrectly. I'll fix that with a Shortcut. I'll run a query on my photo entities, filtering on images with a date added of today, and then use the built in Shortcuts rotation action to apply a 90 degree rotation to the result of the query. When I run the shortcut, it was able to rotate the images files in my app. If I go back to the app, we see that the file changes were detected and the UI updated.

These new APIs are powerful because they let your entities have meaning to other apps and let other apps’ entities have meaning to yours, opening up new possibilities with Siri and Shortcuts.

Next, lets talk about universal links and how you can represent them with your App Intent types. Universal Links help people access your content, whether or not they have your app installed. My Trails app already has support for a universal link to open a specific trail. Here’s an example URL that links to the details for a specific trail. The 1 indicates the identifier of the trail to be opened.

You can now express your AppEntity, AppEnum and AppIntent’s as having a URLRepresentation. This will allow Siri and Shortcuts to treat them like they are a link to specific content, allowing actions to open the URL or making them sharable. I want to be able to deep link to a specific Trail from Shortcuts, so I’ll extend my TrailEntity and add support for URLRepresentableEntity.

I’ll provide a static URLRepresentation using my universal link as the template.

Notice that I'm using the entity’s identifier as a interpolated value. You can use an entity’s ID or any of it’s properties with the @Property attribute as interpolations in the URL string.

Now that TrailEntity supports URLRepresentableEntity, lets look how I can use this with the new URLRepresentableIntent. I’ll create an OpenTrailIntent conforming to OpenIntent that targets TrailEntity.

And I’ll also add conformance to URLRepresentableIntent.

And that’s it. I don’t even have to implement a perform. App Intents will take care of invoking my existing URL handling code.

Let’s see what happens when I use this App Intent from Shortcuts. My daughter and I love biking, so I have a Shortcut that finds trails near a region and picks a random one for us to go ride.

But all it does right now is show the name of the trail. It would be even better if it could take me right to the Trail in my app, so let's add our OpenTrailIntent.

I'll add my Open Trail action to replace my Show Result action.

Now when I run the shortcut, the URL representation for the TrailEntity is used to deep link right to the trail details. Another way you can use URLs with App Intents is by returning the new OpenURLIntent from your perform. When your AppIntent is performed by Siri and Shortcuts they can open the provided URL to link to relevant content.

Or if I had a “CreateTrail” AppIntent, I could initialize an OpenURLIntent with a newly created TrailEntity allowing Shortcuts to open the app and navigate directly to the newly created Trail.

Adopting URLRepresentable makes it easy to provide universal links for your AppIntent types, reusing your existing URL handling code and helping people access your content. This year, we’re also bringing some improvements to the developer experience, making it easier to create AppIntents. We’ll talk about a new type of parameter, as well as some improvements to how you can define and reuse your AppIntents.

First up, UnionValue. Sometimes you have a parameter or a property that can be represented by one of a set of types. My Buy Day Pass AppIntent accepts a TrailEntity, but I would like to have an option to buy a day pass for either a single trail or for an entire park. I’ll use the new UnionValue macro to do this.

First, I’ll create an enumeration and define my two purchase options. One taking a ParkEntity and the other taking a TrailEntity.

And add the new UnionValue macro to my enumeration.

Now I can update my parameter to use my new Day Pass Type enumeration.

And in my perform method I can use a switch on the passType parameter, just like any other enumeration.

It’s important to note that each case in the enumeration must have exactly one associated value.

A case with no associated value is not valid.

And each associated value must be distinct. For example, I already have a case with a TrailEntity, so I cannot add another. Use UnionValue when you have some existing types and you want to take one of them. You could think of this as an “or” parameter.

If you’ve used AppIntents before you might have noticed that some of my AppIntent examples were missing a title in their @Parameter attributes. When building with Xcode 16.0 you no longer have to provide a title for your AppEntity Properties or AppIntent Parameters.

So my SuggestedTrails intent can be condensed from this, to this. Xcode will intelligently generate a title string for you based on the name of the struct’s property.

But in the case of TrailCollection, I want the title to display as “Featured Collection”.

So I’ll manually specify a title, just like before.

And last but not least, we’ve improved how you can use AppIntents in frameworks.

Before, all your App Intent types had to be in the same module. You couldn’t have an AppIntent in your app use an AppEntity defined in one of your frameworks. The AppIntent had to be in the framework too.

In Xcode 16, that limitation is gone allowing you to define app entities in a framework and reference them in your app and extension targets.

Only frameworks are supported at this time. Libraries outside of a framework are not.

These are just some of the great new features and improvements to AppIntents. To dive deeper, check out the AppIntents documentation.

Turn off 2x speed for a moment and let's wrap this up. Index your app entities in Spotlight, helping Siri find and understand things it never could before. Provide meaningful representations of your app entities, allowing other apps to understand your content. Try out URLRepresentable and our new developer improvements and let us know what you think. And AppIntents is powering many other features across the device. I recommend you checkout these other great videos to see some of the other exciting features using AppIntents.

We can’t wait to see how you intents-ify your apps. Thanks for watching.