Select Page

Extract

Siri shortcuts are a new feature introduced in iOS 12 that allow the user to trigger an action in your app via a “Hey Siri” phrase. Shortcuts can also be triggered by a tap on a shortcut from the lock screen, search screen, etc.

In a previous post we created a simple Siri shortcut demo app using an NSUserActivity-based approach. In this post we develop a more complex Intents-based app.

Download the demo app source from GitHubhttps://github.com/russell-archer/ShowRandomColor

 

What are Siri Shortcuts and how do I create them?

Siri shortcuts are a new feature introduced in iOS 12 that allow the user to trigger an action in your app via a “Hey Siri” phrase. Shortcuts can also be triggered by a tap on a shortcut shown from the lock screen, search screen, etc.

For example:

“Hey Siri, show me today’s events in MySportsApp”

Before proceeding, if you’re not familiar with creating Siri shortcuts and you haven’t yet read my previous post Siri Shortcuts using NSUserActivity, I suggest you look at that first for an overview of the subject, then return here for details on Intents.

 

Overview of how to implement Siri shortcuts

There are two approaches to creating Siri shortcuts:

NSUserActivity

  • Simple to implement
  • Open your app and trigger an action via a shortcut or Hey Siri phrase
  • Little customisation

Intents

  • More complex than NSUserActivity
  • You can customise the experience for the user
  • Open an app extension in Siri via a Hey Siri phrase
  • Include custom responses from Siri and a custom UI that Siri will present

In this post we’ll concentrate on the Intents-based approach.

 

Overview of the demo app

We’ll create a demo app that allows the user to request a random color. The color can be displayed either:
  

  • In the ShowRandomColor app, or
  • By tapping on a Siri shortcut (color is displayed in Siri via an app extension), or
  • Via a Hey Siri phrase (color is displayed in Siri via an app extension)

Setup

Create a new single-view iOS app named ShowRandomColor:

Select the app target and the Capabilities tab, then turn on support for Siri. This adds an .entitlements file to your project and allows you to use to Siri SDK:

Because we want Siri to trigger an action that will run outside of our app (i.e. in Siri) we need to add both an Intents Extension and an Intents UI Extension. These will be added as new targets in the project.
  
Select File > New > Target and choose Intents Extension:
Name the new Intent target ColorIntent
Ensure Include UI Extension is checked and that the Starting Point value is set to None
 
By default Starting Point is set to Messaging. If you don’t set this to None you get a lot of message-oriented code not relevant to our color-related intent:
You will be prompted to see if you want to Activate the new targets. 
Click Activate for both ColorIntent and ColorIntentUI:
You should now see that ColorIntent and ColorIntentUI have been added as targets:

  

Defining a Siri Intent

Add a SiriKit Intent Definition File to the main app target. 
  
Select File > New > File, choose SiriKit Intent Definition File and name the file Intents:
With the Intents file selected, click the + to add a New Intent:
The Custom Intent (.intentdefinition) file will be used by Xcode to generate a protocol and helper code that will enable us to correctly handle input from, and responses to the shortcut. 
 
Configure the input properties of the Intent as follows:

1. Custom Intents

Make sure the Intent is selected, not the Response. This allows us to define the form of the shortcut.

2. Custom Intent

Category. Select View which best describes what type of action our shortcut represents.

Title. Type a title that will appear on the shortcut.
Description. Provide a description for the shortcut.
Default Image. Select one of the images added to the asset catalog. This will appear on the shortcut.
Confirmation. Don’t check this as our action isn’t of the type that the user will need to confirm (e.g. a purchase, deleting an item, etc.).

3. Background

Make sure that Supports background execution is checked. This allows the intent to run outside of the app, hosted in Siri.

Now select the Response part of the custom intent. This defines how Siri will respond when our shortcut is run. 

In this case we want Siri to respond with something like “Here’s the color colorName”

1. Response
Defines how Siri will respond when the shortcut is run.

2. Properties
Define one or more properties (variables) that can be used in the various types of response.
Add a colorName property of type String. This will be our app’s description of the random color displayed to the user

3. Response Templates
Create two response templates, one for failure and one for success. Notice how both templates use the colorName property.

Now switch back to the Intent view of our custom intent:

Look in the File inspector pane. Make sure that all three targets are checked and that Intent Classes is selected.
This will ensure that the code generate by Xcode from the contents of our intent definition file will be available to all targets:
 
With the Intents definition file still selected, open the the Identity inspector pane. 
Note the class name for the Xcode-generated Custom Class is currently IntentIntent:
Name the Intent ShowRandomColor. Xcode will then rename the custom intent class ShowRandomColorIntent:

Build the project. Xcode will generate the following:

public class ShowRandomColorIntent: INIntent
Encapsulates the user’s request

public class ShowRandomColorIntentResponse: INIntentResponse
Encapsulates the app extension’s response

public protocol ShowRandomColorIntentHandling: NSObjectProtocol
Implement this protocol in the app extension to confirm the intent and handle the request

public enum ShowRandomColorIntentResponseCode: Int
Enum used for response codes (success, failure, etc.)

To view the source generated by Xcode (with the intent definition file still selected) select the Identity inspector.

Click the reveal arrow next to Custom Class Name:

The generated code in ShowRandomColorIntent.swift is stored in Xcode’s DerivedData directory. 
You can find the location for this by right-clicking the filename in the navigation bar:

This shows the directory where the generated code is stored:

Normally there’s no need to directly access the generated code as Xcode will re-generate it as and when required.
Note also that ShowRandomColorIntent.swift is not added to source control because it’s regenerated from the .intentdefinition file as required.
  

Create the app’s UI

We can now create the (extremely simple) UI in the main app.

In Interface Builder, add a UIView and then add constraints so it fills the containing view (within the safe area).
Set its background to some random color:

Create an outlet for the UIView in the associated ViewController:

We now need a means of getting a random color so we can set colorView.backgroundColor to that color. We need to make use of this in all of the following situations:

 

  • When the app starts we set a random color during viewDidLoad(), or
  • When the user taps on our Siri Shortcut which invokes our app extension to display a random color, or
  • When the user activates our app extension with a Hey Siri phrase we display a random color

We need to design our app so that code (see Frameworks below) and data (see App Groups below) can be shared between the app, the app extension, and the app extension UI. So, for example, when a random color is generated by (say) the app extension we want the app extension UI to be able to see the value of that random color. 

 

This mean seem a trivial requirement, but isn’t straightforward. To see why, let’s review a little bit about how app extensions work.
  

App extensions

App extensions allow you to host portions (extensions) of your app in other apps. For example, with Siri shortcuts you can get Siri to host an app extension and a custom UI extension.

Even though an app extension bundle is physically delivered inside the containing app’s bundle, when executing an app extension and the containing app have no access to each other’s processes or containers (a container may be regarded as a sandboxed file system). Normally, the containing app is not even running when the app extension is running.

A containing app and its extension (and extension UI) can’t share common variables, other data or files as their processes and sandboxed containers are isolated from each other:

  

App Groups

If you want to share data between a containing app and its app extensions and app extension UI the solution is to use an App Group.

An app group creates a secure, shared container that multiple processes can access:

The shared container can be used to exchange data using either UserDefaults or by directly working with files using FileManager. In both cases you need to use the app group identifier.

 
 

Adding an App Group to the project

Creating an app group in Xcode is simple. Select the root project node in Project Navigator:
Select the first target which needs to share data (i.e. the main app). 
  
On the Capabilities tab turn on App Groups support.
Click the + to add a new app group and then supply an id in the form group.company.com.app-name.Shared (e.g. group.com.rarcher.ShowRandomColor.Shared):

Now select the next target which needs to share data and turn on app groups support. This time you’ll be able to select the the previously created group:

Repeat this process until all targets that require it have app groups support turned on. We can now use UserDefaults to share data (a random color value) between the app, the app extension, and the app extension UI as required. 

 

Frameworks

We’ve seen how we can use App Groups to share data, but what about sharing code between the app, app extension and app extension UI?

The easiest (but definitely not the best) method is to create a ColorGenerator class and copy its .swift file into all three modules. Obviously, this is not a great way to share and reuse code. First, you have three copies of the code. If you need to make changes you always have to remember to make the exact same change to each copy of the class. If not, things get out of sync and you can end up with some very hard to find bugs and inconsistencies. Plus, it’s a lot more work to make changes (three times as much).

The best solution is to use a FrameworkExamples of common frameworks are Foundation, UIKit, AVFoundationCloudKit, etc.
  

 

Adding a Framework to our project

Select File > New > Target and choose the Cocoa Touch Framework template:

Name the framework ColorGenerator:
Name the framework ColorGenerator:
Select the root of the project in Project Navigator, then select the ShowRandomColor target.
 
In the General tab, you can see that ColorGenerator framework has been added as both an Embedded Binary and a Linked Framework:
Now add the framework to the ColorIntent app extension.
Select the ColorIntent target and click the + to add a framework:
Repeat the previous set add the framework to the ColorIntentUI target:

  

Add code to the framework

Select the ColorGenerator framework and add a new Swift file named Generate.
Add the following code:
The operation of both methods is very straightforward, with randomColor() returning a tuple containing a UIColor and the name of the color generated. 
 
The rgb(colorName:) method takes a color name and returns the equivalent RGB value as a tuple.
 
Notice that we’ve marked the class and the two methods as public.
  
 
  

Using the ColorGenerator framework in the main app

We can now use the ColorGenerator framework to set the colorView.backgroundColor to a random color during viewDidLoad() processing:
Notice how we import the ColorGenerator framework.
 
Running the app in the simulator produces a random color:

We also get a couple of warnings:

We’ll have to fix the root of this issue or it would be rejected on submission to the App Store. 
 
 

Frameworks in App Extensions

If you attempt to use a framework in an app extension you’ll get a warning:
 
linking against a dylib which is not safe for use in application extensions
 
 This because there are some APIs Apple says you should not access in an app extension. For example, you cant access the shared UIApplication object, nor can you access the camera or microphone on an iOS device.
 
Apple’s documentation states: 

To configure an app extension target to use an embedded framework, set the target’s “Require Only App-Extension-Safe API” build setting to Yes. If you don’t, Xcode reminds you to do so by displaying the warning “linking against dylib not safe for use in application extensions”.

More details about APIs forbidden in app extensions are available here:
https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/ExtensionOverview.html#//apple_ref/doc/uid/TP40014214-CH2-SW6

To remove the warnings we need to rebuild the framework with the Require Only App-Extension-Safe API build setting set to Yes:

Note that the app extension and app extension UI should already have their Require Only App-Extension-Safe API build settings set to Yes.
 
  
 

Donating the Intent in the main app

Apple states that you should “donate” your shortcut to iOS every time the user performs the specific action associated with the shortcut (either through normal interaction with your app, or through the shortcut. The donation of your shortcut allows Siri to build a usage pattern for how the user interacts with your app. If a particular action is performed a number of times then Siri may offer the user a shortcut to that action (if the app has defined one) on the lock screen, search screen or in Siri Settings.  
  
 
The first place we need to donate our shortcut is in viewDidLoad() in the main app’s ViewController:
Note that in donateIntent() we create a ShowRandomColorIntent object. This class, which derives from INIntent, was generated for us by Xcode from the Siri intent definition file. We set a suggestion for a Siri trigger phrase, although the user can use something entirely different if they choose when they record a trigger phrase to launch our shortcut. An INInteraction object is created to wrap up the intent and donated to iOS.  
   
  
 

Adding the app extension code

In the ColorIntent app extension create a new Swift file and name it ColorIntentHandler.swift.  
  
 
Add the following code:
This class implements the ShowRandomColorIntentHandling protocol, which was generated by Xcode from the Siri intent definition file.
  
If you look at the ShowRandomColorIntentHandling protocol you’ll see that the code comments state that the confirm(intent:completion:) method is:

Called prior to asking the app to handle the intent. The app should return a response object that contains additional information about the intent, which may be relevant for the system to show the user prior to handling. If unimplemented, the system will assume the intent is valid, and will assume there is no additional information relevant to this intent.  

The method is marked as optional in the protocol, so it’s not actually necessary to implement it. In our simple demonstration case we just signal to iOS that we’re ready to handle the shortcut. In production code we might check to see if (for example) the resources required to handle the shortcut were available, and if not return a failure code.
 
The handle(intent:completion:) method handles the actual shortcut. It gets a random color and then saves the name of the color to UserDefaults in our shared container. This is so that the app extension UI (see below) has access to the color that was generated. We save the color name, rather than the actual UIColor object because, without additional work, a UIColor object can’t be persisted to a UserDefaults plist.
 
Still in the ColorIntent app extension, open the IntentHandler.swift file. This file was generated for us when we added the Intents Extension (app extension) target earlier. The handler(for:) method handles all intents by default.
  
Because we need custom behaviour for the ShowRandomColorIntent intent we can create an instance of our ColorIntentHandler class. Amend handler(for:) as follows:

Defining the app extension UI

We can now define the UI that will be hosted in Siri when our shortcut is triggered.
  
In ColourIntentUI, open the MainInterface storyboard and add a UILabel and UIView.
Set the view’s background color to some random default and then add constraints for the label and view:
Open Xcode’s Assistant Editor and add outlets in IntentViewController for the label and view:

  

Adding the app extension UI code

Import the ColorGenerator module and then modify the configureView(for:of:interactiveBehavior:content:completion:) method as follows:

  

Record a Hey Siri phrase

We’re now ready to test out our shortcut. Build and run the app.
  
A random color is displayed via the viewDidLoad() method in the main app’s ViewController. Once that happens we know that the shortcut has been donated to iOS.
  
Now goto Settings > Siri & SearchYou should see our “Show a random color” shortcut is available – tap it:
You can now record a Hey Siri trigger phrase. Notice how our suggested trigger phrase is displayed.
  
Tap the red record button:
Record your trigger phrase (I used our suggested “Random color”):

Tap the record button to finish:

You can now see in Settings > Siri & Search that the phrase “Random color” is able to trigger our “Show a random color” shortcut:
Now goto the home screen and say “Hey Siri Random color”.
  
You should see that our app extension is hosted in the Siri process and that a random color and its name is displayed.
Siri also say’s “Show random color says here’s the color yellow” (or whatever color was generated):

  

Add support for the user tapping our shortcut

We’re almost done. However, the user can also trigger our shortcut by tapping on our shortcut. When this happens the method application(_:continue:restorationHandler:) in our AppDelegate class is called to handle the shortcut.
  
Add the method as follows. It simply calls our app’s main ViewController to display a random color and donate the shortcut:
Build and run the app. If you now do a search you’ll see our shortcut is available:

Tapping on the shortcut foregrounds the app and displays a random color:

  

Conclusion

We’ve seen how incredibly useful and powerful Siri Intents-based shortcuts are. Unlike the more simple NSUserActivity-based approach, an Intents-based approach allows Siri to actually host your app extension.
  
However, as I said in a previous post, Siri shortcuts are not quite as flexible and powerful as they first appear to promise!
Initially I thought I’d be able to be support actions like:
 
“Hey Siri, ShowColor green“
  
Where green is a “voice parameter” to be passed to the target app ShowColor.
 
Sadly, (unless I’ve misunderstood how this works) you can’t yet do this sort of thing. It seems like all the support is in-place to support passing parameters, but Siri isn’t actually capable of it.
  
The parameters section in the intents definition file only (as far as I can see) allows you to define specific variations for a shortcut. And each variation is itself a specific shortcut. So, if you wanted to be able to display all the colours of the rainbow you’d need to define shortcuts for “ShowColor red”, “ShowColor green”, “ShowColor blue”, etc.
 
I’m sure support for parameters of this sort will come in iOS 13, but for now you are limited to to the creation of very specific actions which are then made accessible by a shortcut.