WWDC 2019 Session Notes

Usually it is much faster to read through some bullet points instead of watching a 50 min session video. Then if you find something interesting you can still watch it.

Sessions that are exceptionally mentionable are highlighted using a ★.

This is work in progress since it is a lot of effort to watch all the videos by myself. So either please be patient or just open up an issue to make a suggestion which session notes you like to see here asap :)

Contribution

Feel free to submit a PR if I got something wrong or you have an improvement suggestion. Also have a quick look in CONTRIBUTING.md if you want to contribute.

Thanks so much to EVERYBODY who contributed and improved the overall quality of the notes and those who added complete notes to the list:

@matthew_spear, @rukano, @Borzoo, @viktorasl, @ezefranca, @0xflotus, @lachlanjc, @Sherlouk, @serralvo, @Gerriet, @soucolline, @DmIvanov, @tommy60703, @speedoholic, h0shy, @sidharthshah, @sgrgrsn, @mcichecki, @wacumov

Mentions

This repo has been already mentioned in the following places:

Interesting WWDC-Related Links

Apple Developer Documentation by @Apple
Xcode Release Notes (overwhelming)
iOS & iPadOS Release Notes
WWDC 2019 - The Things You May Have Missed by @BalestraPatrick

Video ToC

Platforms State of the Union ★
What's New in Swift ★
What's New In App Store Connect
What's New in Xcode 11
What's New in Authentication
What's New in Safari
What's New in Safari Extensions
What's New in iOS Design
What's New in Universal Links
What's new in Machine Learning
Introducing SF Symbols
Introducing Sign In with Apple
Introducing Core Haptics
Introducing Multi-Camera Capture for iOS
Introducing Combine
Introducing ARKit 3
Introducing RealityKit and Reality Composer
Introducing the Indoor Maps Program
Introducing Accelerate for Swift
Introducing Desktop-class Browsing on iPad
Introducing SwiftUI: Building Your First App
Introducing iPad Apps for Mac
Introducing Parameters for Shortcuts
Introducing SiriKit Media Intents
Advances in Foundation
Great Developer Habits ★
Writing Great Accessibility Labels
Building Great Shortcuts
Designing Great Shortcuts
Integrating with Siri Event Suggestions
Modern Swift API Design
Core NFC Enhancements
Testing in Xcode ★
Advances in Networking, Part 1
Advances in Networking, Part 2
Getting Started with Xcode
Modernizing Your UI for iOS 13 ★
Designing Audio-Haptic Experiences
Expanding the Sensory Experience with Core Haptics
Cryptography and Your Apps
All About Notarization
Implementing Dark Mode on iOS
Advances in Speech Recognition
Optimizing App Launch
Accessibility Inspector
Visual Design and Accessibility
HLS Authoring for AirPlay 2 Video
AUv3 Extensions User Presets
Game Center Player Identifiers
Text Recognition in Vision Framework
Understanding Images in Vision
Advances in UI Data Sources
Data Flow Through SwiftUI
Binary Frameworks in Swift
Creating Great Localized Experiences with Xcode 11
Getting the Most Out of Simulator
SwiftUI on watchOS
Core ML 3 Framework
Debugging in Xcode 11
Accessibility in SwiftUI
Designing Great ML Experiences
Creating an Accessible Reading Experience
Create ML for Object Detection and Sound Classification
Auditing Web Content with Web Inspector
Embedding and Sharing Visually Rich Links
Combine in Practice ★
Adopting Swift Packages in Xcode
Creating Swift Packages
What’s New in MapKit and MapKit JS
App Distribution – From Ad-hoc to Enterprise
(ToDo) Designing iPad Apps for Mac
(ToDo) Adding Indoor Maps to your App and Website
(ToDo) Advances in CarPlay Systems
(ToDo) Advances in Collection View Layout
(ToDo) Advances in Natural Language Framework
(ToDo) Advances in macOS Security
(ToDo) Architecting Your App for Multiple Windows
(ToDo) Building Activity Classification Models in Create ML
(ToDo) Building Custom Views with SwiftUI
(ToDo) Creating Great Apps Using Core ML and ARKit
(ToDo) Creating Independent Watch Apps
(ToDo) Designing for Privacy
(ToDo) Drawing Classification and One-Shot Object Detection in Turi Create
(ToDo) Exploring New Data Representations in HealthKit
(ToDo) Exploring Tinted Graphic Complications
(ToDo) Extended Runtime for watchOS Apps
(ToDo) Font Management and Text Scaling
(ToDo) Integrating SwiftUI
(ToDo) Introducing Multiple Windows on iPad
(ToDo) Introducing PencilKit
(ToDo) Introducing the Create ML App
(ToDo) Large Content Viewer- Ensuring Readability for Everyone
(ToDo) Making Apps More Accessible With Custom Actions
(ToDo) Making Apps with Core Data
(ToDo) Mastering Xcode Previews
(ToDo) Metal for Machine Learning
(ToDo) Network Extensions for the Modern Mac
(ToDo) ResearchKit and CareKit Reimagined
(ToDo) Streaming Audio on watchOS 6
(ToDo) Supporting Dark Mode in Your Web Content
(ToDo) Supporting New Game Controllers
(ToDo) SwiftUI Essentials
(ToDo) SwiftUI On All Devices
(ToDo) System Extensions and DriverKit
(ToDo) Taking iPad Apps for Mac to the Next Level
(ToDo) Targeting Content with Multiple Windows
(ToDo) Training Object Detection Models in Create ML
(ToDo) Training Recommendation Models in Create ML
(ToDo) Training Sound Classification Models in Create ML
(ToDo) Training Text Classifiers in Create ML
(ToDo) Understanding CPU Usage with Web Inspector
(ToDo) Using Core Data With CloudKit
(ToDo) What's New for Web Developers
(ToDo) What's New in Apple File Systems
(ToDo) What's New in Core Bluetooth
(ToDo) What's New in Core Location
(ToDo) What’s New in AppKit for macOS
(ToDo) What’s New in ClassKit
(ToDo) What’s New in File Management and Quick Look
(ToDo) What’s New in Managing Apple Devices
(ToDo) Window Management in Your Multitasking App
(ToDo) Advances in AR Quick Look
(ToDo) Bringing OpenGL Apps to Metal
(ToDo) Bringing People into AR
(ToDo) Building AR Experiences with Reality Composer
(ToDo) Building Apps with RealityKit
(ToDo) Building Collaborative AR Experiences
(ToDo) Delivering Optimized Metal Apps and Games
(ToDo) Metal for Pro Apps
(ToDo) Modern Rendering with Metal
(ToDo) Ray Tracing with Metal
(ToDo) Working with USD
(ToDo) Advances in Camera Capture & Photo Segmentation

Platforms State of the Union ★

https://developer.apple.com/wwdc19/103

App Store requirement to adapt to all screen sizes from Spring 2020
tvOS gets Multi-User support plus support for Xbox and Playstation game controllers
SwiftUI
- Less Code. Better Code. Everywhere. Declarative Syntax.
- Round images by Image(name).clipShape(Circle()).shadow(radius: 3, y: 3)
- CornerRadius by Image(name).cornerRadius(3)
- Creating a label by Text("Done!").font(.headline).color(.gray)
- Changing label to button by Button("Done!", action: done).font(.headline).color(.gray)
- Automatic Spacing & Insets, Localizability, Right-To-Left, Dynamic Type, Dark Mode, ...
- Animations are Repsonsive, Interruptable, Automatic with minimum amount of code
- Build powerful layouts through composition leveraging VStack and HStack
- Adopt BindableObject for model objects that can change
- Live Development Experience
- Adopt PreviewProvider for views/view controllers (surrounded by #if DEBUG) to live-preview in canvas
- Multiple previews with different configurations displayable on one canvas
- Adopt SwiftUI gradually at your own pace
Swift
- Introduction of Module Stability which ensures compatibility for binaries of current/future versions of Swift compiler
- GitHub adds Swift Package support to the GitHub Package Registry
- Swift Packages for apps on all platforms seamlessly supported by Xcode
Xcode 11
- Add additional editors whenever and whereever you want
- Use related content (Live View, Assistants, ...) in any editor of your workspace. They disappear automatically when there is no content.
- Minimap enables straigt-forward code navigation. Highlights MARK declarations, Issues, Test Failures, ...
- Add Documentation feature auto-adds missing function parameters
- Refactoring also changes documenatation parameters
- Source-control history moves to the Inspector pane on the right side. Can now always be open.
- Editor displays inline code-change comparison
- TestPlans enable the developer to create many different test combinations (localizations, environment settings, sanitizers, devices). Works with Xcode Server!
- Enable device conditions in Xcode: Network Link Conditioner, Thermal State
- New app performance metrics for released apps: Battery Usage, Launch Time, Hang Rate, Memory Use, Disk Access
- TestFlight user feedback lets users share screenshots and comments
macOS Catalina
- Bring your apps to the Mac by just enable one checkbox in Xcode
- Move kernel extensions out of the kernel into user space with DriverKit which improves macOS stability for all users
- System volume will become read only. User data and apps are on a read/write volume.
- New permissions for apps that want to do Keystroke Recording, Screen Capture, Screen Recording
- New permissions for apps that want to access directories like Documents, Desktop, Downloads, iCloud Drive, Removable Media, Network Volumes
watchOS
- The watch becomes independent from the phone by running its own apps, by just checking a checkbox in Xcode
- Independent apps are highlighted on the watchOS App Store
- App Store has fully featured app descriptions. Apps are searchable by dictation and scribble. Direct download of the new, small bundles.
- Notifications, CloudKit, Complication Pushes, Text Fields, Sign In With Apple
- Streaming audio to the watch
- SwiftUI support
iOS
- New semantic colors that automatically update for the brand new Dark Mode
- New set of adaptive materials and vibrant content filters with variable transparency which support ligh/dark content mode
- Modal view controllers are now displayed as cards by default
- Peek and pop redesigned to show previews with contextual actions
- SFSymbols are 1500+ free, vector assets, variable scales and weights
- Re-designed share sheet
iPadOS
- Major enhancements to multi tasking by new UIWindowScene API
- New SceneDelegate to manage multiple windows at the same time from one app
- New state restauration system using NSUserActivity
- PencilKit framework for customize drawing in your app
- Major simplification in text selection. UITextInteraction API to fix conflicting-gesture issues
- ScrollViewScrubbing: Grab and drag the any scrollview scroll indicator
Accessibility Features
- Accessibility has been added to the iOS Quick Start guide
- Accessibility has been moved one hierarchy level up into the Settings main menu
- Accessibility sub-menus menu got icons for better discoverability
- Voice Over can display a grid for better selectability. You can show accessibility labels and much much more. It's an incredible feature!
- Showing accessibility labels via Voice Over completely changes how we can test them. They are now just displayed on screen!
Privacy
- Good Practice: Process user data on device whenever you can & Ask for permission when processing useer data & Encrypt user data
- Allow once is a new option when requesting location permission
- Apps location usage shown on a map by apple from time to time
- Sign in with Apple provides fast, easy sign in without tracking by tapping a Sign in with Apple Button and a quick FaceID
  - No more email address verification since apple already done this!
  - All AppleID-emails are protected by 2FA
  - Built in fraud detection
  - Apple provided email relays protect users real email address
  - Cross platform support: Works on Android and Windows devices (on the web)
  - No tracking by Apple
Machine Learning
- Happens on-device
- Engine is capable of 5 trillion operations per second
- Image Saliency provides image heatmap
  - Highlights important objects and where users are likely to focus
  - Helps auto-cropping photos
- Text Recognition
  - Search text from signs, posters and documents
  - Take advantage of the document camera capability from the Notes app
- On-Device speech API with support for 10 languages
- CoreML supports more than 100 layer-model types which brings e.g. breakthrough natural language processing to your apps
- Core ML On-Device Personalization enables developers to update models from inside the app (in the background)
- Create ML becomes a macOS app enabling users creating models without writing code
  - Choose from different model templates
  - Build multiple models with different data sets and define parameters of each
  - Real-time feedback on model training
  - Supports transfer learning (e.g. for image classification) which speeds up training
  - Experiment and preview models, e.g. use microphone to test sound classification models
Siri Shortcuts
- Make them discoverable by adding the Add to Siri button to your apps
- Apple made them conversational by adding Parameters
- Shortcuts app is build into iOS and iPadOS
- Added Automation which allows to set triggers for when to run specific shortcuts
- Configurable 3rd-party actions enable combination of multiple actions (multi-step shortcuts)
- Synced via iCloud to all of your devices
ARKit
- Integration of AR Quick Look with Apple Pay to try out clothes, glasses, and other products (and immediately by them)
- Worlds largest AR platform
- 3 technologies that make it easy to develop AR experiences: ARKit, RealityKit, Reality Composer
- Reality Composer lets you layout AR experiences in 3D on macOS, iOS and iPadOs
- RealityKit is a modern real-time 3D engine. Designed for AR. Easy integration with existing apps.
  - Uses physically based rendering, Data driven rendergraph, Multi-threaded
  - Access through native Swift API framework
- ARKit 3
  - Simultaneous from and back camera
  - Motion capture by ML algorithm
  - People occlusion detection by ML algorithm
Metal
- Modern high performance (100x OpenGL) GPU programming. Low overhead. Easy to use.
- Core layer under all of Apples frameworks: UIKit, AppKit, Core Animation, SpriteKit, RealityKit, ARKit, MapKit, WebKit, Core ML, Create ML, VisionKit, Core Image, Camera Processing
- Full support in iOS Simulator brings major performance improvements
- Metal Compute provides building blocks for general purpose computation: Shading Language, Compiler, Compute Encoding, Metal Performance Shaders,
- Metal Tile Shading combines compute shaders and fragment part into one highly-efficient render pass
- Metal Indirect Compute Command Encoding lets you build GPU compute commands right on the GPU
- Amazing Metal-based, animated Raytracing demo with computationally expensive multi-reflections and more
- "Built for Pros" 😅

What's New in Swift ★

https://developer.apple.com/wwdc19/402

ABI Stability (Application Binary Interface): App and Framework built by different compilers can now be used together (in the past ensured by building with same compiler)
Module Stability: Swift module file and App build by different compilers can now be used together by introducing stable and textual .swiftinterface file
Binary Frameworks: Result of ABI and Module stability. Can be shared with others.
obsolete return if it is the only line in a closure / function
auto-generated struct initializers now allow to pass only some of the arguments
Opaque Return Types: Let you return the same type without leaking implementation details. Use keyword some
Property Wrappers: Wrapper type to define custom access patterns. Property can adopt this by adding attribute to its declaration. In the case of UserDefaults you can define properties that know how to read/write from/to UserDefaults.
Embedded DSLs

What's New In App Store Connect

https://developer.apple.com/wwdc19/301

App Store for Messages with full TestFlight capabilities
Transporter Mac app for submission of IPA, PKG, ITMSP files. ApplicationLoader deprecated.
Build Activity View shows all uploaded builds, download sizes for each device, detailed information for each build
TestFlight Feedback
- lets users take screenshots in TestFlight and send them with a description to developer
- accessible/downloadable from ASC
- adds a custom onboarding screen to the app when it is tested via TestFlight and opened the first time.
- can be enabled/disabled via testflight for each tester group
Crashes automatically trigger the feedback dialog. Feedback viewable/downloadable in ASC.
App Deletions 🎉 as metric on APC. Opt-in. Only from iOS 12.3+. Only for homescreen and storage deletions. Resetting device doesn't count.
24h live dashboard hourly updated statistics for all apps to better understand sales data (e.g. in-app purchase) as it is rolled out

What's New in Xcode 11

https://developer.apple.com/wwdc19/401

Editor splitting is now unlimited and allowed horizontally and vertically
Source control log info moved to inspector and can thus be used for any file any time
Editor Options Button configures now every split editor seperately.
- Assistent and Authors are moved there
- Holding ⌥ + ⇧ when clicking a new file shows the destination selector that lets you choose in which split editor to open the file
Editor and Canvas option shows the current views preview if there is one
Canvas / Assistent hides automatically if there is nothing to see for the currently selected file
The Minimap is loaded with functionality
- Landmarks, Syntax coloring, Navigation, shows symbol names on hover, highlights search results
- Holding ⌘ shows all symbols in the file when hovering over the minimap
Powered up Documentation formats the doc block in a markdown like style
- add missing documentation / or fill up gaps by selecting Editor > Structure > Add Documentation
Show Change after click on change bar shows inline diff
Code completion works more reliably: function overloads, enum cases, compiler control statements
Swift Package Manager is now part of Xcode
- GitHub, GitLab and BitBucket integration
- used to integrate packages for all platforms
- Project Editor > Project > Swift Packages Tab shows all personal and starred repos when logged in
- Source browsing of packages is possible
- helps sharing own packages with the world
Source Control extended by stashing and cherry pick
Asset Catalog can provide assets and colors for dark mode. Image assets are localizable now.
Environment Overrides in the debug bar lets you change dynamic text size, appearance and accessibility settings
Device Conditions in the devices window let you change network throughput and thermal state of your device
Test Plans define a set of tests and are sharable via Schemes
- describe arguments, environment variables, sanitizers, language, ...
Watch Simulator is now stand alone
Simulator build on top of Metal
- apps build in Metal can be run in the simulator with great performance
- Apps on top of UIKit are much faster
- 60 fps • up to 90% less battery life • warm boots are 2x as fast
SwiftUI enables Canvas previews, Code and Canvas Editors, Code Hot Swapping, Rich Preview API, Preview Pinning, Instruments Template, On-Device Previews, Action Popover Actions, Library Views and Modifiers, Canvas Editor for Code, Preview Debugging, Development Time Assets
Most Awesome SwiftUI Tutorials I've Ever Seen Right In Xcode's Documentation Window

What's New in Authentication

https://developer.apple.com/wwdc19/516

Sign in With Apple
- lets users choose which information is delivered
- creates random email and forwards it directly to your AppleID to disguise your original email
Password Auto-Fill for iPad apps on the Mac
Weak Password Assistant detects weak passwords and offers you to change it
ASWebAuthenticationSession provides easy OAuth workflow and deprecates SFAuthenticationSession

What's New in Safari

https://developer.apple.com/wwdc19/515

Desktop-class website browsing
Legacy Safari Extension Support dropped for sake of Content Blockers, Share Extensions, Safari App Extensions
Safari Extensions
- deliver bundles with app OR after notarization via web site
- Get the visible content of the web page (screenshot)
- show and dismiss popovers
- delegate informs about navigating/redirect to new site
Content Blocker
- associate content blocker with Safari Extension to get notified when content is blocked
Universal Links for macOS so ordinary https links open app if installed

What's New in Safari Extensions

https://developer.apple.com/wwdc19/720

Distributable via
- App on App Store (show up immediately in Safari)
- App on your web site after running through the notarization service (show up in Safari after first launch of the app)
Unsigned Extensions must be allowed each time Safari is run from the Developer menu
ContentBlocker can now tell Safari App Extensions bout its activity
Page Navigation Delegate informs about navigating/redirect to new site
Adopting ContentBlocker and PageNavigation extensions allows
- replacing arbitrary content on websites
- updating toolbar icon badge, e.g. with content items blocked count
- blocking certain content from web sites you browse on
Screenshots of web sites are now possible
Tab, Window, Page management
- Get base URI from native code without injection of script
- Directly navigate to certain tabs without scripting from the extension
- All open tabs and windows are visible from an extension
Popovers can be shown/dismissed programmatically
Communication between App and Extension
- Possible by NSXPCConnection (extension and app must be part of the same app group)
- Share data by using UserDefaults(suiteName:)
- Sending messages from app >>> extension: SFSafariApplication.dispatchMessage
- Receive message in extension by implementing SFSafariExtensionHandling.messageReceivedFromContainingApp (Possible also when safari is not running - launched eventually)

What's New in iOS Design

https://developer.apple.com/wwdc19/808

Dark Mode
- all apps should support Dark Mode
- adopt iOS design system to maintain familarity, provide platform consistency, convey clear information hierarchy
- always use semantic colors like backgroundColor, highlightColor, ... instead of red, green, ...
- color hierarchy: Title > Subtitle > Placeholder > Disabled Text
- color hierarchy naming: background, secondaryBackground, tertiaryBackground, quaternaryBackground, ...
- don't just use color inversion between light/dark mode
- avoid alpha values since it might look broken when different colors overlap
- check colors with online contrast calculator should give 4.5...1 or higher
- layered interfaces should use drop shadow in light / elevated bg color in dark mode
- all controls are drawn with semantic colors - perfectly blend between light/dark
Card Style modal presentation style
- default - opt-out by viewController.modalPresentationStyle = .fullScreen
- pull-down to dismiss (can be prevented when there is a mandatory decision in the modal)
- Modals are for switching modes. Don't use them because you like the animation
Contextual Menus
- activate by tap & hold
- puts the focus on actions not on the preview (like peek & pop)
- available on any device since it doesn't rely on force touch
- consist on a list of actions and an optional previews
- all actions should be available somewhere else in the UI. Don't rely on the fact that users discover your menu.

What's New in Universal Links

https://developer.apple.com/wwdc19/717

macOS 10.15 supports Universal Links with AppKit / UIKit
Securely associated between your app & website
recommended over custom URL schemes
Webserver Config
- HTTPS certificate mandatory. Custom root certificates not supported
- Add the file at https://yourDomain.com/.well-known/apple-app-site-association
- URLs and pattern matching are ASCII
App Config
- add Associated Domains capability and add applinks:www.example.com entries (patterns possible - specific subdomains prioritized)
- userActivity.activityType == NSUserActivityTypeBrowsingWeb in func application(_ application: UIApplication, continue userActivity: NSUserActivity, ...) indicates Universal Link
- get URL components URLComponents(url: userActivity.webpageURL, resolvingAgainstBaseURL: true)
For macOS Apps
- app must be on local volume
- App Store distribution recommended
- Developer ID signed apps must be launched first

What's new in Machine Learning

https://developer.apple.com/wwdc19/209

Presented by Gaurav Kapoor
Apple Goals: easy, flexible, powerful - Machine Learning for Everyone
Many Domain APIs
- Vision - Vision | Apple Developer Documentation
  - Image Saliency: most interesting image area
  - Text Recognition: OCR, Document Camera
- Natural Language
  - Sentiment Analysis
  - Word Embeddings
- Speech and Sound
- Combine Domains, for example semantic search in images
Core ML 3
- on device, all platforms, hardware accelerated
- model flexibility: layer types, conversions from TensorFlow and later ONNX
- model personalization: fine-tuning on device (improved privacy) possible as background processing

Introducing SF Symbols

https://developer.apple.com/wwdc19/206

Alignment is perfect surrounding text
1500 vector symbols designed by Apple in all weights available
Margins of symbols can be different so they are perfectly aligned on screen
Scales (small, medium, large) ensure
- symbols maintain same weight as neighbouring text
- symbols are vertically centered to neighbouring text
SF Symbols app to play around and export existing symbols (as SVG template) for customization
UIImage(systemName: String) is new - just copy the name from the Symbols app
Multiple images with the same name in Asset Catalog are now possible. Apple first looks for symbol and then for non-symbol (ensures backwards compatibility to iOS 12: uses image in iOS 12 and symbol in iOS 13)
Constraining symbol images is discouraged. Use natural size or set font size
Any font can be used with symbols via UIImage.symbolConfiguration
Vertical text alignment works with single-lined and multi-lined text
Baseline alignment to align symbol to first line of text
Regular images are aligned baseline aligned by generating a new image with image.withBaselineOffsetFromBottom(GGFloat)
Prefer horizontal and vertical center alignment instead of edge alignment
Buttons with symbol images are created using UIButton.system(image:target:action)
UIButton Symbol Configuration changed by UIButton.setPreferredSymbolConfiguration(config:forState:)
Auto-scaling UIBarButtonItems from SymbolScale.large to SymbolScale.medium in landscape
No frame settings necessary for NSTextAttachment for placing images in text
UIImage tint support image.withTintColor(UIColor) - yeah 🥳

Introducing Sign In with Apple

https://developer.apple.com/wwdc19/706

Secure - backed by 2FA of the AppleID
Private - not tracked by Apple
Fast and Easy - user doesn't even need a keyboard
User controls which data to share
Seamless Across Device recognizes that already signed in on other device
Requires Capability for Apple Sign In has to be added
Private E-Mail Relay links random mail to your AppleID
- Apple never retains messages
- Can be used for any email communication like receipts, ...
- Two-Way Relay
Anti-Fraud Detection can tell if a robot tries to sign in or not
Cross Platform iOS, macOS, watchOS, tvOS, JavaScript (Android Websites)
ASAuthorizationAppleIDButton creates button that has different visual styles and labels
Authentication Request returns
- UserID which is unique, stable and team-scoped and can be used as the key to the user
- Verification Data identity token and short-lived code to refresh token
- Full Name as PersonNameComponents which contain first/last name separately
- Verified email your server doesn't need to verify this email again
- Real User Indicator high confidence indicator that likely real user
- Credential State tells if UserID is authorized (let user pass), revoked (handle unlink) or not Found (show login)
Always check on AppStart with provider.getCredentialState(userID) which runs very fast
Listen to NSNotification.Name.ASAuthorizationAppleIDProviderCredentialRevoked and sign the user out if called
Password Autofill integrates with Apple Sign In. Triggered if the device detects a stored credential

Introducing Core Haptics

https://developer.apple.com/wwdc19/520

Event based audio-/haptic rendering API - a synthesizer
same feel on iPhone 8, 8 Plus, X, XS, XR, XS Max
Not a replacement for UIFeedbackGenerator! Continue using it for impact, selection and notification
Use Core Haptics for custom haptics and audio patterns
Content Classes: CHHapticPattern, CHHapticEvengt, CHHapticParameter
Playback Classes: CHHapticEngine, CHHapticPatternPlayer, CHHapticAdvancedPatternPlayer
Haptics can be coded inside the app or delivered as AHAP (Apple Haptic Audio Pattern) resource file
- AHAP describes pattern as text
- schema for JSON
- Can use Swift Codable
- Separate content from code
Read more at the updated Human Interface Guidelines (HIG) for haptics

Introducing Multi-Camera Capture for iOS

https://developer.apple.com/wwdc19/249

Supported on iPhone XS, XS Max, XR, iPad Pro 3rd Gen
AVCaptureMultiCamSession
- Multiple AVCaptureDeiceInputs, AVCaptureDeiceOutputs of the same type, AVCaptureVideoPreviewLayers
- Don't use implicit connection forming but use addInputWithNoConnections, addOutputWithNoConnections or AVCaptureVideoPreviewLayer.setSessionWithNoConnection
AVCaptureSession is still the way to got for single cam session
- Simultaneous photo shooting, movie recording, barcode scanning, etc.
Limitiations
- only one input per camera in a session
- connecting one camera to multiple video data outouts is not possible
- no presets supported on session since different cams might run with different qualities
- multi-cam session has hardwareCost reporting. Session runnable when 0 <= cost <= 1
- lower cost by lower resolution, choose binned format, deviceInput.videoMinFrameDurationOverride = CMTimeMake(1, 30) to set max framrate override /* 30 FPS */
- lower system pressure like temperature, power demands, infrared projector temperature by
  - lowering frame rates, throttle GPU/CPU processing code, disable one camera
- run indefinitely with multiSession.systemPressureCost < 1.0; device shutdown with cost > 3.0
- iOS can run only one session at a time (with mutli cams though)
Virtual Camera is the new name wor software cameras like True-Tone- or Dual-Camera
device.isVirtualDevice - get its physical devices by device.constituentDevices for e.g. synchronized camera streaming
AVCaptureDataOutputSynchronizer ensures you get two outputs in one callback
virtual devices have secret ports so you can get 2 streams - you need to explicitly query them
Dual Camera Hography Aids
- CameraIntrinsics (Optical center / focal length) and CameraExtrinsics (rotation matrix / translation vector for both wide- and tele cameras)
Multi-Microphone capture
- By default front cam uses front mic and back cam uses back mic
- actually front mic and back mic are a lie since different devices have multiple mics but not explicitly fron/back ones. This is achieved by Microphone Beam Forming - done automatically by CoreAudio micInput.ports(for: .audio, sourceDeviceType: micDevice.deviceType, sourceDevicePosition: .front).first
- Beam forming only works with built-in mics
- audio can be arbitrarily configured by creating custom AVAudioSession

Introducing Combine

https://developer.apple.com/wwdc19/722

Combine is a unified, declarative API for processing values over time
- Generic
- Type safe
- Composition safe
- Request driven
Key concepts
- Publisher
  - Defines how values and errors are produced
  - Value type (struct)
  - Allows registration of a Subscriber
- Subscriber
  - Receives value
  - Reference type (class)
- Operator
  - Adopts Publisher
  - Describes a behavior for changing values
  - Subscribes to a Publisher (“upstream”)
  - Sends result to a Subscriber (“downstream”)
  - Value type
Relations
- Subscriber is attached to Publisher
- Publisher sends a Subscription
- Subscriber requests N values
- Publisher sends N values or less
- Publisher sends completion
- The value sent from the Publisher should be the same type as the one expected from the Subscriber. In case of a mismatch you need to create an Operator to transform the value.
Combining Publishers
- Zip
- CombineLatest
Zip
- Converts several inputs into a single tuple
- A "when/and" operation
- Requires input from all to proceed
Combine Latest
- Converts several inputs into a single value
- A "when/or" operation
- Requires input from any to proceed
- Stores last value
Examples of usage
- Process a NotificationCenter post with filter
- Await completion of two network requests with zip
- decode the data of a URLResponse

Introducing ARKit 3

https://developer.apple.com/wwdc19/604

Andreas Moeller, Thomas Berton

Review of modern ARKit apps
Three pillars of ARKit
- Tracking: where is the device in relation to environment; tracking world, faces, images.
- Scene Understanding: identify surfaces, images, 3D objects
- Rendering: SceneKit, SpriteKit, Metal and new RealityKit
New: People occlusion (available on A12 or later)
- People correctly occlude rendered objects that they are in front of
- Based on segmenting people and depth estimation
- Works for multiple people and for people only partially in the scene
- Use frameSemantics in ARConfiguration with .personSegmentation or .personSegmentationWithDepth or directly access segmentationBuffer in ARFrame
- Example in code
New: Motion Capture (available on A12 or later)
- Track the body of a person, enables transferring to a virtual character in real time
- Provides skeleton representation (in 2D and 3D)
- .frameSemantics-Option .bodyDetection
- Joints are named (e.g. rightFoot, leftShoulder, …) and a definition gives the hierarchy
- ARBodyTrackingConfiguration
- 3D: additional ARBodyAnchor with estimatedScaleFactor and transform
- 3D skeleton has more joints
- Example application for animating 3D characters. Needs a rigged mesh
- BodyTrackedEntity
New: Simultaneous Front- and Back-Camera (available on A12 or later)
- userFaceTrackingEnabled in ARFaceTrackingConfiguration
Collaborative Sessions
- In ARKit 2 only one-time map-sharing between devices
- New: continiously share mapping information and ARAnchors
- Maps will be integrated internally
- setupMultipeerConnectivity(), isCollaborationEnabled , sending ARCollaborationData, using session()-Callbacks for sending and receiving
AR Coaching UI - guiding the user is important and can be difficult
- Additional to HIG now AR Coaching View: overlay to guide users for good tracking experiences
- On-Boarding and guide with different consistent overlays
- Add as child of ARView, connect to session, optional delegates and specification of coaching goals
Multi Face tracking
- Up to 3 faces simultaneously
- Persistent face anchor IDs
New: ARPositionalTrackingConfiguration for low power consumption tracking only
Scene Understanding Improvements
- Up to 100 images
- Detect scale
- Image quality at runtime
- Faster and more robust object detection
- Plane estimation improved (more accurate, faster): door and window additional to wall, floor, ceiling, table, seat
Raycasting: new API
- not only vertical/horizontal
- tracked over time
- RaycastQuery
Visual coherence enhancements
- Depth of Field
- Motion Blur
- HDR environment textures
- Camera Grain
Record and replay with Reality composer app to use in Xcode - improved developer experience

Introducing RealityKit and Reality Composer

https://developer.apple.com/wwdc19/603

Cody White, Tyler Casella

RealityKit is a new AR-first Swift framework for realistic rendering and simulation (iOS and macOS)
Reality Composer is a macOS and iOS tool for simple AR-based content creation
RealityKit takes care of
- Rendering: physical-based shading, built with Metal
- Animation supports skeletal animations, transform animations and motion capture
- Physics
- Synchronization: multi-device
- Entity-Component System: composition vs. inheritance
- Audio: map audio to objects in 3D
- Reality File: optimized content in one file - faster uploading
Basics of RealityKit
- ARView sets up the environment, gesture support, realistic camera effects
  - shadows, motion blur, depth of field, camera noise
- Entity: building block of every AR object, establishes scene structure, provides transform hierarchy
- Hierarchy: ARView > Scene > Anchor > Entity
Entities and Components: entities made up of individual components
- AnchorEntity attaches to real world objects, tracks target
- ModelEntity represents visual parts of a scene (loaded directly from used or reality file or directly from code)
  - contains mesh resource
  - contains Materials (SimpleMaterial, UnlitMaterial, OcclusionMaterial)
- Animation
Reality Composer to get started with AR and 3D (download here)
- provides content library, layout
- Pre-Visualization (with or without AR)
- Simple Interactions (add behavior)
- Separate app, highly integrated with Xcode: Swift-file auto-generated for Reality File

Introducing the Indoor Maps Program

https://developer.apple.com/wwdc19/245

Enables public places to use indoor maps
Uses a format called “Indoor Mapping Data Format” (IMDF) which is represented as a set of GeoJSON files.
IMDF conforms to RFC 7959.
The building owner has to join the “Indoor Maps Program” and produce the IMDF.
The IMDF map has to be created with a professional GIS or BIM tool.
Anyone can create an IMDF map and display it an app or website using MapKit or MapKit JS.
The Indoor Maps Program and indoor positioning is only available for large properties with more than 5 million annual visitors.
Public building owners can choose to publish their indoor maps to the official Apple Maps.
Apple provides an IMDF sandbox where IMDF maps can be validated and tested in a browser based interface. This sandbox is available for all Apple Developers and not only members of the Indoor Maps Program. See the demo of the sandbox at 15:30.
Indoor positioning:
- Uses WiFi fingerprinting
- Expected accuracy: 3-5 meters
- The indoor location on the indoor map can be obtained via CoreLocation.
- To set up indoor positioning an indoor survey needs to be done. This is done by using the “Indoor Survey App” to collect WiFi info in the building, which is then uploaded for analysis and activation. The app is also used for testing the accuracy of the indoor positioning. See the demo of the app at 22:32.
The content of an IMDF map:
- Building Footprint
- Levels
- Units (room, walkaway, elevator etc.)
- Openings (doors)
- Kiosks (typically in the walkaway)
- Labels and icons
- Sections (highlights areas on the map eg. a food court in a mall)

Introducing Accelerate for Swift

https://developer.apple.com/wwdc19/718

Accelerate provides thousands of low-level math primitives running on CPU to support image/signal processing, vector arithmetic, linear algebra and machine learning
provides excellent performance combined with energy savings on all platforms
vDSP: Digital signal processing functions
- Vector and matrix arithmetic, e.g. 4D-Vector multiplication
- Type conversion, e.g. converting floting point to integer values
- Fourier transforms, e.g. forward and reverse transforms very simplified and convenient
- Biquadratic filtering reduces the necessary code from ~50 to some ~5 lines of code. Appllying the filter is now a single function call.
- Convolution and correlation
vForce: arithmetic and transcendental functions
- Arithmetic functions: floor, ceil, abs, remainder, ...
- Exponential and logarithmic functions: exp, log, ...
- Trigonometric functions: sin, cos, tan, ...
- Hyperbolic functions: sinh, asinh, ...
- Example: calculate sqrt for each element of large array - up to 10x faster in contrast to for loop
Quadrature (area under a curve): Numerical integration functions
vImage: Image-processing functions
- Core Graphics / Video interoperability
- Capable of alpha blending, format conversions, histogram operations, convolution, geometry, morphology
- Flags are now Swift OptionSet
- Throws proper Swift errors
- Enums for pixel formats and buffer types
- Hides requirements for unmanaged types and mutable buffers
- Moves free functions to properties on buffers and formats
LINPACK Benchmark (built on top of BLAS)
- measuring peformance in GFLOPS on iPhone XS: 1.8 without vs. 44.75 with Accelerate
- comparing Eigen to Accelerate using SGEMM on iPhone XS: 51 vs. 122
Use Acelerate since it is tuned to the platform

Introducing Desktop-class Browsing on iPad

https://developer.apple.com/wwdc19/203

New Features: Download Manager, Keyboard Shortcuts, Full Page Zoom, ...
Websites designed to show lots of information, e.g. GitHub, are zoomed to make use of iPad's full width
Hovering with the mouse on desktop now feels natural on an iPad
Speaker covers the topics Link following, Web Browser, Hybrid Apps and Authentication for WKWebView apps
Prevent horizontal scrolling in WKWebView: Compile with iOS 13 SDK and set webViewConfiguration.applicationNameForUserAgent = "Version/1.0 MyBrowser/1.0" (user agent auto-completed by Web Kit)
Set preferredContentMode = WKWebpagePreferences.ContentMode.[recommended, mobile, desktop]
Demo: "Shiny Browser" iPad App
- it is now easier to implement toggles between mobile/desktop websites by leveraging WKWebViewDelegate
Optimization potential for web developers
- Pointer Events
  - Adopt by if (window.PointerEvent) element.addEventListener("pointermove", updateInteraction); else element.addEventListener("mousemove", updateInteraction);
  - Cancel default web browser behavior by event.preventDefault(); and additional the CSS variant touch-action: none; (more granualar on disabling events)
- Hover
  - Provide alternative way to access hover content
  - Avoid two tap for common interactions
  - Keep hover snappy
- Accelerated Scrolling
  - works on all frames, not only the main frame
  - Frameworks don't need workarounds to simulate acceleration of sub frames anymore
- View Port and Text Sizing
  - Some webpages are build in fixed width, wider than an iPad
  - Now on iPadOS the paremeter width=device-width is ignored for websites laying out wider than iPad width. The site is zoomed to max iPad width.
  - Add content parameter shrink-to-fit=no to the viewport meta tag to prevent auto shrinking
  - Implement proper responsive websites to get around the new iPadOS auto adjustments
- Visual Viewport API
  - Visual VP is the area of your webpage visible to the user, e.g. between top and top of the iPadOS keyboard
  - react to changes, e.g. keyboard will show, by visualViewport.addEventListener("resize", visualSizeChanged);
- Streaming Video
  - Media Source Extensions now available on iPad
Best Practices
- Build one responsive website instead of two for mobile/desktop
- Use feature detection instead of switching on user agent
- Don't use Flash - support will be dropped in Safari on all platforms
- Let users decide if they want audio
- Remember that some desktop browsers don't have mice
- Use built-in APIs

Introducing SwiftUI: Building Your First App

https://developer.apple.com/wwdc19/204

SwiftUI merges the benefits of Storyboards (new preview canvas) and code
Drag and Drop of UI elements in the canvas also updates your code
⌘ + Click On UI Element in Canvas lets you edit properties of that element which will be reflected in code.
Modifiers are the functions that modify an UI element: .font(), .foregroundColor()
Conform to Identifiable to make a struct usable as element in List
Objects Conforming to PreviewProvider are a way to display test data in the preview canvas
NavigationView is the replacement for UINavigationController and shows a navigation bar
Wrap List Content in NavigationButton to make cells selectable.
- Provide the new view hierarchy as destination in NavigationButton(destination:)
Delete Cells by appending a .onDelete(perform: action) at the end of your List views
Move Cells by addpending a .onMove(perform: action) at the end of your List views
Use ExtractView on a selected code block to automatically create a new view struct and make your code modular
Achieve Small Navigation Bar Titles by .navigationBarTitle(Text("..."), displayMode: .inline)
Views are structs that conform to View protocol
@State variables are used to make your views react on changes by re-execute the view's body function
- SwiftUI observes when @State variables are read/written
- Are sometimes referenced to as Source of Truth - while other variables are just called Derived variables
- Further described in Data Flow Through SwiftUI
SwiftUI eleminating UI inconsistencies (interesting description at 24:45)
Animations are added by e.g. .tapAction { withAnimation { self.zoomed.toggle } } to toggle between zoomed in/oput state
- Animations are interactive and interruptible
Use flexible frames to get screen-filling Image view with image appearing centered in the middle of the screen: Image().resizable().aspectRatio(.fit).frame(minWidth: 0, maxWidth: .infinity, minHeight: 0, maxHeight: .infinity)
Use Combine to make your data dynamic, e.g. create your own object and let SwiftUI listen for changes to it (see little example at 38:26)
Preview Environmental Changes like Dynamic Font by appending .environment(\.sizeCategory, .extraExtraExtraLarge) to your preview UI element.
- Change color scheme .environment(\.colorScheme, .dark)
- Change layout direction .environment(\.layoutDirection, .rightToLeft) and \.locale
- Change locale .environment(\.locale, Locale(identifier: "ar"))

Introducing iPad Apps for Mac

When to consider iPad app for Mac
- Make iPad app available on the Mac
- Bring older Mac app to feature parity with iPad app
- Replace Mac app built on non-native technology
Not all apps are candidates
- iPhone only app
- Apps that are built around mobile features (e.g. ARKit)
Goals
- Easy to get started
- One project, one source base, one target
- iPad app on the inside
- Mac app on the outside
Things you get for free
- Default menu bar
- Window management (resize, full screen, split view and Sidecar support)
- Dark mode
- Scroll bars
- Settings
- Touch bar (basic layout)
- Open and Save panels
- Text sizes (iOS 13 pt, mac 17 pt) scaled
- Copy and past
- Drag and drop
- Printing
- Multiple windows
- Application lifecycle
Features to adopt
- Mac icon
- Custom menus
- Toolbars
- Touch bar (if you want to add custom layout and icons)
- Hover events
- Help
Deprecated frameworks
- AddressBook -> Contact
- AddressBookUI -> ContactsUI
- AssetsLibrary -> Photos
- OpenGLES -> Metal
- GLKit -> MetalKit
iOS Specific Frameworks
- ClassKit
- HealthKit
- HomeKit
Hardware Specific frameworks
- ARKit
- CarPlay
- VisionKit
Framework Differences
- Core Location
- Core Motion
- Core Telephony
- Core NFC
- Core Bluetooth
- AVFoundation
API Availability
- Annotations are used to indicate availability for every platform
Conditionalizing code
- Use #if targetEnvironment() to conditionalize code
Bundle Format
- As bundle format are not the same on iOS and MacOS don’t hardcode bundle path, use NSBundle to handle the work automatically

Introducing Parameters for Shortcuts

https://developer.apple.com/wwdc19/213

Shortcuts can be used via Suggestions (at the right time), Voice (to use any capabilities of your app just by asking Siri) and Multi Step Shortcuts (by composing multiple shortcuts in a new one)
Support for conversational shortcuts which lets users customize your Shortcut behavior
Shortcuts app redesigned and build in to iOS
- Adds Automation tab which lets users run shortcuts on certain conditions (Sunset, NFC-Tag, Before I leave for work, ...)
- Gallery with hundreds of pre-build Shortcuts - also from 3rd party apps
- 3rd party Shortcuts can provide outputs and thus be connected to other Shortcuts
Let users discover Shortcut capabilities by showing the Add to Siri button at appropriate stages
Redesigned Add to Siri sheet where users can enter the activation phrase and tweak the Shortcut parameters
Shortcut Customization
- In the Intent Editor you can
  - Create Shortcuts your app offers to your users
  - Customize an intents parameters even with custom types (in iOS 13)
  - Control if parameters are exposed to the Shortcuts app
Parameter Resolution
- Resolve (prior Confirm and Handle) as newly added Shortcut lifecycle step
- Siri goes through each parameter until it knows what to fill in for each
- After resolution Siri invokes the same confirm and handle methods as in iOS 12
- Resolution result can be: .needsValue, .disambiguation, .unsupported, .confirmationRequired, .success, .notRequired
  - User-facing message for each of them can be customized in intent editor
Related Parameters
- Express parameter relationships in the Intent Editor > Section "Shortcuts App"
- Developers have to
  - Identify parent and child parameters
  - Establish parameter relationships
  - Update summaries for each parameter combination
Dynamic Options
- Check the Dynamic Options checkbox in Intent Editor
- Provide the supported values and default values from delegate functions
Outputs
- Specify your own type that contains the properties you want to pass on to other actions
- Define a new property of your output type and designate it to be an output

Introducing SiriKit Media Intents

https://developer.apple.com/wwdc19/207

Possible Phrases For 3rd Party Apps
- "Tell that I love pop music"
- "Play Khalid on "
- "I don't like this song"
- "Add this to my library"
Intents
- INPlayMediaIntent to allow playing audio
- INAddMediaIntent to add media items to playlists and libraries
- INUpdateMediaAffinitiyIntent to express affinity to media items
- INSearchForMediaIntent to search for specific media in your app
- Support for playback controls let users say "Play Billie Eilish shuffled in <MyApp>"
- Supported audio types: Music, Podcasts, Audiobooks, Radio
- Even search for unsupported media ypes possible - caveat: search queries will be untyped
Handling SiriKit Media Requests
- Handling requests through Siri app extension
- Don't forget to add the Siri capability to your app
- Request processing involves Resolve > Confirm > Handle
- See 10:15 or an extensive example about Adding intents to your app, Specify supported intents and media types, Implement resolve, handle for INPlayMediaIntent and INAddMediaIntent
Best Practices
- If Siri support already added:
  - The new API uses existing code for handling background app launch
  - You need to add resolve methods
  - You need to update intents extension with supported media types
- Apple Watch
  - Foreground app launch via INPLayMediaIntentResponseCode.continueInApp
  - Intent is handled by your WKExtensionDelegate
  - Prefer on-device cache in your resolve method - only use network if absoloutey necessary
- Process results in resolve method case insesnitive becaue Siri might give you upper case results
- Write an effective search method - be flexible since
  - Siri might understand certain media types, e.g. video if video is part of what the user searches
  - Siri might understand sun or son
- Always populate title, artist and typein the returned INMediaItem since they all influence Siri's output
- Handle error cases gracefully
  - Most common error is not found: INPlayMediaMediaItemResolutionResult(InMediaItemResolutionResult.unsupported())
  - List of possible errors in INPlayMediaMediaItemUnsupportedReason
- When you support playback controls in your app also support them in Siri
  - Play <Song> on repeat in <MyApp>
  - Play <Playlist> on shuffle in <MyApp>
  - Resume <Podcast> at double speed in <MyApp>
  - Play <Artist> in <MyApp> next|later
- User says Play <MyApp>
  - Don't ask what to play!
  - Choose something interesting automatically or resume the queue
- User vocabulary helps Siri recognize important named entities
- Global vocabulary is appropriate for global app terms

Advances in Foundation

https://developer.apple.com/wwdc19/723

Ordered Collection Diffing
Compression API to transmit resources compressed (CompressionAlgorithm.[lzfse, lz4, lzma, .zlib])
New Units bits, bytes, nibbles
RelativeDateTimeFormatter for get localized, relative date descriptions
ListFormatter formats list contents localized
- correct separator and localized and before the last element
- specify itemFormatter to control how the list items are formatted
OperationQueue
- BarrierBlock lets run a completion task safely when all other tasks are finished
- progress reporting
Scanner lets you split string files in one line now: scanner.scanUpToCharacters(CharacterSet)
Use DataProtocol instead of UInt8

Great Developer Habits ★

https://developer.apple.com/wwdc19/239

Use functional groups for project organization. Group related information.
Mirror project and file structure
Large Storyboards should be splitted
Update Project Settings as soon as possible whenever Xcode prompts you
New Build System should be used asap
Unused code should be deleted, not commented out. We all use source control, right?
Warnings should be resolved immediately. Never check in code with warnings. Treat them as errors.
Commit small changes often with a short title and a descriptive body.
Use branches to isolate changes that belong to a certain feature.
Write documentation about WHY the code has been written and WHAT it is doing
Use descriptive variable names instead of id, state, etc.
Write Unit Tests to avoid regressions and run them before each commit. Keep tests running swiftly so you can execute them often.
Use diagnostics tools that can be enabled in the scheme settings (Main Thread Checker, Address Sanitizer, Thread Sanitizer, Undefined Behavior Sanitizer)
Gauges are very useful to see if you have problems with network, memory, disk and CPU usage
Network Link Conditioner helps tracking down bugs on low connectivity
Do Code Reviews to track down issues and get a common understanding of the code and its style
- understand each changed line
- build and run the project
- run the tests
- proofread comments and documentation
- look for spelling and grammar errors
- ensure consistency in codebase
Decouple your code by creating packages and shared frameworks to reduce code size and share code between multiple app targets
Dependencies should be used responsibly
- Understand how a dependency works
- Read carefully the changes when it is updated
- Have a plan when a dependency goes away or is not maintained anymore
- Ensure privacy

Writing Great Accessibility Labels

https://developer.apple.com/wwdc19/254

Accessibility Label is a human understandable label
- gives meaning to the elements of your app
- value depends completely on the context
- should not contain the type of the view. Automatically provided by accessibilityType
- should contain context if multiple similar elements visible (multiple add buttons)
don't add redundant context, e.g. if you're in a music player avoid song from the accessibility label of next, previous, ...

Building Great Shortcuts

https://developer.apple.com/wwdc19/805

Summary of an Action should be short and only contain necessary parameter
Best way to add shortcut is right inside the app after the action that should be added has been completed
Offer to add shortcuts for repeatable actions
Activation Phrase should be short and pronounceable
Suggested Shortcuts List can be updated by app via API a often as wanted. Apple also populated this list based on device usage like recently used apps.
Input and Output concept used to make actions work together. Actions can now output information for others to use, e.g. action could find a note wheas action 2 processes this. Both can be chained together.
Intent Editor lets the developer modify intents

Designing Great Shortcuts

https://developer.apple.com/wwdc19/806

Shortcut Use Cases are Accelerate an Action, Present Concise Information, Multi-Step Shortcuts
Great Shortcuts wrap actions the user has to to often in your app. Offer adding an action to Siri after this action has been completed
Provided Add to Siri Button should be used to add an action to Siri
Activation Phrase should have a great default
Action should provide as much detail as possible to the user has to fill out as less as necessary
Prompts can be used to collect a value from the user.
Minimize Disambiguation Prompts by providing an options list upfront
Pronounciation Hints can be provided on devices lacking a display
Providing Synonyms to options helps Siri to understand the user
The Final Confirmation Dialog should be used to summarize everything the user has specified. Keep in mind to design a more descriptive one for devices without display!
Continuing in App dialog is one big button, therefore avoid elements that look interactive.
Never include the App Name this is shown automatically
Don't Include Users Name since it might sound repetitive
Avoid first-person pronouns use something like There are the following options

Integrating with Siri Event Suggestions

https://developer.apple.com/wwdc19/243

Apps can donate information to the system via INInteraction so the system can show them to the user at appropriate time, location or in related apps
All this magic happened directly on the device
Intents framework supports reservations acroll many categories like Flights, Restaurant, Movies, Ticket Events, Car Rentals, Train, Hotels, Flights
INReservation, INGetReservationDetailsIntent and INGetReservationDetailsIntentResponse to donate reservation information to the system
NSUserActivity is used to handle app launches from Siri suggestion throughout the iOS system
- can be used to provide web-based flow if app is not installed > NSUserActivity.webpageURL
Shortcut appears on the lockscreen and in the search UI when specified parameters match (user is at specified location, time of the event is close, …). On tap it launches the app.
Reservation Details are automatically synced across the users devices using end-to-end encryption
Donating Reservations
- If you don't have the coordinate provide 0, 0. Siri will only use the postal address.
- Make sure to set the correct time zone of the event location for the event
- If no end time is available - set it nil
- In case of multiple INReservation items:
  - provide itemReference which must be unique for each reservation
  - can be structured as you want, e.g. reservation number combined with flight number
- Donation should be done
  - in-app when the reservation is created
  - always when reservation viewed because it might have changed
  - from the background when the app is closed and the reservation changed
- Donate should not be done
  - explicitly via UI elements
- If reservation is cancelled, donate the reservation with reservationStatus set to .cancelled

Modern Swift API Design

https://developer.apple.com/wwdc19/415

Clarity at point of use is the most important goal as an API designer
No prefixes in Swift-only frameworks
Prefer Structs over Classes - Use classes only when
- reference counting or deinitialization is needed
- the value is shared
Use private backing variables and additionally a computed property for reference types if you need to copy them
Only manually copy reference types if isKnownUniquelyReferenced returns false.
Don't start off with protocols rather start with concrete types • discover the need for generic code • try to use existing protocols • consider generics over protocols. This will also decrease binary size and increase compile time.
Get familiar with SIMD types which are perfectly suited for geometric calculations
Dynamic Member Lookup to define a single subscript operation that exposes multiple properties on a type
- attribute type by @dynamicMemberLookup
- requires implementing subscript that takes a KeyPath or ReferenceWritableKeyPath
- any property is automatically exposed as computed property
Property Wrappers offer code reuse out of computed properties by using Generics.
- Describe the policy behind your data access
- Eliminate boilerplate & get more expressive UIs
- add the attribute @propertyWrapper to your generic type (requires implementing public var value: Value )
- Use Your Property Wrapper by defining a property this way: @YourPropertyWrapperType var name: String which will internally expand to a private backing property as well as a computed property that uses the implementation of your generic type.
- @UserDefault(key: "BOOSTER_IGNITED", defaultValue: false) static var isBoosterIgnited: Bool
- @ThreadSpecific var localPool: MemoryPool
- @Option(shorthand: "m", documentation: "Minimum Value", defaultValue: 0) var minimum: Int
- Examples from SwiftUI are @State, @Binding
Use protocols for code reuse not for classification

Core NFC Enhancements

https://developer.apple.com/wwdc19/715

Support for reading Passports, interacting with smart cards from iPhone 7 ++
NFCTagReaderSession allows to scan native tags like ISO14443, ISO15693, ISO18092
NFCNDEFReaderSession supports discovery of NDEF tag objects and read/write operations for those
Workflow to Use NFC
- Enable Xcode Capability
- Use one of the sessions described above
- receive tags in discovery callbacks
- connect to tag
- perform operations
- invalidate session
Native Tag Reading supports ISO7816
- info.plist must contain a list of ApplicationIdentifiers (AID)
- detect-tag callback invoked when tag is ISO7816 and AID is contains in info.plist
Native Tag Reading supports also MIFARE, ISO15693, FeliCa (from Sony - heavily used by transit and payment systems in Japan)
Watch technical session video for implementation details to all supported Native Tag Reading

Testing in Xcode ★

https://developer.apple.com/wwdc19/413

Testing Pyramid Refresher
- Unit Tests as foundation help to test a single piece of code - write many
- Integration Tests validate larger section of code (clusters of classes). Verify that different parts behave correctly together. You need less of these than unit tests.
- UI Tests observe the user-facing behavior of your app. Requires more maintenance, run slow. You need the least ones
XCTUnwrap is equivalent to: assert not nil; guard let <condition> else throw
XCTAssertEqual gains an accuracy argument for comparing double values
XCTAssertThrowsError has a closure to evaluate the thrown error. Ideal to write negative test cases.
UITests property isHittable ensures that an element exists, is on the screen and you can interact with it
Test Plans
- allow running tests more than once with different settings
- defines all testing variants in one place
- can be shared between multiple schemes
- supported in Xcode and xcodebuild for CI Xcode Server
Test Plans let you specify the following by using Configurations
- language, region, user location, automatic screenshots, localization screenshots, execution order, code coverage, sanitizers, API checking, memory management options, arguments and environment variables
Test Configuration represents a single run of your tests with certain set of options
Shared Settings are inherited by all test configurations
Localization Screenshots for preserving all screenshots from all runs with different localizations
Example Configurations With Different Focus Are No Problem Anymore: Memory Checking (Address Sanitizer + Zombie Objects), Concurrency (Thread Sanitizer, Undefined behavior Sanitizer, Random Order), Extra Diagnostics (ENABLE_LOGGING=1, Keep Attachments)
Xcode Server is finally usable to share test configuration between different bots. The ideal (and free) solution to test a white-labelled app with lots of targets
Result Bundles
- 4 times smaller due to optimized file format
- viewable in Xcode
- programmatically accessible via xcresulttool
- used to get code coverage diff via xccov diff --json Before.xcresult After.xcresult

Advances in Networking, Part 1

https://developer.apple.com/wwdc19/712

Low Data Mode enables configuration per wi-fi and cellular network how expensive the connection is
- delays discretionary tasks
- disables background app refresh
- apps have to adopt to it by reduce image quality (symbols vs. images), reduce prefetching, synchronize less often, mark background tasks as discretionary, disable auto-play, don't block user-initiated work
- first try expensive prefetch by allowConstrainedNetworkAccess = false. On failure error.networkUnavailableReason == .constrained try low data alternative
- URLSession.allowExpensiveNetworkAccess
- never check for connection type rather check if the connection is expensive or constrained
DataTaskPublisher uses Combine framework in URLSession. Useful to achieve the following in asynchronous networking:
- get rid of capturing self in completion handlers
- avoid duplicate network code like checking for status code
- achieve retry operation in just one line - since network operations are expensive: low retry count
- Pretty nice example of generic low-data-mode adaptivePublisher at minute 26:07
WebSocket
- Advantages
  - bi-directional connection over TLS/TCP connection
  - works with firewalls / CDNs
  - proxy support
- URLSessionWebSocketTask is exciting simple! Example at 30:00
  - NWConnection / NWListener offers client/server support for partial or complete WebSocket messages
Mobility Improvements never turn off Wi-Fi anymore when walking off home with slowly fading connection
- Improved Wi-Fi Assit: Cross-Layer Mobility Detection from all Frameworks (Network.framework, URLSession, Wi-Fi, Cellular)
- Apps can adopt Multipath Transport which requires the server to do so too
- High-Level APIs URLSession / Network gain benefits from improvements

Advances in Networking, Part 2

https://developer.apple.com/wwdc19/713

Bonjour available on EVERY native platform
- powers wide-area service discovery
- native browser support in Network framework via NWBrowser (discover)
- NWListener provides advertising support
- NWConnection handles the connection
Building Framing Protocols
- Developers can write their own framing protocol that runs in the same networking thread as TCP
- encapsulate or encode application messages
- used to write common code across TCP/UDP transports
Collecting Metrics more metrics for URLSession and newly created ones for Network
Optimistic DNS enabled by default and improves performance for answers with short time-to-live
Request and Response Metrics extended
- byte counts, packet counts, round-trip-times
- start and end reports to correspond to application activity
- multiple reports simultaneously
- per-path breakdown for multi-path protocols
iPad Apps For Mac opt-in to allow incoming connections - outgoing on by default
CNCopyCurrentNetworkInfo needs capability Access Wi Fi Information & must meet at least 1 of:
- permission to location
- VPN enabled
- if app is hotspot configurator it can access the info for networks it configured
Never switch on the current network device but rather do:
- allowsExpensiveNetworkAccess = false
- waitsForConnectivity = true
- taskIsWaitingForConnectivity delegate is called where you can switch to cellular data
Deprecations
- PAC files for schemes file:// and ftp://
- SPDY replaced by HTTP/2
- Secure transport doesn't support TLS 1.3 / Use URLSession / Network.framework instead

Getting Started with Xcode

https://developer.apple.com/wwdc19/404

1h presentation about all the power Xcode offers
You learn about all the UI elements and functionality of Xcode
- how to compile
- fix issues in SwiftUI
- add capabilities
- create Swift Packages
- how to add dependencies via Swift Package Manager
- unit testing
- target memberships
- running and debugging
- documentation
- and many more...
It is perfectly suited for new developers that have never worked with Xcode, but also for experienced ones to get to know the new UI of Xcode 11

Modernizing Your UI for iOS 13 ★

https://developer.apple.com/wwdc19/224

Deprecation of LaunchImages use Launch Storyboards instead
New App Store Requirement Any app linking to iOS 13 must ensure correct layout at any size
Split Screen Multitasking is strongly recommended for iPad Apps
- Apple is going to expect that most of the applications, unless you want to provide a truly immersive experience, are going to support split screen multitasking
UINavigationBarAppearance customize .standardAppearance (iPhone portrait), .compactAppearance (iPhone landscape), .scrollEdgeAppearance (if attached to scrollView and scrolled to top, bar uses this one)
- settable per navigationItem so customization per view controller possible (including color, transparency, …) let appearance = navigationBar.standardAppearance.copy() /* configuration */ navigationItem.standardAppearance = appearance
UIBarButtonAppearance, UIToolBarAppearance
UITabBarAppearance customize .stackedLayoutAppearance (text below icon), .inlineLayoutAppearance (text right to icon; iPads), .compactInlineLayoutAppearance (text right to icon; landscape; smaller phones)
UIModalPresentationStyle.[automatic | formSheet | pageSheet] are able to stack behind each other
- .automatic is the new default; gets resolved at presentation time
- get the old behavior by viewController.modalPresentationStyle = .fullScreen
- sheets get gesture recognizer added automatically to support pull-to-dismiss - opt-out possible (see link in description for example project)
- Appearance Callbacks Not Called for .PageSheet and .FormSheet on presentingViewController: viewWillAppear, viewDisappear, viewWillDisappear, viewDidDisappear
Search UI
- offers possibility to hide cancel button and scope control
- expose UISearchTextField on UISearchBar so customization becomes possible
- Search Tokens that represent complex searches
  - are copy&pastable and support drag&drop
  - can be created by user/developer from selected text in UISearchTextField
UITextInteraction the new & easy way to control selection UI
Multi Selection in Table-/Collection Views simply by Two-Finger-Swipe
- opt-in by implementing only 1 new delegate method for Tables and Collections
New Editing Gestures
- pinch 3 fingers in: copy selected text
- pinch 3 fingers out paste copied text
- 3 finger tap || 3 finger swipe back undo
- 3 finger swipe forward redo
- opt out by responder.editingInteractionConfiguration = .none
UIContextMenuInteraction context menus with rich preview, nested menus, in-line sections
- transform to context menus on macOS
- UIMenu & UIInteraction provide hierarchical menu construction system
- adopt by view.addInteraction(UIContextMenuInteraction(delegate: self))
- customize how view and menu will be presented: UITargetedPreview & UITargetedDragPreview
- convenience delegate methods for context menus on UITableView & UICollectionView
- UIViewControllerPreviewing (peek & pop) is deprecated for sake of this new API
- replace long-press driven behavior/menus by this new API

Designing Audio-Haptic Experiences

https://developer.apple.com/wwdc19/810

Core Haptics allows to use taptic engine fully in iPhone
Developers can change the Haptic Intensity and the Haptic Sharpness
3 Guiding principles: Causality, Harmony, Utility
Causality means that haptic feedback and the event causing it must match our expectations
Harmony means how sound/haptic/visuals work together
Utility add audio/haptics that provide clear value to your UX
Work with Primitives Transient (short) and Continuous (long)
- both can be modulated by haptic intensity/sharpness which work different with the 2 primitives
use transient building blocks for sharp, crisp and short sounds
use continuous primitives for smooth, extended sounds

Expanding the Sensory Experience with Core Haptics

https://developer.apple.com/wwdc19/223

This is basically a merge of sessions Designing Audio-Haptic Experiences and Introducing Core Haptics
For an in-depth example scroll the video to 40:10

Cryptography and Your Apps

https://developer.apple.com/wwdc19/709

Cryptography is hard to get right don't do it yourself unless you are an cryptography expert
Use Data Protection to protect your data on the device: try data.write(to: fileURL, options: .completeFileProtection) if you don't do so yet
Use Keychain for Credentials and Keys. Never write them to the UserDefaults!
Apple Watch can now be used to authenticate operations done on macOS
Use CloudKit to share data across devices/users by encrypting assets in private CK DB
Use TLS 1.3 (Transport Layer Security) to ensure a secure network connection. Usage is ensured by using Network.framework or URLSession
Use SecTrust to verify remote parties.
Adopt CryptoKit
- AES encryption inn just one line
- strongly typed interfaces
- memory management
- equatable conformances
- generics
- Hash Functions (SHA-256, SHA-384, SHA-512), Symmetric Key Cryptography (HMAC, AES-GCM), Public-Key Cryptography (Curve25519: P-256, P-384, P-512), Insecure Module (MD5, SHA1)
Using a signature to authorize an operation: try P256.Signing.PrivateKey()
Constrain key usage by generating them with the SecureEnclave, e.g. only usable when unlocked try SecureEnclave.P256.Signing.PrivateKey(accessControl:)
Customizing Authentication Context with LAContext: try SecureEnclave.P256.Signing.PrivateKey(accessControl:authenticationContext:)
Performant through usage of CoreCrypto, hand-tuned assembly code and usage of the A12 Bionic

All About Notarization

https://developer.apple.com/wwdc19/703

I highly recommend watching this session if you ever run into any problems regarding notarization since this is a complex topic and this video is packed of useful examples
Notarization identifies and blocks malicious software prior to distribution. It is NOT an App Review!
comes on top of your Developer ID - no new registration
Notary Service performs automated security checks
Process Local Development > Distribution Signing > Notarization Attachment > Distribution via Website
On App Download the Notarization attached to your app is checked by Notarization Service. Gatekeeper permits/denies installation.
Benefits
- prevents the developer from shipping a malicious dependency
- apps with the hardened runtime are more secure by default
- users are ore likely to try and download new software
- audit trail of software notarized by your Developer ID account
Software signed on or after June 1, 2019 must adopt
- complete and correct signing
- the hardened runtime
Complete and Correct Signing involves
- signing everything (Bundles, Mach-Os, Installer packages) with your Developer ID Application Certificate and include a secure timestamp
- Executables must opt-in the hardened runtime
- Sign Installer Packages with Your Developer ID Installer Certificate
- Sign Disk images with Application Certificate and include secure timestamp
- Enable Xcode Automatic Codesigning - it does it for you
Hardened Runtime extends macOS system integrity protection features to your apps
- Runtime code signing enforcement
  - configurable via entitlements
  - Adopt via codesign --sign "Developer ID" --timestamp --options runtime My.app
  - Verify via codesign --display --verbose=2 My.app and make sure runtime is printed next to flags
  - Look into 12:04 for detailed description
  - Look into 12:22 if your app crashes because you use JIT
  - Look into 13:54 if your app crashes because you patch system frameworks - don't do this
  - If your app crashes on auto-update: create a new file when you update a signed file
- Library Validation
  - protects your app from code injection and dylibs hijacking
  - prevents loading unsigned or adhoc-signed code
  - Detailed solutions for common issues can be fount at 16:00, e.g. App loads plugins from other devs in-process,
- DYLD Variable Environment Protection
  - can inject libs and modify framework and lib search path - useful for testing
  - Blocks DYLD_LIBRARY_PATH, DYLD_INSERT_LIBRARIES, DYLD_FRAMEWORK_PATH by default
  - Don't use DYLD environment variables when shipping to customers
  - You can use com.apple.security.get-task-allow entitlement during debug build
- Debugging Protection
  - disables debugging hardened processes by default
  - You can use com.apple.security.get-task-allow entitlement during debug build to get around this - Xcode does it automatically
- Protected Resource Access
  - App needs to declare its intent to access protected resources, e.g. location, photos, contacts, ...
  - settable via entitlements - see 20:46
- Use only entitlements really needed
- Set those entitlements only for processes that need them
- Set resource-access entitlements only on main bundle; get inherited by other bundles
Notarization can be done easily via the Archive menu from within Xcode
xcrun altool --notarize-app ... to submit an app via command line and check via xcrun altool --notarization-info <request_id_from_submission> … for the current status
Use xcrun altool --notarization-history … to get on overview of all the software submitted on your account

Implementing Dark Mode on iOS

https://developer.apple.com/wwdc19/214

Use Semantic Colors from now on! They are actually color independent. Name stays the same in light/dark mode.
Color can have different appearances light/dark
Asset Catalog lets you easily customize colors and imagesfor light/dark
New Set of Dynamic Colors automatically switch between light/dark versions
New Materials are more than colors. Can include blur effect.
Auto-Updating Materials by UIBlurEffect(style: .systemThinMaterial) and UIVibrancyEffect(blurEffect: blur, style: .secondaryLabel)
Static Color by dynamicColor.resolvedColor(with: traitCollection)
Initilize Dynamic UIColor by UIColor(dynamicProvider: (UITraitCollection) -> UIColor) and switch traitCollection.userInterfaceStyle {}
CALayer won't understand dynamic colors - you need to do this manually
Create Dynamic Images with UIImageAsset.registerImage:withTraitCollection
Check for Changes In Trait Collection with override func traitCollectionDidChange(_:) use traitCollection.hasDifferentColorAppearance(comparedTo: previousTraitCollection)
Traits predicted during Initialization traitCollectionDidChange(_:) only called for changes
Debugging with launch argument UITraitCollectionChangeLoggingEnabled=YES
Override Specific UI by UIViewController./UIView.overrideUserInterfaceStyle: UserInterfaceStyle
Complete App [light|dark] with Info.plist.UIUserInterfaceStyle=[Light|Dark]
var overrideTraitCollection: UITraitCollection? on UIPresentationController
func setOverrideTraitCollection(_: UITraitCollection?, forChild: UIViewController) on UIViewController
UIStatusBarStyle.[default(automatic)|light|dark]
UIActivityIndicatorView styles deprecated Use .style = [.medium|.large] and .color.
Use NSAttributedString always with dynamic .foregroundColor to have the auto-switching effect
iOS/tvOS apps built using iOS/tvOS 13 SDK supposed to support Dark Mode

Advances in Speech Recognition

https://developer.apple.com/wwdc19/256

Support for macOS with 50+ languages which requires privacy approval from the user and Siri to be enabled.
On-device speech recognition, speech is private, stays on-device, network connection not required and no cellular data consumed. To utillise on-device learning set requiresOnDeviceRecognition on SFSpeechRecognitionRequest to true.
On-device limitations with lower accuracy than the server and reduced set of available languages (English, Spanish, Italian, Brazilian Portuguese, Russian, Turkish and Chinese).
On-device support includes devices with an A9 chip and above and all mac devices.
Server accuracy is higher in comparison to on-device recognition but is limited to 1 minute max audio duration with a limited number of requests per day.
Results from Transcription include alternative interpretations, confidence level and timing information. New parameters include speaking rate, average pause duration and voice analytics.
Voice Analytics Features include:
- Jitter – Measures variation in pitch
- Shimmer – Measures variations in amplitude
- Pitch – Measures frequency characteristics of voice
- Voicing – Identifies voiced regions in speech

Optimizing App Launch

https://developer.apple.com/wwdc19/423

Cold Launch (slow) > Warm Launch (medium) > Resume (fastest)
iOS13 caches runtime dependencies to improve warm launch
Measure LaunchTime in clean environment
- Reboot and let system settle 2-3 min
- Enable airplane mode or mock network
- Log out from iCloud
- Use release build of your app
- Measure warm launches
Improve Launch
- Target for startup time ~400ms
- Defer work unrelated to first frame
- Move blocking work off main thread
- Reduce memory usage
- Choose the right quality of service for tasks
- Optimize algorithms & data structures
- Cache resources and computations
- Flatten View Hierarchies
- Reduce Autolayout Constraint Count
- Initialize your view controllers in scene:willConnectToSession:options: when adopted UISceneDelegate
- Minimize work in UIApplication/UIApplicationDelegate subclass or move it to another thread
- Avoid linking unused frameworks and Dynamic Library Loading during launch
- Hard-link all your dependencies
- Expose dedicated init API in frameworks
- Reduce impact to launch by avoiding +[Class load]
- Use +[Class initialize] to lazily conduct static init
New App Launch Template of instruments to measure launch time
- Select profile scheme (long press button) to install release-mode app
- Hitting the record button in instruments launches your app
- Allows you to analyze each pase of app launch (finds problems with static class initializers, blocking code in app delegate)
Use XCTest performance tests to measure app launch time regularly. measure(metrics: [XCTOSSignpostMetric.applicationLaunch], options:block:)
Monitor Customer Launches in Xcode Organizer
Custom power/performance metrics by adopting MetricKit

Accessibility Inspector

https://developer.apple.com/wwdc19/257

Helps find and fix accessibility issues
Open via Xcode > Open Developer Tool > Accessibility Inspector and run an audit to find possible issues
AccessibilityLabel should be set to meaningful and localized string
Contrast Debugging useful if ratio between foreground & background is not sufficient
Open Color Contrast Calculator in Accessibility Inspector via Window > Show Color Contrast Calculator

Visual Design and Accessibility

https://developer.apple.com/wwdc19/244

Dynamic Type
- make as much text as possible dynamic
- use as much of the screen width as possible
- don't truncate text
- scale glyphs (icons/images) next to your text with your text
- iOS provides 11 text styles
- custom font support made easier in iOS 11
Reduce Motion
- adapt animations if UIAccessibility.isReduceMotionEnabled == true (there is a notification too)
- consider disabling autoplaying videos if UIAccessibility.isVideoAutoplayEnabled == false (there is a notification too)
Differentiate Without Color is new in iOS13
- do not rely on color alone
- add additional indicators (icons) where color is the only way to convey information if UIAccessibility.shouldDifferentiateWithoutColor == true (there is a notification too)

HLS Authoring for AirPlay 2 Video

https://developer.apple.com/wwdc19/507

AirPlay directly built into TVs
Video Requirements
- sync variants
- avoid changes at discontinuities
- full range of variants for each codec
- 10% partial encryption
- provide compatible format: HDR content with only HDR formats, WebVTT for subtitles, use recommended MIME types
Cahnges to Validation
- HLS validation by mediastreamvalidator & hlsreport.py
- always use both validation tools
- HLSReport now checks all rule-sets by default

AUv3 Extensions User Presets

https://developer.apple.com/wwdc19/509

Presets fine-tunes set of parameter values
Capture Snapshot of state of Audio Unit's parameters
Loading preset restores Audio Unit to same state
AUAudioUnit.factoryPresets built into the unit by manufacturer. Immutable.
AUAudioUnit.userPresets are new and are created by user. Mutable. Unit exposes them to all host applications.
supportsUserPreset: Bool used to verify support by the host
saveUserPreset(_:) & deleteUserPreset(_:)
presetState(for:) throws -> [String : Any] to get state.
isLoadedInProcess: Bool is a macOS only feature
Methods above have default implementations. Can be overridden.

Game Center Player Identifiers

https://developer.apple.com/wwdc19/615

GKLocalPlayer represents authenticated player. Has persistent teamPlayerID & gamePlayerID
GKPlayer provides info about other players. Uses scoped IDs
Scoped IDs teamplayerID and gamePlayerID
- properties on GKPlayer
- increase player privacy
- replace the playerID (has been deprecated) with scoped IDs (save game data / backend)
- perform conversion after next authentication
- teamPlayerID scoped to development team
- gamePlayerID scoped to game
loadPlayersForIdentifiers:withCompletionHandler:

Text Recognition in Vision Framework

https://developer.apple.com/wwdc19/234

Not necessary anymore to use OCR / CoreML Model to detect text using Vision Framework
VNRecognizeTextRequest that returns [VNRecognizedTextObservation] introduced as new request class
Request can configure speed / accuracy of recognition using VNRequestTextRecognitionLevel
Fast mode should take 0.25s / Accurate mode should take 2.0s
Possibility to set custom words to recognize. Minimum height of words. Priority of languages. Possibility of auto-correct detected content.
More about VNRecognizedTextObservation
- Call topCandidates to get a list of VNRecognizedText
- VNRecognizedText is the type of object that gives the String detected
Best Practices
- Specify language in use
- Set custom words when need domain-specific text
- Consider increse the accuracy when text is confusable or illegible
- Manage progress of request using progressHandler / cancel()

Understanding Images in Vision

https://developer.apple.com/wwdc19/222

New: Image Saliency: attention and objectness based (presented by Brittany Weinert)
- attention based saliency
  - heat map - where do humans look (e.g. faces)
  - using contrast, faces, subjects, horizons, light
  - perceived motion
- objectness based saliency
  - object segmentation for foreground objects
- Highlighting Areas of Interest in an Image Using Saliency | Apple Developer Documentation
- small buffers (float values: 0-1) and bounding boxes (multiple for objectness)
- VNGenerateAttentionBasedSaliencyImageRequest and VNGenerateAttentionBasedSaliencyImageRequest
- Use as graphical mask, to zoom in on relevant parts of an image or to defer interesting parts to classification
New: Classification (presented by Rohan Chandra)
- Bringing the complex task of classifying images on device into the API using the same network as for photo search.
- multi-label model: identifying multiple objects in one image
- Hierarchical taxonomy for label and confidence
- NClassifyImageRequest
  - Interpreting the result, we have to pick thresholds
  - Precision vs. recall (by filter(hasMinimumPrecision, …) vs. filter(hasMinimumRecall(), …))
New: Image similarity (demo)
- should regard the content instead of pixel values (semantic similarity)
- feature vector of classifying computation featurePrintObservationForImage() -> computeDistance()
Face technologies (Sergey Kamensky)
- Face recognition / understanding with VNDetectFaceLandmarksRequest
  - improved pupil detection
  - confidence per 76 face points (landmarks)
  - face capture quality (comparative measure of same subject) VNDetectFaceCaptureQualityRequest
New detectors, tracking and integration (presented by Sergey Kamensky)
- human detector (upper body) - bounding box
- animal detector(cats + dogs) - bounding box + label
- Tracking improved for occlusion, efficient in background, ML-based VNSequenceRequestHandler
- Integration with CoreML now also for multi image inputs

Advances in UI Data Sources

https://developer.apple.com/wwdc19/220

The new diffable data source API helps you handle changes in your data source to reflect on your UI in a new, easy and safe way.
Current state-of-the-art : UICollectionDataSource is straightforward and flexible, but generating UI updates can be challenging
Batchupdates
- When updates go wrong -> crash - Invalid number of sections ...
- Usually we would use reloadData(), which works, but at the cost of the user experience as there are no animations to emphasis the changes.
Where is our truth ?
- Or data source and current UI state must always agree
- Current approach is error prone
- No centralized truth
A new approach : Diffable Data Source
- A declarative approach to UI State
  - performBatchUpdates() -> Crashing, hassles, complexity
  - apply() -> Simple, automatic diffing
Snapshots
- Truth of UI State
- Unique identifiers for sections and items
- No more IndexPaths
3 steps to update
- Create snapshot
- Modify the snapshot with new data
- apply() the snapshot
Diffable data source updates
- Conforms to UITableViewDiffableDataSource or UICollectionViewDiffableDataSource
- Always call apply()
- Never call performBatchUpdates(), insertItems(), etc
Constructing Snapshots
- Empty snapshot
  - let snapshot = NSDiffableDataSourceSnapshot<Section, UUID>()
- Current data source snapshot copy
  - let snapshot = datasource.snapshot()
Identifiers
- Must be unique (UUID)
- Conforms to Hashable
- Data model or identifier
Performance
- Fast
- Safe to call apply() from a background queue
  - Although always call exclusively from the main queue or a background queue and stay consistent (if you choose to go background, always go background)
apply()
- Available for iOS, tvOS and macOS
- Automates animation
- Easy, fast and robust

Data Flow Through SwiftUI

https://developer.apple.com/wwdc19/226

Luca Bernardy, Raj Ramamurthy

Main principles
- Data access is a dependency
- Every UI piece of data has its source of truth (have a single one)
@State property wrapper
- framework allocates the storage and initiate redraw when it's changed
- it is a source of truth
@Binding property wrapper
- doesn't create a new state (another source of truth), just a reference to an existing state
SwiftUI as a remedy for Massive View Controller
Standart components (Slider, Toggle, TextField) expect some bindings (so they don't retain the state)
Adding reaction to external actions - via Combine's publishers
BindableObject protocol to conform when creating bindable data models
@ObjectBinding property wrapper
- the same as @Binding but for reference-type objects
Using Environment object for indirect dependencies
Prefer immutable access (Swift properties, Environment) over the mutable one (Binding) wherever possible
Use state effectively
- limit the usage, use bindings instead
- when using consider the context of it

Binary Frameworks in Swift

https://developer.apple.com/wwdc19/416/

Harlan Haskins, Jordan Rose

Two main ways to distribute 3rd-party code
- for shipping the source code - Swift Packages
- for shipping binaries - XCFrameworks (new format for distributing binary libraries)
XCFramework
- One bundle for different binaries (iOS Device, iOS Simulator, macOS, tvOC, watchOS)
- Ability to distribute frameworks and static libraries
- Support for Swift and C-based code
Security/Trust consideration
- Entitlements and permissions granted to the app are granted to the frameworks as well (and all this dependencies)
Creating a XCFramework
- Build Settings -> Build library for distribution
- New Swift Module Interfaces - a text file with the metadata about the compiler (version, flags) and all the publich API of the framework
- Xcode (or $ xcodebuild archive command) creates bundle for different targets/architectures and puts it to the Archive folder
- $ xcodebuild -create-xcframework to bundle all the binaries together (seems like no UI in the Xcode for this step so far)
Framework Author Consideration
- Carefully consider versioning (frameworks plist), use semantic versioning (An example how to pick a new version number)
- Keep you API small (the smaller the more flexible)
- Choose names and requirements carefully
- Avoid unnecessary extensibility (Crossing ABI boundary is like using dynamic method dispatch in ObjC)
- Use compiler optimizations notation when possible (so the client code's compiler can optimize the usage of the framework):
  - @inlinable functions (with @usableFromInline property modifier)
  - @frozen enums and structs
- Minimize and document your framework's entitlements, permissions and dependencies
- Handle the permissions' denial gracefully
- All the framework's dependencies have to be build with this new mechanism themselves
- Binary dependencies cannot depend on packages (source code modules)
- Switch off ObjC headers generation if you don't have ObjC interfaces

Creating Great Localized Experiences with Xcode 11

https://developer.apple.com/wwdc19/403

New Features
- Language setting per app
- keyboards for 38 languages & easier keyboard switching
- handwriting for Chinese
- Localized relationships in Contacts
- language detection for dictation
- enhanced language setup for multilingual users
Per-App Language Setting
- Possible in watchOS (via phone sync), macOS
- No developer action required
- Changing language in code is not supported anymore
- Use UIApplication.shared.open(URL(string: UIApplication.openSettingsURLString)!) to guide the user to the setting
- Changing language will re-launch the app
- Use State Restoration to make language switching seamless: viewController.restorationIdentifier
- Scene state restoration by implementing SceneDelegate.stateRestaurationActivity(for scene:) -> NSUserActivity
- Use Bundle.main.preferredLocalizations.first to get current running app language
- If you have a list of languages (obtained from e.g. a server) get best matching language by Bundle.preferredLocalizations(from: availableLanguages).first
15x faster localization export/import by xcodebuild -importLocalizations -project <pname> -localizationPath <path> && xcodebuild -exportLocalizations -project <pname> -localizationPath <path> -includeScreenshots
New stringdict rule NSStringDeviceSpecificRuleType to specify device-specific localizations
- accepts the values appletv, applewatch, ipad, iphone, ipod, mac
- can be combined with plural and variable width rules
Settings bundle included in Localization catalog
Localization of image/symbol sets in AssetCatalog
Use Test Plans to test Localization topics like clipping, truncation, layout overlapping, right-to-left languages
- gather screenshots for localizations for the use cases "App Store Screenshots", "Forward them to Localizers To Provide Context"
- UI Test Attachments, e.g. screenshots, can be added to Localization Catalog
- on localization export Xcode even provides the frame of localizable content on the screenshot so tools can visualize it appropriately

Getting the Most Out of Simulator

https://developer.apple.com/wwdc19/418

What is Simulator?
- isolated userspaces (iOS, watchOS, tvOS) running on macOS kernel
- seperate launchd, deamons, URLSessions, mach bootstrap, ...
- same filesystem, different $HOME
- memory and CPU limits are NOT simulated
- App Sandbox is not enforced
- Simulates case-sensitive filesystem
- thread sanitizer supported
Burning FAQs
- Zoom in by Click&Drag while holding ⌥
- Simulate Drag&Drop by drag an app; hold ⌃ to simulate a hold; select the page you want to drop the app; release ⌃
- Select Used Autio-Output Device from menu Hardware > Audio Output.
- Log in To iCloud to access all your photos, calendars, etc
- Trigger iCloud Sync to manually sync iCloud with Simulator
- Install additional (older) Simulators by pressing ⇧ + ⌘ + 2, click on Simulators and then the + button at the bottom left
- Change SImulator Window Size from the Window menu
- Activate Slow Animations to debug animations by Debug > Slow Animations (⌘ + T)
- Siri supported on all Simulators
- Control tvOS with software remote, hardware remote (after paired), game controllers, bluetooth keyboard
- Using Xcode 10 with iOS 13 Simulators is possible by this weird combination of actions (look at 12:32)
- Drag&Drop App bundles, locations, images, videos, URLs
- Share via Share Sheet to multiple simulators at the same time
- Simulate Custom Location via Debug > Location > Custom Location...
- Simulate Active-Call Statusbar by Hardware > Toggle In-Call Status Bar (⌘ + Y)
- More Available Features: External Displays, Dark Mode Toggle, Setting Alternate Backgrounds, Dictation, Alternate Software Keyboards, Quick Path (Swipe-To-Type)
simctl
- xcrun simctl help to show all the commands of the simctl command. See some examples below.
- xcrun simctl list devices to list all devices.
- xcrun simctl --json outputs JSON for automation purposes.
- xcrun simctl create to create new simulators. Outputs the device id.
- xcrun simctl spawn <device> <command> <argument> spawn command in simulator, e.g. change application user defaults.
- xcrun simctl diagnose collect logs on disk, dumps sytsem state, etc. Great for test failures in automated systems. Use -l flag to skip privacy hint. Attach when filing feedback!
- xcrun simctl launch to launch an application.
- xcrun simctl launch --console-pty booted com.apple.example -MyDefaultKey YES to display the log output in the current terminal window and provide (overwrite) launch arguments.
- xcrun simctl boot <device> to boot simulator with the given id.
- xcrun simctl shutdown all to shutdown all simulators.
- xcrun simctl delete <device> to delete simulator with the given id.
- xcrun simctl delete unavailable to delete simulators using an unavailable runtime.
- xcrun simctl pair <watch> <phone> to a watch simulator with a phone simulator.
- xcrun simctl addmedia <device> <file1> <file2> to add media to the simulator.
- xcrun simctl get_app_container <device> <bundle id> <type> get app container path.
- xcrun simctl install <device> <example.app> to install an app.
- xcrun simctl io <device> screenshot <output.png> to take a screenshot.
- xcrun simctl clone <device> clone name to clone the simulator instance by taking advantage of the APFS file sytem. Needs to be shutdown before cloning.
- xcrun simctl status_bar <device> override --time "11:00" --batteryState charge to override displayed statusbar values
Metal in Simulator
- Simulator is much faster and is fully GPU accelerated
- Metal hardware-accelerated renderings are now possible the first time
- MTLGPUFamilyApple2 support (common to all simulator devices)
- All Apple frameworks use Metal as well
- Metal takes advantage of GPU of the underlying Mac system
- Texture storage modes are different on macOS/iOS
- Always test performance on actual devices! (For additional infos 37:50)

SwiftUI on watchOS

https://developer.apple.com/wwdc19/219

watchOS 6
- Independent watchOS apps (decoupled from iOS apps)
- Extended run-time sessions
- Building experiences: Complications, Notifications, Siri, etc.
- Prioritize quick interactions
Full Power of SwiftUI
- Declarative syntax Whole new UI framework, new fetures and APIs
- Integration Watchkit controllers with SwiftUI Views
  - InterfaceController inherits WKHostingController
- Lists WatchOS flash cards app
  - Keep model and List in sync using @ObjectBinding
  - Use Command + Click to bring up the inspector and use different contextual options while coding
  - Use .listStyle(.carousel) to get the carousel effect while scrolling the list
  - Swipe to delete, drag to reorder
  - Use .onMove and .onDelete blocks to manage the movement and deletion of items in the list
- Interactive Notifications Timely and contextual info
  - Short look (Info from payload + App icon) Immediately upon wrist raise
  - Long look (Scrolling interface with custom body and action buttons)
  - NotificationController inherits WKUserNotificationHostingController
    - didReceive method allows us to extract info from notification
    - body property is re-evaluated after didReceive is called
- Digital Crown Series 4 watch can make use of haptic crown (e.g. workout app, custom timer)
  - Building following custom interfaces requires .digitalCrowRotation and .focusable modifier
  - Free scrolling interface (no concrete spots between elements)
    - binding (source of truth)
    - from
    - through
  - Picking between discrete elements
    - binding (source of truth)
    - from
    - through
    - by (stride along which haptic feedback is provided)
  - Moving around circles (not limited to either end of the sequence)
    - binding (source of truth)
    - from
    - through
    - by (stride along which haptic feedback is provided)
    - sensitivity (how much rotation need to be applied to move from one element to the next)
    - isContinuous (don't stop at either limit of sequence)

Core ML 3 Framework

https://developer.apple.com/wwdc19/704

Michael Brennan, Anil Katti, Aseem Wadhwa and Allen Lin

New: Create ML-App to create models (beside tools for importing from different sources)
How can we personalize to each user without going back to the cloud (so that data stays private, we have no server costs and no connectivity needs)? Models are usually immutable on-device.
- Core ML 3 adds update parameters and update interface to the model
- You just add training examples and get a new personalized model
Supported for Nearest Neighbor Classifiers, Neural Networks, Pipeline Models
Demo: Personalized Grading App
- Learn how own sketches map to emoji - only some samples to learn
Three steps for updating
1. Get the bundle
2. Prepare the training data
3. Create an MLUpdateTask
More complex: updatable Neural Networks
- configure some layers as updatable (convolutional and fully connected)
- works with different loss types and optimizers
- MLModelConfiguration parameters can be changed at runtime
Core ML in the Background: BGTaskScheduler and BGProcessingTaskRequest
Correspondence between network layers and computation graphs
- new in Core ML 3: control flow (conditions, loops), dynamic layers
- lots of new layers, close to current research
Improved converters: „Smooth road ahead“
Natural Language Processing: asking questions about an article (demo)
- BERT-Model, trained with TensorFlow, converted to protobuf with just three lines of Python
- Integrated with Speech-To-Text, NaturalLanguage API, Text-To-Speech
Multiple Models with shared submodels - use a linked model multiple times for memory reduction (like a dynamic library)
MLFeatureValue Image Extension for automatic scaling and format conversion (without calling Vision framework)
MLModelConfiguration has new options preferredMetalDeviceand allowLowPrecisionAccumulationOnGPU

Debugging in Xcode 11

https://developer.apple.com/wwdc19/412

Device Conditions
- Thermal State (fair, serious, critical) is now simulatable via Xcode
  - Subscribe to notification ProcessInfo.thermalStateDidChangeNotification to react to changes
  - reduce resource-intensive operations when runninng on serious or critical
- Network Connection Quality (Network Link Conditioner) is now simulatable via Xcode
Environment Overrides
- Light/Dark Mode, Dynamic Type & Many other Accessibility Settings (INcrease Contrast, Bold Text, Reduce Transparency, etc.)
- Easily settable while debugging in the debug bar and live-previewable in the new canvas view
- Shows example code of an AdaptingStack which automatically updates stack axis when not enough space for UI element (11:58)
SwiftUI Runtime Issues
- found when process is running
- process continues executing
- details in issue navigator
- supported for all platforms and run destinations
- new kind of breakpoint which get hit only on runtime issues
- start/stop debug session from play/stop button in canvas
Debugging SwiftUI View Hierarchies
- SwiftUI is interoperable with UIKit
- Debug session is tied to lifetime of the Preview Canvas (make sure you don't close it)
- Views are Value Types
- Platform Specific Views at Runtime
- Inspector on the right shows
  - colors are Dynamic/Static
  - trait collection (dark/light/size-class) for views/view Controllers
  - names of images, if it is a symbol image as well as the SymbolImageConfiguration
  - much more details regarding constraints. Looks like Interface builder now
- View properties automatically inspected and shown in Inspector on the right
- Custom Inspector properties by adopting CustomReflectable
- Using live Preview Canvas and the Inspectors you almost don't need to compile the whole project anymore

Accessibility in SwiftUI

https://developer.apple.com/wwdc19/238

Introduction to Accessibility
- Three Examples
  - VoiceOver - Screen Reader, reads UI Elements and Gestures
  - New: Voice Control - Control interfaces by speech
  - New: Full Keyboard Access (also general productivity improvement)
- Accessibility User Interface
  - Consists of accessibility elements
  - An element consists of a labels, trait/roles and action
- Key Principles
  - Understandable: element labels should be descriptive
    - Question to ask: Do the displayed strings provide enough information?
  - Interactable: elements should have clear actions
    - Question to ask: Will a custom action simplify the interaction?
  - Navigable: Ordering and Grouping
    - Question to ask: Can you speed up navigation?
Automatic Accessibility with SwiftUI
- SwiftUI generates accessibility elements automatically for all standard controls
  - SwiftUI Text autogenerates Accessibility Label "Text of Label" and Trait/Role "Static Text"
  - SwiftUI Button autogenerates Accessibility Label "Text of Button", Trait/Role "Button", and default action "Press/Tap"
- Accessibility Notifications
  - Tell Voice Over if content of UI Elements changes so it re-reads the element
  - Since SwiftUI is state-driven, so all accessibility notifications are build in - even for custom controls
- Custom Controls
  - You can easily create custom buttons with colors that make them more visible
  - Image accessibility: You can label images with using label: on an image
  - You can exclude images form Voice Over by using decorative: in the constructor, this will prevent the creation of its accessibility element
- Control Labels
  - Some controls, like Picker, don't have a label by default and the visible value doesn't always give enough context
  - SwiftUI Picker has a build in label which can be used to explain its purpose
  - Many SwiftUI controls have this label already, the rest will follow in the future
  - In a future seed, controls will also have optional visible labels, which Voice Over will automatically use as an accessibility element label
- SwiftUI Accessibility API
- If the autogenerated label is not precise enough, the .accessibility(label:) modifier can be used on controls
- Accessibility traits can be added: For example, the modifier .accessibility(addTraits: selected ? .isSelected : []) will add the selection state after reading the label and the button type
- The same can be done with values: .accessibility(value: Text("\(value)")) - more descriptive text could be add here to explain the value
- You can also add custom actions with .accessibilityAction(named: Text))
- Add any kind of hints that might useful to understand the App State better
Accessibility Tree
- Use hierarchies
  - To reduce the amount of elements, combine them in a meaningful way and label them specifically
  - .accessibilityElement(children: .combine) on an HStack with a Text and two buttons will merge all provided accessibility into one accessibility elements, using the label-text as element-label and the two buttons as automatically converted custom actions
- The default ordering might not always be ideal
  - Use .accessibility(sortPriority: 1) to make Voice Over start with this element
Evaluating Accessibility
- Most importantly, use your app with VoiceOver, Full Keyboard, and Voice Control
- In addition, use Accessibility Inspector (extra session)

Designing Great ML Experiences

https://developer.apple.com/wwdc19/803

Products where Machine Learning (ML) is used
- Air Pods: Summon Siri and answer calls
- FaceID: Fast and secure authentication for iOS devices
- Improve Typing experience: Increase/decrease of tap area based on inputs
- Photos: Create albums and memories. ML is used to regonize entities within pictures, e.g. searching for dogs by typing
Design more than just the interface
- ML for Photos, e.g. diffentiate between dog breeds as a category
- ML model is trained as a function, e.g. Siri is a model that does voice to text translation
- Design how it works and how it looks and feels (Model & Interface)
Model: Data
- Needs a lot of diverse data for categories people want to search for
- Support for 1000s of categories
- Choosing data for improving existing model
- Data determines the behaviour of the model
- Data needs to be designed
- Historically facial recognization hasn't worked well for people of color
- Data engineering team gathered lots of data from ethnicity and gender
- Collect data intentionally, i.e. with special care to inclusiveness and avoid bias
- Optimize for customers you want
- Update data as products change
Model: Metrics
- Evaluate model by testing it, e.g. 75% of accuracy
- Design metrics how model will work and thus the experience
- Metrics reflect values
  - Understand mistakes: Mistakes are not equal (build better understanding overtime)
  - Design failure scenarios
  - Evaluate the experience
  - Evolve metrics
- Metrics are proxies for what we actually care about
- AppStore example
  - Recommendation for new apps based on what you've downloaded
  - Diversity is important, recommendations are balanced by editorial content
Interface
- Mapping between Input and Output
- Human Inteface guidelines which has more details
- Outputs are design medium
  - Multiple Outputs: Allow to choose best from multiple options
    - Siri: Suggestion based on time, location and past interactions
    - Ranking adjusted based on interactions
    - Maps example
  - Attribution: Explaination on how app makes decisions
    - App Store: Explains how suggestions are made
    - Siri: Explains how a answer was generated, e.g. sources
  - Confidence: Measurement of uncertainty
    - Numbers might not work in all situation because its difficult to interpret them
    - Ask for user input in case of low confidence
  - Limitations: Difference between mental model vs. how apps actually work
    - Acknowledge app's limitation and suggest user how to live with it
    - Inline coaching tips for Emojis in low-light settings
    - Suggest alternative to achieve goals
- Inputs
  - Calibration: Essential information to engage in experience
    - Example of Hoop app
    - FaceID uses calibration
    - Guiding and confirming calibration
  - Implicit Feedback: Information arises from user interaction and uses it to improve experience
    - Siri: Using time, location for suggestions, e.g. home vs office
    - Customization of suggestions, e.g. for privacy results
  - Explicit Feedback: Allows to collect information by asking specific questions
    - Priortize negative feedback over positive
    - Clearly describe each option and its consequences
    - Act immediately and persistently
  - Correction: Fix mistake model has made on known task
    - Corrections while typing with regards to suggestions

Creating an Accessible Reading Experience

https://developer.apple.com/wwdc19/248

For custom views set isAccessibilityElement = true
Enable Accessible Text Content by adopting UIAccessibilityReadingContent
- In accessibilityLineNumber(for point: CGPoint) -> Int use hit testing to identify subviews
- In accessibilityContent(forLineNumber: Int) -> String? return the accessibilityLabel for the subview matching line number
- In accessibilityFrame(forLineNumber: Int) -> CGRect return the accessibilityFrame for the subview matching line number
- In accessibilityPageContent() -> String? return the concatenated accessibility labels for all subviews
Enable Automatic Page Turn by
- setting view.accessibilityTraits = UIAccessibilityTraits.causePageTurn on your page view
- implementing accessibilityScroll(_ direction: UIAccessibilityScrollDirection) -> Bool (lets voice over turn pages)
Customizing Speech by adopting the protocol UIAccessibilityReadingContent
- customize language by setting NSAttributedString.Key.accessibilitySpeechLanguage
- customize pitch by setting NSAttributedString.Key.accessibilitySpeechPitch
- ...

Create ML for Object Detection and Sound Classification

https://developer.apple.com/wwdc19/406

Alex Brown, Dan Klingler

Object detection
- Download existing object detectors for broad categories, but train your own models for specific subtle differences
- Image classification describes the complete image, but Object detection can identify and locate multiple objects
- For training object detection we need to annotate labeled regions (label, center (x,y), width, height) and store that as JSON file in the same folder as the images.
- Demo: use Create ML to build a model for recognizing the numbers on the top of (multiple) dice
  - Drag folder to training data and Create ML will do some basic checking/validation
  - Training object detection takes much longer than image classification
  - Visualizing training and results
  - Test with new images directly in Create ML (use Continuity to access the iPhone camera)
  - Export mlmodel-file to use in your app
- Considerations:
  - Balanced number of images in each class
  - 30+ images per class
  - Real-world data: multiple angles, backgrounds, lighting conditions, different other objects
  - A single class can be enough (model learns to locate this class in images)
- Use Vision framework to integrate into app
Training Sound Classification Models in Create ML
- Identify the source of the sound (e.g. guitar vs. drums or nature vs. city) or identify properties (laugh vs. cry)
- Demo: use Create ML to build a model for classifying musical instruments from sound
  - Sound files from different instruments in different folders
  - Drag into Create ML, automatically separated into Training and Validation
  - Test the model in Create ML on files or live on microphone
  - Export mlmodel-file to use in your app
- Considerations
  - Add background noise as a separate category
  - One class per file (split if necessary)
  - Real-world audio environments
  - AVAudioSessionMode selection
- Integration
  - New framework: Sound Analysis for automatic channel mapping, sample rate conversion and audio buffering before applying the model
  - Results are in block size (e.g. one second) overlapping by 50% (default)

Auditing Web Content with Web Inspector

https://developer.apple.com/wwdc19/514

Web Inspector includes built in audit system which can catch code changes and inconsistencies in webpage content
Enable Safari Develop menu from Preferences > Advanced > Show Develop menu in menu bar
Use ⌘ + ⌥ + I to open Web Inspector
Audits consist of TestGroups and Tests
Audit results stay visible across page reloads and cleared when Web Inspector is closed
Results can be im-/exported from/to JSON
Write custom Audits in JSON format - makes them easily sharable

Embedding and Sharing Visually Rich Links

https://developer.apple.com/wwdc19/262

Tim Horton

Present rich links in 3rd party apps via the LinkPresentation.framework
Retrieve Metadata
- URL > LPMetadataProvider > LPLinkMetadata
- metadataProvider.startFetchingMetadata(for: url)
- LPLinkMetadata is serializable and can be used for local file URLs too
- LPLinkMetadata should be cached locally to not always hit the internet for a request
Presenting Links
- Use LPLinkView to present the meta data in a way well known to the user
- Alternatively use LPLinkMetadata properties to create a custom view
Accelerating the share sheet
- Provide cached metadata to UIActivityViewController via UIActivityItemSource
- Build the LPLinkMetadata object yourself if you have all data at hand
- Return cached or self-made LPLinkMetadata object in func activityViewControllerLinkMetadata(_: UIActivityViewController) -> LPLinkMetadata?

Combine in Practice ★

https://developer.apple.com/wwdc19/721

Michael LeHew, Ben D. Jones

Combine defines unified abstraction that describes API that can process values over time.
Publishers
- Conform to Publisher protocol
- Associated types: Output (kind of published values) and Failure (constrained to Swift.Error)
NotificationCenter
- NotificationCenter now exposes its notifications with Publishers (NotificationCenter.default.publisher(for:))
- Notifications can never fail (Never)
Operators
- Calling functions like map or flatMap returns new Publishers operators
- Dedicated operator for decoding decode(Foo.self, JSONDecoder()
Error Handling
- Every Publisher describes how they can fail
- Operators can help recovering from errors and dealing with them
- assertNoFailure() asserts that failure can never happen. When error arrives, program will trap because error type is Never.
- catch is useful for recovering from an error
  - Provides a closure
  - Catch lets us replace the current Publisher with a new one, e.g. using Just to publish just one value
  - Error type is Never
- Other failure handling operators:
  - retry
  - mapError - receives and can return different type of error
  - setFailureType - used for matching error type with subscriber
FlatMap
- Transforms all elements from an upstream publisher into a new or existing publisher
- Lets you perform multiple operations like returning value, decoding it and catching error
- Handles the details of subscribing to the nested Publisher while offering its values downstream
Scheduled Operators
- Describe when and where a particular event is delivered
- Supported by RunLoop and DispatchQueue
- Examples:
  - delay - Delays delivery of all output to the downstream receiver by a specified amount of time on a particular scheduler.
  - debounce - Publishes elements only after a specified time interval elapses between events.
  - throttle - Publishes either the most-recent or first element published by the upstream publisher in the specified time interval.
  - receive(on:) - Specifies the scheduler on which to receive elements from the publisher.
  - subscribe(on:) - Specifies the scheduler on which to perform subscribe, cancel, and request operations.
Subscribers
- Receive published values
- Two associated types: Input and Failure
- receive(subscription:) - called exactly once
- Publisher provides zero or more values to Subscriber
- Publisher sends exactly one completion with two possible outcomes:
  - Publisher has finished
  - Failure has arisen
  - No further values can be emitted after completion signal
- assign(to:on:) - assigns emitted values to the specified key paths on the instance of an object
- sink - closure which gets called when new value is received
AnyCancellable
- Combine allows terminating subscription to free up resources associated with it
- cancel() - cancels the activity
Subjects
- Behave like both Publisher and Subscriber
- send(_:) - sends a value to the subscriber
- Two useful types:
  - PassthroughSubject - stores no values, you will receive values emitted after subscribing
  - CurrentValueSubject - maintains a history of the last value it received
SwiftUI BindableObject
- SwiftUI owns the Subscriber
- BindableObject has a single associated type which is constrained to Publisher protocol with Failure equal to Never
- didChange property notified when your type has changed
- Further described in Data Flow Through SwiftUI
Integrating Combine
- By adding @Published (property wrapper) we can add a Publisher to any property, for example: @Published var password: String = ""
- Using targetAction you can assign new values to the @Published value: password = sender.text ?? ""
- CombineLatest - combines the most recently emitted values
  - using map you can modify received values and change Publisher's type
- eraseToAnyPublisher() - returns an AnyPublisher of the given type and error
- debounce(for:scheduler:) - lets you specify a window by which you’d like to receive values and not receive them faster than that. It is useful to reduce number of requests when filtering data using text fields
- removeDuplicates() - Publishes only elements that don’t match the previous element
Future
- initialized with closure that takes a promise (closure that accepts either success or failure)

Adopting Swift Packages in Xcode

https://developer.apple.com/wwdc19/408

Swift Packages
- Manage dependencies with Semantic Versioning
- Are suitable for private coorporate or public open source framworks of any size
- Are used to share code among multiple apps
- Can be created directly via the UI of Xcode 11
Using an Open Source Package
- Add via File > Swift Packages > Add Package Dependency...
- If GitHub account added to Xcode the dialog shows all of your and all starred repos plus you can enter an URL
- After that Xcode shows the dependency in the Project Navigator on the left and it can be imported via import <Framework>
- Xcode creates a new subdirectory <project_workspace>/xcshareddata/swiftlpm, with a Package.resolved file, which contains the version information of your dependencies
- Package.resolved should be comitted so team members use the same versions
A Closer Look at Packages
- Contains the files/directories Package.swift (Swift package manifest), Sources and Tests
- The manifest file describes the framework name, (test-) targets and dependencies
- Specify compatible language versions using swiftLanguageVerisons = [.v4, .v4_2, .v5]
- See Package Manager Docs for an example manifest file
Package Resolution in more Detail
- Specify version 2.0.0 - up to next major to let Xcode update the highest version available up to but excluding 3.0.0
- Always create a direct dependency to a sub dependency B of a framework A -> B when you want to use functionality from B
- Update packages via File > Swift Packages > Update To Latest Package Versions
Resolving Package Conflicts
- You can only have one version of a package in a workspace
- Examine your package A sub-dependency B requirements if creating a direct dependency to it produces a resolution error
- Upgrade A to a higher version when it supports the desired version of sub dependency B

Creating Swift Packages

https://developer.apple.com/wwdc19/410

Creating Local Packages
- Think of them as sub-project in workspace
- Platform independent by nature
- Great for collecting reusable code
- Not versioned - but can be published/versioned in just a feww steps
- File > New > Swift Package and add the package to your project and a root group
- Link with your app by adding the package product to Project > App Target > General > Frameworks, Libraries and Embedded Content
Publishing Packages
- First Semantic Versioning is explained
- During testing apps a pre-release ID (5.0.0.-beta.1) can be added to your version and should be removed once done with testing
- Provide a thoughtful README.md
- Make sure your tests are green
- Create a local Git repository via Source Control > Create Git Repositories...
- Publish your repository remote by SCM Navigator > ⌥ + Click Your Project > Create <project_name> Remote...
- Select Private if you want to share the package only within your team
- Publish your first version by SCM Navigator > ⌥ + Click Your Project > Tag <branch>.... Provide verison and message. After that push your tag to your remote.
- Read Adopting Swift Packages in Xcode for how to integrate remote packages
Package Manifest API
- Build via Swift language
- Documentation can be found when ⌘ + clicking the PackageDescription of each Package.swift
- swift-tools-version:5.1 specifies the Minimum Compiler Version, Manifest API Version and is used for Dependency Resolution
- Package(name: "") describes the package name
- Package(targets: [.target(""), .testTarget(name: "", dependencies: [])]) describes the targets of your package including test targets and the targets each test target is testing (dependencies)
- Package(products: [.library(name: "", targets: [])]) describes the outcome of your package, in this case a library
- Package(platforms: [.iOS(.v13)]) describes the minimum deployment version for your package
- Describe package dependencies using an url and a version parameter
- For version use .from: "2.0.0", .upToNextMajor: "2.0.0", .upToNextMinor: "2.0.0", .exact: "2.0.0", .branch: "master", .revision: "85cfe06"
- Adopting SMP for existing frameworks by just adding and configuring a Package.swift file
- Include path in .target(...) if your source is not located under the standard sources directory
Editing Packages
- Package dependencies are locked for editing since they're managed by Xcode
- Checkout package as standalone project and drag & drop it to project. This automatically replaces remote package dependency and makes the package editable (see local package above).
- This mechanism can also be used to edit packages you don't own
SPM Open Source Project
- SPM is platform independent
- Use swift package to perform various non-build operations on a package
- Use xcodebuild to build packages on command line and CI
- SPM is build on top of libSwiftPM
- SPM Website
- Getting Help
- Submitting Issues

What’s New in MapKit and MapKit JS

https://developer.apple.com/wwdc19/236

New ground-up rebuilt basemap, data from 4 million miles of road (car fleet/planes)
Improved address detail, more accurate search and directions
New map available in US end of 2019, additional countries 2020
Automatically replaces old Map in your app once released
Better support for decoding and presenting GeoJSON using MKGeoJSONDecoder and MKGeoJSONObject
Everything mentioned below is also supported in some way in MapKit JS
Snapshots
- Snapshots (non-interactive images of map for native apps) can now be done on web
- Example: snapshot.apple-mapkit.com/api/v1/snapshot?center=37.78,-122.42&size=640x300
- Add param "&colorScheme=dark" for darkmode snapshot
- Snapshot urls require param "&signature=", signature obtained from MapKit JS API
- Can request 25,000 snapshots per day
- Snapshot helper tool availble at developer.apple.com/mapkitjs
Dark Mode
- MapView in your app will automatically adapt to dark mode from view hierarchy trait
- MKMapSnapshotter for non-interactive maps in your app, is not aware of view hierarchy so you must pass the view's traitCollection with:
- let options = MKMapSnapshotter.Options()
- options.traitCollection = myView.traitCollection
Filtering points of interest in map views
- Previously you had to remove all points of interest if any were conflicting with your annotations, now you can filter out/in specific MKPointOfInterestCategory, such as .restaurant or .hotel
- example: mapView.pointOfInterestFilter = MKPointOfInterestFilter(excluding: [ .hotel ])
Filtering search and autocompletion results
- MKLocalSearchCompleter feeds your search bar autocomplete and auto-suggestions with results show on map using MKLocalSearch
- Now you can apply MKPointOfInterestFilter to both MKLocalSearchCompleter and MKLocalSearch
- MKLocalSearchCompleter now also has ResultType, which can be .address, .pointOfInterest (Disneyland), .queries (coffee)
- MKLocalSearch handles ResultType as well, but not .queries type
- MKMapItem now has a pointOfInterestCategory property
Improving overlay performance
- MKMultiPolygon and MKMultiPolyline for adding large groups of same-style shapes and lines to map overlay for performance efficiency
- Use MKMultiPolygonRenderer or MKMultiPolyLineRenderer to add fill, stroke, lineWidth in mapView:rendererFor delegate method
- MapKit now renders these as vector graphics, rather than bitmaps (can be turned off)
Taking control of the map view camera
- Add MKMapView.CameraBoundary to constrain the map view's center point and restrict panning
- Boundary strictly enforced, even setting region programmatically will not move the camera outside of the bounds
- Center Coordinate Distance of camera is the "distance up" above the map, should not be treated as altitude, as the camera can change it's pitch and the distance is the same but the altitude is different
- MKMapView.CameraZoomRange can constrain zoom distance

App Distribution – From Ad-hoc to Enterprise

https://developer.apple.com/wwdc19/304

Distribution methods
- Ad Hoc
- App Store
- In-House
- Custom apps
Personal Team
- Intended for students, teachers and getting started
- Apps will be deployed to devices you own
- Intended for a few apps and a few devices
- Certain capabilities are not available with free accounts (ex: CloudKit, Siri)
- Apps cannot be distributed on the App Store
Ad Hoc Distribution
- Distribute your app to testers on registered devices
- 100 devices, per product family, per year can be registered
- Meant for testing apps on registered devices
- Short term distribution solution
- Apps expire and will eventually stop working
- Device limits reset once per year 
TestFlight
- 25 internal testers
- 10k testers via email or a public link
- Builds are active for 90 days
- Builds for external testers go through beta app review
- Available to members of the Apple Developer Program only
App Store
- Individuals or organizations can sign up
- Apps are available to the general public
- Apps are hosted on the App Store
- Apps are reviewed by Apple
- Know and follow the App Store Review Guidelines
- Keep the app current
- Make sure the app is appropriate for the general public
In-House
- Proprietary apps build by internal developers for their employees
- Organization owns and maintains the source code
- Distribution is outside of the store
- Organization distributes the app, usually via MDM
- Users must be employees
- Distribution certificates should be protected
- Certificates lifecycle needs to be managed
- DIY Beta Testing and Hosting
- Apps require periodic access to the internet
Custom App
- Part of the Apple Developer Program
- Privately offer customized apps to customers you identify for yourself
- Apps for partners, clients, franchisees, internal employees and affiliates
- Distribute licenses via MDM or Redemption codes
Benefits of Custom Apps
- One program to manage all of your apps (internal and external)
- Apps don’t expire
- Apps are managed individually
- Apps can be distributed to much larger audiences
- Easier to work with third-party software vendors
- Additional App Store features, TestFlight and App Store Connect tools
Custom Apps Expectations
- Customer is required to have Apple Business Manager
- Apps have to support the countries they will be distributed in
- Redemption codes will not be made publicly available
- Reviewers need to access the full functionality of the app
- Once submitted, apps can’t be moved between public and private availability