Privacy and Security

How is Apple ensuring models are not hijacked through prompt injection techniques?

• Indirect prompt injection involves an attacker sending malicious instructions within a document that an agentic tool might read, causing the model to perform unintended actions.
• Models are most at risk when they have access to private data, can perform actions, and have access to untrusted context.
• Mitigation strategies include deterministic methods like confirmation prompts and probabilistic methods like highlighting untrusted content.
• A WWDC session on securing agentic features provides further details.

What happens to data sent to Private Cloud Compute or a third-party model?

• For third-party AI APIs (e.g., Anthropic, Google), Apple's PCC guarantees do not apply. Developers must review those providers' terms.
• When using Apple's PCC interfaces, data is processed with stateless computation and non-targetability guarantees.
• Data sent to PCC is only used to fulfill the request and is not accessible by Apple or anyone else.
• Data is removed from PCC nodes once the request is complete, and nodes are wiped clean upon reboot.
• PCC guarantees are cryptographically provable and verifiable through documentation and tooling.

What distinction does Apple draw between developer-collected data and third-party processor data?

• Developers are responsible for declaring all data collected from their app, regardless of whether it's sent to their own servers or a third-party service.
• This responsibility is tied to App Store "Nutrition Labels," where developers must accurately represent their app's privacy practices.
• Developers must understand what third parties do with the data and their specific uses to correctly inform users.

What security risks are there when using the new agentic coding features in Xcode, and what are the best practices to mitigate them?

• Risks are similar to prompt injection vulnerabilities, requiring a secure-by-design approach.
• Xcode features include additional mitigations like allow-listing common tools that Xcode can call.
• Each use case is evaluated to determine the appropriate set of mitigations for the user experience.
• Security is a primary consideration during the design and development of these features before release.

What is the core architecture behind Private Cloud Compute (PCC) for cloud AI processing, and how does Apple mathematically ensure no one can see this data?

• PCC offers verifiable privacy and transparency, going beyond traditional confidential compute with additional security layers.
• For PCC infrastructure running on Google Cloud, two verifiable attestations for every piece of hardware from two separate vendors are required.
• Complex data parsing (e.g., images) is isolated into separate nodes with extra sandboxing.
• PCC systems are hardened operating systems purpose-built for safe and isolated user data processing.
• A virtual research environment is provided, allowing independent researchers to verify Apple's claims by running the same software on their own machines.
• This research environment is eligible for Apple's security bounty program and is publicly available.
• Continuous transparency is a key aspect; backend changes are noticeable and verifiable in real-time.
• Compromised nodes will have changed attestations, causing devices to automatically reject them.

Are there new features for auditing an app's privacy and security posture?

• AI models can be used as tools to review an app's privacy and security guarantees and check expected behavior.
• Developers should leverage security and privacy features like out-of-process pickers to help build secure apps.
• WWDC sessions on security and privacy offer valuable guidance.
• Frameworks like Network Framework, when using TLS, default to the latest post-quantum cipher suites for enhanced security.
• CloudKit provides a secure way to sync user data between devices without requiring developers to manage their own servers.
• Developers are responsible for the privacy and security of their users' data.
• Focus on inputs to the application and where user data is vended to others.

What privacy and security tools should new Apple platform developers learn first?

• Privacy:
  • CloudKit for syncing data between devices.
  • Keychain for secure on-device storage.
  • CryptoKit library for cryptographic operations.
  • Encourage users to adopt Passkeys.
  • Practice data minimization: consider if access to user data is truly necessary.
  • Use out-of-process pickers for data like photos and contacts to allow user selection.
  • For location data, consider if only one-time access is needed.
• Security:
  • Utilize data protection classes for Keychain and file storage to classify data sensitivity (e.g., "available after first unlock").
  • Leverage platform-native methods like CloudKit APIs for secure data handling.
  • Focus on inputs to the application and how much trust is placed in them.
  • Consider where user information is vended to others.
• Telemetry:
  • Start with data minimization and define the specific question developers need to answer.
  • Ensure telemetry is actionable (e.g., leads to code changes or business decisions).
  • Communicate to users what data is being used and why.
  • Consider identifiers: rotating, session-based, or device-based.
  • Employ strategies like bucketing and de-identifying data.
  • Explore Private Federated Learning and Differential Privacy for collecting sensitive data with mathematical guarantees of non-identifiability.
• Passkeys:
  • Adopt Passkeys from the start for user accounts, as they are designed to replace passwords and are phishing-resistant.

If a user stores highly personal journal entries, readings, or history in Swift data, what are Apple's recommended approaches for protecting that data while still enabling search and synchronization?

• On-device protection:
  • Use data protection classes for Keychain keys (e.g., "only when unlocked," "only when biometrically unlocked").
  • Set data protection classes for files on disk (e.g., "only when device is unlocked").
• Synchronization with CloudKit:
  • Use CKRecord.encryptedValues for end-to-end encryption, especially for users with Advanced Data Protection.
• Enabling search and indexing:
  • Keys may remain viable for some period of hours after device lock, allowing background indexing — the exact API name and configurability are unclear.
  • After this period, the key locks and the data becomes inaccessible.
• Example: Contacts data protection class ensures data is only available after the phone is fully unlocked with a passcode.

With the Xcode AI coding features, can Xcode be run inside a virtual machine on an air-gapped Mac walled off from the internet to help improve security and privacy?

• Existing code completion features are entirely on-device and work offline.
• Other AI features, particularly those integrating with third-party coding tools or providers (e.g., Claude Code or OpenAI Codex), will require an internet connection.
• The ability to run in an air-gapped environment depends on the specific AI feature being used.

How does the new Siri AI ensure privacy and security?

• Siri AI leverages Private Cloud Compute (PCC) for processing requests that require larger models hosted in the cloud.
• On-device protections, including entitlements and sandboxing, ensure that only authorized processes access user data before sending it to PCC.
• A secure daemon processes prompts and prepares them for PCC.
• When interacting with third-party apps, data is minimized and only the data relevant to the query is sent.
• User choice is respected; for example, users can control whether Siri uses their location.
• Apple prioritizes on-device processing, only using PCC when necessary for large foundation models or operations too complex for the device.
• Multi-turn conversations may involve back-and-forth communication with PCC, with on-device protections still applying.
• Apple Silicon enables significant on-device processing, creating security boundaries where platform technologies can be applied.
• Optimization of models to run on-device by the AI/ML and Silicon teams enhances privacy and reduces reliance on PCC.
• Developers can also use the Core AI framework to run custom on-device models with platform guarantees.

What is TCC (Transparency, Consent, and Control)?

• TCC refers to frameworks and protections on-device that allow apps to request user access to additional data.
• Developers use the TCC framework to prompt users for access to data like photos or microphone access.
• Users can grant or deny these requests, ensuring apps only access data with explicit permission.
• This provides users with a high level of trust that an app does not have unauthorized access to their sensitive data.

Will Siri AI have a new configurable policy in an MDM solution when released?

• Yes, existing MDM controls for Siri will continue to apply.
• Enterprises can configure how they want to use Siri AI features, potentially selecting specific functionalities.
• Safari automatic password-changing was also mentioned as an enterprise enabler for small businesses.
• Developers can submit feedback requests for proposed MDM configurations.

Are there any new enhancements to Safari's Intelligent Tracking Prevention (ITP) on the new OSs?

• Apple continues to invest in and evolve ITP.
• While there are no specific new enhancements to reference this year, ITP remains an important technology.
• Further details can be found on WebKit.org.

How can apps communicate security and privacy without sounding like marketing?

• App Store Nutrition Labels: Provide an easy way for users to understand an app's privacy practices before downloading.
• In-App Transparency: Allow users to see what data the app has collected about them.
• Privacy Assurances: Define simple, understandable privacy promises for each feature that act as a guiding principle for development.
• Consistency: Continuously demonstrate commitment to privacy and security through ongoing development and clear communication.
• Security Story: Clearly articulate the security claims and guarantees for features (e.g., Memory Integrity Enforcement, PQ3 encryption for iMessage).
• User Experience: Build trust by providing features like Passkeys or Sign in with Apple, and by only requesting necessary user data.
• API Usage: Use platform APIs like Accessory Setup Kit instead of broader permissions (e.g., full Bluetooth access) to provide a better user experience and limit data collection.
• Avoid Vague Claims: Do not use terms like "bank-grade" or "military-grade" encryption, as they are often meaningless to users.
• Transparency: Share source code for parts of the system (like PCC) to demonstrate verifiable actions.
• Lean into Apple's Ecosystem: Integrating with Apple technologies and adhering to the Apple experience can convey a sense of trust.
• Contextual Permissions: Request permissions (e.g., camera, microphone) only when the feature requiring them is being used, not at app launch.
• Purpose Strings: Clearly explain why specific data access is needed in purpose strings to help users understand the value exchange.
• Sensor Indicators: Ensure sensor data (microphone, camera) is only used when the user expects it, as indicated by device sensor lights.

What security concepts should students learn first for Apple platform development?

• For Students:
  • Platform Security Guide: Provides in-depth information on data protection, the Secure Enclave, and OS security technologies.
  • security.apple.com: The security blog offers details on the bug bounty program, vulnerability reporting, and in-depth articles on security technologies.
• Low-Level Security:
  • Binary exploitation and reverse engineering.
  • Understanding how systems work is crucial for both attacking and defending.
• Networking:
  • Focus on protocols like TLS and how the internet functions.
• General Concepts:
  • Understanding how something works is fundamental to securing or attacking it.

How is Apple approaching stale, expired, or no longer valid passkeys in the Passwords app?

• The new Signal API will notify apps and websites when changes occur, prompting them to update credentials.
• Passkeys are part of a wide standard, and Apple, as part of the governing body, works with other companies to ensure cross-platform syncing and lifecycle management.

Does Apple recommend encrypting secrets while resident in RAM, or are platform protections generally considered sufficient?

• macOS:
  • Enable Hardened Runtime: Prevents other applications from reading the application's memory and mapping malicious code into the process.
  • This protection is deeply integrated into the operating system.
• Encryption vs. Destruction:
  • For secrets needed only briefly in memory, destruction is the answer, not encryption. Destroy tokens once used.
  • For longer-term secrets, consider DRM-level solutions, but pure security may not be the primary focus.
• CryptoKit:
  • Use CryptoKit APIs for handling keys and other sensitive data in memory.
  • CryptoKit ensures data backing keys is zeroized. Avoid custom memory management that might not zeroize correctly.
• Secure Enclave:
  • Bind keys to the Secure Enclave hardware. This makes keys non-exportable, even if the device or app is compromised.
  • Use the attestation interface to verify that a key is held by the Secure Enclave.

How should developers balance privacy with useful telemetry?

• Great Features and Great Privacy: Aim for both, requiring creativity and work.
• Privacy Pillars:
  • Data Minimization: Collect only the necessary data to answer a specific question. Avoid collecting full-grain timestamps if daily or weekly aggregation suffices.
  • On-Device Processing: Perform aggregation and data minimization on-device before sending data to servers.
  • Transparency and Control: Ensure users have control over data collection and are shown what data is being collected.
  • Security:
    • Technically enforce user choices regarding data collection.
    • Secure collected data against attackers.
    • Implement data deletion policies.
• Aggregation: Distill data to its most essential form to protect privacy and reduce network bandwidth.
• Telemetry Example: For alarm telemetry, collecting only whether an alarm went off is sufficient, rather than precise wake-up times, which could reveal personal patterns.
• User Control: Users should have ultimate control over their data.
• Actionable Insights: Telemetry should lead to concrete actions, like code changes or business decisions.