[BDD 2025 - Mobile Development] Exploring Apple’s On-Device FoundationModels

BDD
Exploring Apple
On-Device Foundation
Models
Rizal Hilman | Tech Mentor at Apple Developer Academy @Inﬁnite Learning Batam

Rizal Hilman
Experiences:
● Mentor, Apple Developer Academy @Inﬁnite Learning
● Apple Professional Learning Specialist
● SwiftBatam Organizer

Agenda
Meet the FoundationModels
HandsOn with Guided Generation & Tool Calling
Safety & Best Practices

“By 2027, most applications/computer programs will have some kind of
Artiﬁcial Intelligence implementations.”
Research by Gartner, McKinsey & Sopra Steria.

Zero Cost Offline On-Device Privacy Focus

Meet Apple
FoundationModels
01

The Foundation Models framework is Apple’s new framework that gives
you direct access to the on-device large language model used by Apple
Intelligence.
developer.apple.com

What can FoundationModels do?
Summarization
Text Extraction
Classiﬁcation
Tagging
Personalization
Content Generation
Many more..

About FoundationModels
Approximately 3 billion parameters, quantized to 2 bits.
Outperforms the Llama 3.2 3B model.
Below the performance of Qwen 3 4B and Gemma 3 4B models.
A device-scale model, optimized for speciﬁc use cases.
Knowledge cutoff: October 2023.
Supports Guided Generation and Tool Calling.
Custom adapters can be used to train new skills.

Import FoundationModels framework
1
Create a Session
2
Send a Prompts & receive Response using
.respond(to:) or .streamResponse(to:)
3
How to use FoundationModels?

import FoundationModels
let session = LanguageModelSession()
let response = try await session.respond(to: "Tell me a joke about Bandung")
How to generate Complete Response?
Output:
Sure, here's a light-hearted joke about Bandung:
Why did the coffee shop in Bandung open at midnight?
Because they wanted to keep up with the local "bandung" rhythm!☕🌙

import FoundationModels
let session = LanguageModelSession()
let stream = try await session.streamResponse(to: "Tell me joke about Bandung!")
for try await chunk in stream {
print(chunk.content)
}
How to generate Stream Response?
Why did the …
Why did the band stop in …
Why did the band stop in Bandung? …
Output:

Recap: Meet FoundationModels
● The Foundation Models framework provides on-device access to Apple's large
language model.
● The model has about 3B parameters, optimized for device-scale use cases.
● Key beneﬁts: zero cost, offline operation, and a focus on privacy.
● Core capabilities include summarization, text extraction, classiﬁcation, and content
generation.
● The model supports advanced techniques like Guided Generation and Tool Calling.
● You can train new skills with a custom adapter.

Guided Generation &
Tool Calling
02

Problem with Traditional Response
Unpredictability
Parsing Issues
Increase Complexity

What is Guided Generation?
● Guided generation lets you deﬁne a custom Swift type for the desired output.
● When making requests to the Foundation Model, specify this type in your
prompt.
● The framework guarantees that the generated output matches the deﬁned
structure, using constrained decoding to prevent invalid results.

Deﬁne structured output using @Generable
1
Add optional @Guide hints to properties
2
Call Guided Generation using
.respond(... generating:)
3
How to use Guided Generation?

@Generable
struct RecipeDetails {
@Guide(description: "A clear recipe name, maximum 60 characters")
let name: String
@Guide(description: "Cooking time in minutes, between 5 and 240")
let cookingTime: Int
@Guide(description: "Step-by-step cooking instructions")
let instructions: String
}
// Using Generable
let recipe = try await session.respond(to: "Butter chicken biryani",
generating: RecipeDetails.self)

@Generable struct RecipeRecommendation {
@Guide(description: "The name of the dish")
let name: String
@Guide(description: "Brief cooking instructions")
let instructions: String
@Guide(description: "The cuisine type this recipe belongs to")
let cuisine: CuisineType
}
@Generable enum CuisineType {
case italian
case mexican
case asian
case mediterranean
}

@Generable(description: "Metadata describing the recipe properties")
struct RecipeMetadata {
@Guide(description: "Recipe difficulty level", /^(Easy|Medium|Hard)$/)
let difficulty: String
@Guide(description: "Prep time in HH:MM format", /^d{2}:d{2}$/)
let prepTime: String
}
Note: For string properties requiring specific formats, you can use regex patterns.

Recap: Guided Generation
● Guided Generation deﬁnes a custom Swift type for the model's desired
output.
● The framework guarantees output structure using constrained decoding for
reliability.
● This approach solves unpredictability and common parsing issues of raw
model output.
● Implementation uses the @Generable attribute and optional @Guide
property hints.
● You call it with the .respond(to:generating:) method on the
language model session.

developer.apple.com
Limited knowledge
cutoff Oct 2023
󰚁
Don’t have access to
real-world knowledge
󰚁
Can’t perform actions 󰚁

What is Tool Calling?
● Tool calling let’s the model execute function you deﬁne
● Extending its capabilities beyond on the limited knowledge
● The model can access real-world data (e.g: fetch from HealthKit, fetch from
REST API) or perform actions (e.g: creating reminders)

Deﬁne tool as callable functions
1
Add tools to the session
2
How to use Tool Calling?
Model calls tools when needed
3
Return structured data or type to AI
4

Tool Example
struct WeatherTool: Tool {
let name = "weatherTool"
let description = "Provides current weather information for a city"
@Generable
struct Arguments {
var city: String
}
func call(arguments: Arguments) async throws -> Weather {
// fetch weather data logics
return Weather(content)
}
}

Recap: Tool Calling
● Tool calling enables the model to execute developer-deﬁned functions.
● This extends the model's capabilities beyond its limited knowledge base.
● The model can access real-world data or perform external actions.
● Deﬁne functions, add them to the session, and the model calls them when
needed.

● Input ﬁltering
Checks your prompts, instructions, and tool calls for harmful content before they reach
the model.
● Output ﬁltering
Reviews the model’s response after generation to catch anything unsafe that slips
through.
● Model-level safety
Safety training built directly into the model makes it naturally avoid harmful content.
If any of these mechanisms trigger, you’ll get a GenerationError.guardrailViolation.
Built-in Safety Layer

Safety in Instructions
Never include untrusted content in instructions:
// WRONG - Security vulnerability
let unsafeInstructions = """
You are (userRole). Help the user with (userRequest).
"""
// RIGHT - Safe approach
let safeInstructions = """
You are a helpful travel assistant. Help users plan safe, enjoyable trips.
"""
let userPrompt = "I want to plan a trip to (userDestination)

Safety in Instructions
Use clear, ﬁrm language & consider adding negative constraints.
let instructions = """
You are a family-friendly assistant for a children's app.
DO NOT include violence, scary content, or adult themes.
DO NOT use inappropriate language or discuss sensitive topics.
Keep all content appropriate for ages 8-12.
"""

Input Patterns with Risk Mangement
Direct User Input (Highest Risk)
let response = try await session.respond(to: userInput)
let prompt = """
Summarize the following content in a positive, family-friendly way:
Content: (userInput)
Requirements:
- Keep the tone optimistic
- Focus on key highlights only
- Avoid controversial topics
"""
Structured Prompts (Moderate Risk)

Input Patterns with Risk Mangement
let prompts = [
"Generate a happy bedtime story about animals",
"Create a motivational quote for students",
"Suggest a fun family activity for weekends"
]
let selectedPrompt = prompts[userSelection]
let response = try await session.respond(to: selectedPrompt)
Curated Options (Lowest Risk)

// Unclear constraints often produce verbose, unfocused responses
"Summarize this article"
// Length constraints help fit responses into your UI layout
"Summarize this article in exactly two sentences"
// Combined constraints produce more useful, targeted content
"Create a brief product description under 50 words that highlights key features"
Control Output Length
Why?
Token Limit: the context window is limited to 4096 tokens for both input and output combined.
Response Length: limits output size to avoid layout issues and keep the experience consistent.

You are a customer service representative for a fitness app.
Be helpful, encouraging, and focus on solving user problems.
Keep responses professional but friendly.
"""
Specify Roles & Context
Why?
This produces a different output than a generic assistant by providing domain‑specific context and tone
guidance.

// Focused requests produce more accurate results
"Generate five related workout routines for beginners"
// Examples guide the AI toward your desired output format
"Generate five beginner workout routines. Each should be 2-3 words like 'Morning
Yoga' or 'Quick Cardio'"
Write Clear Commands
Why?
Like other language models, Apple's foundation model performs best with clear, specific commands.

You suggest related topics. Examples:
User: "Making homemade bread"
Assistant: 1. Sourdough starter basics 2. Bread flour types 3. Kneading
techniques
User: "iOS development"
Assistant: 1. SwiftUI fundamentals 2. App Store guidelines 3. Xcode debugging
Keep suggestions concise (3-7 words) and naturally related.
"""
Use Examples in Instructions
Why?
Provide a few examples of desired outputs. This helps the model match the desired output format and style

You are a helpful assistant for children's content.
DO NOT include scary or violent content.
DO NOT mention inappropriate topics.
""”
Use Strong Command When Needed
Why?
If you observe unwanted output, use firm constraints.
The model responds reliably to all‑caps "DO NOT" constraints.

● Knowledge is limited with a training cutoff around October 2023.
● Do not rely on the model for factual truth; embed veriﬁed information.
● Avoid using the model for calculator-like precision or complex math.
● For calculations, use dedicated logic outside of the model or use Tool calling.
● Break complex requests into small, concrete pieces for better performance.
Understanding Model Limitations

Recap: Safety & Best Practices
● Built-in safety layers ﬁlter content at the input and output stages.
● Don’t put untrusted content on instructions & consider adding negative constraint.
● If possible, prioritize curated prompt options to help manage content risk.
● Best practices include using clear commands and specifying roles for context.
● Always control the output length to respect UI and token limitations.

developer.apple.com/machine-learning

bit.ly/FMBDD25
More best practices + sample projects:

Resources
● https://developer.apple.com/machine-learning/
● https://developer.apple.com/apple-intelligence/
● https://developer.apple.com/documentation/foundationmodels/
● Book: Exploring FoundationModels by Rudrank Riyam

Thank You
instagram.com/rz.khilman
linkedin.com/in/rizal-hilman
Get in touch
rizal.hilman67@gmail.com

[BDD 2025 - Mobile Development] Exploring Apple’s On-Device FoundationModels

More Related Content

Similar to [BDD 2025 - Mobile Development] Exploring Apple’s On-Device FoundationModels

More from Wintari Yasmin

Recently uploaded

[BDD 2025 - Mobile Development] Exploring Apple’s On-Device FoundationModels