Practical On-Device AI: Building Conversational iOS Features with Foundation Models

Apple released the Foundation Models framework at WWDC 25. This framework gives developers access to the on-device LLM that powers Apple Intelligence. Developers can now use the capabilities of this LLM to empower different capabilities within their own applications. Foundation Models framework has the potential to revolutionize iOS development — not only by enabling a new generation of intelligent, feature-rich applications, but also by significantly enhancing developer productivity. By bringing advanced on-device AI capabilities directly into the iOS ecosystem, it empowers developers to build smarter apps faster, with less dependency on external infrastructure. While this framework is new and Apple will enhance its capabilities each year, it still has many features that are useful to change the mobile app landscape.

There are many features that can now be added to iOS apps with smaller developer effort but elevated user experience. Foundation Models framework capabilities that Apple highlighted at WWDC are not just limited to content generation but can also be used to gather data insight that can lead to superior user experiences like-

• Personalization
• Chat bot
• Semantic search
• Generative quizzes

In this article, I will showcase how I used Foundation Models framework to add a virtual assistant feature within a sports-based app. This feature provides sports enthusiasts access to historic and current game statistics without consuming precious real estate on an app while keeping users engaged. For this sample, I used following capabilities of the Foundation Models framework-

• Topic detection: Classify free form text to generate right response to user query
• Tag generation: detect tags from free text conversation to generate accurate response
• Summarization: Summarize large dataset into a user friendly (ish) format.
• Dialog generation: generate interesting dialogs to keep users engaged

Why Foundation Models Framework?

As AI becomes embedded in everyday digital interactions, users increasingly expect intelligent, contextual features in the mobile apps they rely on. Rather than depending solely on cloud-based large language models (LLMs), Apple’s new Foundation Models framework in iOS 26 introduces a powerful on-device alternative that delivers speed, privacy, and cost efficiency. This shift opens the door for developers to build richer, AI-enhanced experiences without the traditional barriers of infrastructure complexity or operational expense.

One of the most impactful advantages of the Foundation Models framework is its zero-operating cost for inference. Since the LLM runs entirely on the device, developers avoid per-token usage fees associated with cloud models. Apple enforces a token limit per session to control compute usage but also provides patterns for chaining multiple sessions when handling larger queries. For organizations exploring AI features but concerned about variable LLM costs, this cloud model-free approach significantly reduces risk.

Another major benefit is the ability to access the same embedded LLM that powers Apple Intelligence — without bundling large models inside the app package. This keeps app sizes lean while giving developers instant access to a periodically updated model maintained directly by Apple. As Apple continues evolving its on-device intelligence, apps automatically gain improvements with every OS update, ensuring performance, superior user experience while empowering more use cases.

With AI running locally, apps also gain near real-time responsiveness, eliminating latency from server roundtrips. This consistency improves user trust and makes conversational or generative features feel fluid and immediate. Additionally, since no user data is transmitted to external servers, the framework delivers strong privacy and security guarantees, making on-device AI suitable for sensitive use cases and boosting user confidence in AI-driven features.

Overall, Apple’s Foundation Models framework makes it easier for teams to experiment with AI-driven features without taking on the usual cost, performance, or privacy trade-offs. By running intelligence directly on the device, it opens new opportunities to build faster, more responsive, and more trustworthy experiences. As the framework continues to evolve, it is an exciting space for engineers to explore — especially for use cases where low latency, strong privacy, and predictable costs really matter.

App Introduction & Setup

Sports mobile apps often struggle to present extensive event data, sport rules, and stats within limited screen space. By introducing a virtual assistant, apps can offer a more engaging, conversational way for fans to get the information they need — reducing interface clutter and simplifying navigation. In the sample Olympics app I will walk through, I implemented a virtual assistant that allows users to quickly retrieve results from recently completed sporting events or inquire about rules for a sport, demonstrating how conversational design can elevate the fan experience.

The concept behind this sample app is straightforward: historical sports results for an event like the Olympics can be packaged as lightweight JSON files embedded directly within the application. These files act as an in-memory data store, eliminating the need for external API calls and ensuring fast, offline-ready access to results. A virtual assistant then enables users to query this data naturally through conversational prompts. With the Foundation Models framework, free-text input can be interpreted in multiple ways — allowing the assistant to extract intent, identify relevant attributes, and return accurate results. Before diving into these use cases, let us look at the structure of the data itself. We can also rely solely on knowledge of this on device LLM for finding information about rules for common sporting events.

Below is a sample JSON snippet representing an event and its results. This schema forms the basis of the embedded dataset that powers result retrieval within the app:

{ 

  "sport": "Archery", 

  "gender": "Men", 

  "event": "Individual", 

  "gold_medalist": { 

    "name": “Gold Winner”, 

    "country": “ABC” 

  }, 

  "silver_medalist": { 

    "name": “Silver Winner“, 

    "country": “DEF” 

  }, 

  "bronze_medalist": { 

    "name": “Bronze Winner”, 

    "country": “GHJ” 

  }, 

  "placements": [ 

    { "rank": 4, "name": "Fourth Ranker”, "country": "XYZ” }, 

    { "rank": 5, "name": “Fifth Ranker”, "country": “XYZ” }, 

    { "rank": 6, "name": “Sixth Ranker”, "country": “XYZ” }, 

    { "rank": 7, "name": “Seventh Ranker”, "country": “XYZ” } 

  ] 

} 

Here are the data models that I use to support this data-

@Generable(description: "Information About Athlete - name and what country they represent") 

struct Athlete: Codable { 

    let name: String 

    let country: String 

} 

@Generable(description: "Information About placement of an athlete in an event") 

struct Placement: Codable { 

    let rank: Int 

    let name: String 

    let country: String 

} 

@Generable(description: "Information About result of an olympic event") 

struct OlympicResult: Codable, Identifiable { 

    let id = UUID() 

    let sport: String 

    @Guide(.anyOf(["Men", "Women"])) 

    let gender: String? 

    let event: String 

    let goldMedalist: Athlete 

    let silverMedalist: Athlete 

    let bronzeMedalist: Athlete 

    let placements: [Placement]? 


    private enum CodingKeys: String, CodingKey { 

        case sport, gender, event 

        case goldMedalist = "gold_medalist" 

        case silverMedalist = "silver_medalist" 

        case bronzeMedalist = "bronze_medalist" 

        case placements 

    } 

} 

Foundation Models Setup & Configuration

As a first step, I need to initialize the SystemLanguageModel and configure it. I have encapsulated this within my data service class as shown in the code snippet below-

@Observable 

class VirtualAssistantDataService { 

    var results: [OlympicResult] = [ ] //1 

    private var contentTaggingModel = SystemLanguageModel(useCase: .contentTagging) //2 

    private var generalModel = SystemLanguageModel(useCase: .general) //3 

    private var session: LanguageModelSession? //4 

    private var genericSession: LanguageModelSession? //5 

    var isContentTaggingModelAvailable: Bool {    //6 

        contentTaggingModel.availability == .available 

    } 

    var isGenericModelAvailable: Bool {     //7 

        generalModel.availability == .available 

    } 

Let’s walk through each line of code here-

• 1: Empty array to capture results
• 2: Here I initialized the on-device LLM for content tagging use case. Specifying use cases in some cases like this is essential.
• 3: Here I initialized the on-device LLM for general use case so that I can use this model for all other non-specific cases.
• 4 & 5: Variables for storing LanguageModelSession. As per Apple documentation, session is a single context that you use to generate content with and maintains state between requests.
• 6 & 7: It’s a good coding practice to check if the on-device models are available. On device LLMs are only available on devices where Apple Intelligence is supported or enabled in settings.

High Level Data Flow

The following diagram shows the high-level data flow between user and various parts of the mobile application-

High Level Data Flow

In the next few sections, let us walk through each of these flows in detail.

Topic Detection

Users of my app can ask the virtual assistant either about historical stats (result of a past sporting event) or they might be interested in learning about a sport and hence can ask about the rules. The virtual assistant should be able to find what the user is trying to ask before it can answer the question. We can use Foundation Models’ data classification capability to find out if the user is trying to learn about a sport or check on an event result. Following code snippet shows how we can achieve this-

static let modelInstructions = """ 

    You are a friendly Olympics assistant. Your job is to provide right answers to only things related to sporting events in Olympics 

    """ 

@Generable 

enum questionType: Codable { 

    case rule 

    case medals 

    case other 

} 

@Generable 

struct Category: Codable { 

    let questionCategory: questionType 

} 

  func questionCategory(message: String) async -> Category { 

        if isGenericModelAvailable { 

            genericSession = LanguageModelSession(instructions: modelInstructions) 

            do { 

                let response = try await genericSession!.respond( 

                    to: "Classify \(message) into one of the following categories - question about rules of a sport, question about a sporting event result or others", 

                    generating: Category.self 

                ) 

                return response.content 

            } 

            catch (let error) { 

                print(error.localizedDescription) 

                return Category(questionCategory: .other) 

            } 

        } else { 

            return Category(questionCategory: .other) 

        } 

    } 

Because Category (and its nested types) conforms to @Generable, the framework can decode the output directly into your strongly typed Swift data models. That means you get structured results without writing manual parsing, regex, or brittle string matching. In practice, this reduces glue code and makes downstream logic safer and easier to maintain. This is a powerful feature of the Foundation Models framework.

The other important piece here is the session-level instructions (modelInstructions):

• They act as guardrails. By constraining the assistant to Olympics sporting-event topics, you reduce off-scope responses and keep behavior predictable.
• They improve efficiency. When paired with lightweight classification, you can route “other” questions away from expensive workflows — saving tokens and avoiding unnecessary token generation.
• They improve UX. Clear scoping helps steer users toward what the assistant can do well, rather than producing vague or incorrect answers outside its intended domain.

Tag Generation

Once the virtual assistant categorizes a user’s free-text question, the next step is to determine the specific information being requested so that it can generate an accurate and relevant response. To achieve this, I adopted a tag-extraction strategy that identifies key entities and concepts from the user’s input. In the code below, I initialized an instance of model with a specific use case of content tags. I could then open a session with instructions to extract specific type of tags from the user’s query. The extracted tags are then used as parameters to filter and retrieve the appropriate data from the locally embedded JSON-decoded dataset which we discussed in section — App Introduction & Setup.

This approach is not limited to local data and can be applied equally well when querying remote data sources, such as APIs or search services. By using tags as an intermediate representation, the assistant remains flexible, scalable, and easy to extend as new data sources are introduced. Below is the code snippet used to initialize session with right instructions:

static let contentTaggingInstructions = """ 

    Find tags related to sports, events like singles or doubles, and gender. 

    """ 



  if isContentTaggingModelInitialized { 

            session = LanguageModelSession( 

                instructions:  contentTaggingInstructions) 

… 

Here I am checking if the LanguageSystemModel with use case of content tagging is downloaded and available before opening a session. I have provided specific instructions to the model for extracting tags related to sport events, type of event and gender. It is critical to provide the right instructions and be clear but concise on what kind of tags you are expecting the model to extract. Following function then uses this session to extract tags from the user’s question:

private func getContentTags(query: String) async throws -> ContentTaggingResult { 

        do { 

            let response = try await session!.respond( 

                to: query, 

                generating: ContentTaggingResult.self 

            ) 

            return response.content 

        } 

        catch (let error) { 

            print(error.localizedDescription) 

            throw ModelError.unknown 

        } 

    } 

Here I am using the session opened previously to respond to a user input free text (query parameter above) to generate tags and decode them into following struct-

@Generable 

struct ContentTaggingResult { 

    @Guide( 

        description: "Most important sports related key points in the input text.", 

        .maximumCount(2) 
    ) 

    let topics: [String] 

    @Guide( 

        description: "Gender in the input text." 

    ) 

    let gender: [String] 

} 

Topics property captures the specific sports event details and gender is another information that the model tries to capture from the user input. Let’s take an example of a user input and what tags/topics are generated by it-

XCode Debug Log

The results above show that the Foundation Models were able to identify the right topics and gender from user input. I have used @Guide attribute on properties for better context. I can use the same methodology as above to generate appropriate tags when the user asks questions about rules for a sport.

With the relevant topics extracted from the user’s input, the next step is to use those tags to query the local dataset. In this example, the app loads a bundled JSON file containing historical sports data, decodes it into strongly typed models, and filters the results based on the extracted keywords and gender-

private func processQuery(keywords: [String], gender: [String]) async -> String { 

        guard let url = Bundle.main.url(forResource: “sportsdata", withExtension: "json") else { 

            return "Could not find JSON file" 

        } 

        do { 

            let data = try Data(contentsOf: url) 

            let decoder = JSONDecoder() 

            let results = try decoder.decode([OlympicResult].self, from: data) 

            let keywords_lowercased = keywords.map { $0.lowercased() } 

            let filteredResults = results.filter { result in 

                let fields = [result.sport, result.event].map { $0.lowercased() } 

                return keywords_lowercased.contains { keyword in 

                    fields.contains(where: { field in keyword.contains(field) }) 

                } 

            } 

.....

return generateStringResponse(for: sportsWithGender[0]) 

} 

Once a matching result is found, it’s passed to a helper function that translates the raw data into a user-friendly response. That response is then rendered in a chat-style UI, keeping the interaction natural and conversational.

Summarization

In earlier sections, the assistant handled highly specific user queries that mapped cleanly to a single result in the local dataset. However, real-world usage often includes broader or ambiguous questions that can return multiple relevant results. In these cases, presenting a concise, user-friendly summary becomes essential.

For example, a query like “Who won men’s badminton?” can return results for both singles and doubles events. Rather than listing raw records, the assistant generates a readable, news-style summary that consolidates the outcomes into a single response.

Below is the function used to generate this summary:

private func generateSummary(data: [OlympicResult]) async throws -> String { 

        if isGenericModelInitialized { 

            genericSession = LanguageModelSession( 

                instructions:  "Generate a textual and user friendly news like summary of all the sport events and their results where events match closest to user query.") 

            do { 

                let response = try await genericSession!.respond( 

                    to: "generate a textual and user friendly news like summary of this data \(data)", 

                    generating: String.self 

                ) 

                return response.content 

            } 

            catch (let error) { 

                throw ModelError.unknown 

            } 

        } else { 

            return “There is an error” 

        } 

    } 

Once again, model instructions play a critical role in shaping the output. This approach works well when the result set is relatively small (e.g., two to three items). However, with larger datasets, the model sometimes defaults to returning a JSON-style summary, which is not ideal for end-user display. This is an area for further experimentation, particularly around prompt refinement and output formatting strategies.

Dialog Generation

To make interactions with the virtual assistant more engaging, I also added support for follow-up dialog generation. After answering a user’s question, the assistant can proactively prompt the user with a relevant next question, creating a more conversational experience. The following prompt and model were used to generate follow-up dialog:

static let promptNextQuestion = "Generate a dialog asking next question after answering first question around Olympics stats" 

@Generable 

struct Prompt: Codable { 

    let name: String 

    let response: String 

} 

  let response = try await genericSession!.respond( 

                    to: promptNextQuestion, 

                    generating: Prompt.self 

                ) 
.....

This same mechanism can be easily repurposed to create more interactive features, such as quizzes. For example, the following prompt enables quiz-style interactions:

static let quizQuestion = "Quiz me about Olympics" 

By reusing a consistent dialog-generation pattern, the assistant remains flexible and extensible, allowing new conversational experiences to be added with minimal additional logic.

Conclusion

The Foundation Models framework introduces a powerful set of building blocks such as topic detection, tag extraction, summarization, and dialog generation that can be combined to unlock a wide range of new use cases in iOS applications. Rather than being limited to content generation, these capabilities enable more intelligent interpretation of user intent, dynamic data retrieval, and conversational interaction patterns that were previously difficult to implement without significant backend support.

As shown in this example, lightweight techniques like classification and tagging can act as effective orchestration layers, allowing apps to route queries, query structured data, and adapt responses dynamically. Dialog generation further enables more engaging and natural user experiences by supporting follow-up interactions. Together, these patterns open opportunities for building assistants, search experiences, personalization layers, and guided workflows across many domains.

As the Foundation Models framework continues to evolve, adopting these capabilities early will allow teams to experiment with new interaction models, identify high-value use cases, and build internal expertise that can scale alongside Apple’s growing on-device intelligence ecosystem.