Building Native Windows Apps with WinUI 3 & OpenAI

Welcome! I’m known in the real world as Akilesh, and I’ll be guiding you through building your first Windows Desktop app with OpenAI.

For this tutorial on WinUI3 with OpenAI, we’ll build Podkast AI— a smart podcast tool that transcribes audio, summarizes content, recommends similar podcasts, identifies hosts and speakers, and even generates album art. We’ll harness the power of OpenAI’s models using Betalgo’s OpenAI library for C# while creating a sleek, native Windows interface with WinUI 3. The app’s design is powered by the fantastic WinUI Gallery app on the Microsoft Store, which is a great source for modern UI components and layout snippets.

Podkast AI — Podcast Summarization Tool

Introduction: What’s Podkast All About?

Podkast is an all-in-one podcast tool designed to transform how you experience audio content. The key features include:

• Audio Transcription: Converts spoken words into text.
• Content Summarization: Condenses lengthy podcasts into bite-sized summaries.
• Podcast Recommendations: Suggests similar podcasts based on content analysis.
• Host & Speaker Identification: Extracts names of the people featured in the podcast.
• Album Art Generation: Creates eye-catching visuals using descriptive prompts.

Behind the scenes, Podkast uses multiple AI agents that share a common conversation context. This shared context allows the app to build on previous interactions — meaning that each task (transcription, summarization, etc.) is aware of what happened before, resulting in a coherent and intelligent overall experience.

WinUI 3 and Its Advantages

WinUI 3 is Microsoft’s modern UI framework for building native Windows applications. Its declarative XAML syntax makes designing a clean, responsive UI straightforward. By building directly on Windows’ native APIs, WinUI 3 delivers high performance and a look that fits perfectly with the Windows ecosystem. The design elements from the WinUI Gallery app offer great inspiration, demonstrating a variety of controls and layouts that you can adapt for your own projects.

WinUI 3 reduces the overhead commonly found in cross-platform frameworks — there’s no need for extra layers of abstraction, and you get immediate access to all the latest Windows features. This makes it an excellent choice for building apps that require both high performance and a modern look, like Podkast.

Below is a simplified example of a WinUI 3 navigation setup. Notice how the NavigationView and Frame are used to create a fluid navigation experience:

<NavigationView x:Name="nvSample" PaneDisplayMode="Left" IsBackButtonVisible="Collapsed">
    <NavigationView.MenuItems>
        <NavigationViewItem Content="Home" Tag="HomePage" Icon="Home"/>
        <NavigationViewItem Content="Podkast" Tag="PodkastPage" Icon="MusicInCollection"/>
        <NavigationViewItem Content="Settings" Tag="SettingsPage" Icon="Setting"/>
    </NavigationView.MenuItems>
    <Frame x:Name="contentFrame"/>
</NavigationView>

And the corresponding navigation logic in code-behind:

public sealed partial class MainWindow : Window
{
    public MainWindow()
    {
        this.InitializeComponent();
        ExtendsContentIntoTitleBar = true;
        nvSample.SelectedItem = nvSample.MenuItems[0];
        contentFrame.Navigate(typeof(Podkast.Pages.HomePage));
        nvSample.SelectionChanged += (sender, args) =>
        {
            if (args.IsSettingsSelected)
            {
                contentFrame.Navigate(typeof(Podkast.Pages.SettingsPage));
            }
            else
            {
                var selectedItem = args.SelectedItem as NavigationViewItem;
                if (selectedItem != null)
                {
                    switch (selectedItem.Tag.ToString())
                    {
                        case "HomePage":
                            contentFrame.Navigate(typeof(Podkast.Pages.HomePage));
                            break;
                        case "PodkastPage":
                            contentFrame.Navigate(typeof(Podkast.Pages.PodkastPage));
                            break;
                    }
                }
            }
        };
    }
}

Leveraging OpenAI for Agentic AI Features

OpenAI’s models can perform tasks such as transcription, summarization, and image generation with just a few lines of code. Using Betalgo’s OpenAI library for C#, you can easily initialize the service and set parameters that control how the models behave.

Here’s how you initialize the OpenAI service:

var openAiService = new OpenAIService(new OpenAiOptions { ApiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY") });

This single line sets up the service using an API key stored securely in an environment variable. With the service initialized, you can now make requests to different endpoints. For example, you can call methods to perform transcription with Whisper or generate responses with GPT models. Setting parameters like the model type, maximum token count, and response format allows you to fine-tune the behavior of each request.

The ability to choose between models such as GPT-4o or o1-mini for chat-based tasks or Whisper for audio transcription gives you great flexibility. This modular approach means you can use the best and the latest model for each specific task within Podkast.

Maintaining a Shared Chat Context

A critical component of Podkast is the shared conversation context. This context is a list that holds all messages exchanged between the user and the AI agents. It allows the app to “remember” previous interactions and use that information for future tasks — essential for building a coherent and intelligent conversation flow.

Here’s an example of maintaining a shared context:

List<ChatMessage> conversationContext = new List<ChatMessage>();
conversationContext.Add(ChatMessage.FromUser("Initial podcast content goes here"));

Every time an AI call is made, the response is appended to this list. Later requests will include the entire conversation history, ensuring that the AI is aware of previous prompts and responses. This design decision enhances the app’s ability to generate consistent and contextually relevant outputs, making the overall user experience more natural and connected. Using a List to maintain the context between the agents is kind of a dirty trick and works for small scale projects that are built for fun and aren’t really suitable for production-grade projects.

Putting Everything Together

With WinUI 3 providing the robust UI framework and OpenAI powering the intelligence, integrating all components into Podkast is straightforward. Here’s a high-level view of how the components work together:

1. UI Initialization with WinUI 3: Set up navigation using a NavigationView and Frame. Design the UI by drawing inspiration from the WinUI Gallery app for modern controls and layouts.

Arranging cards using Vertical and Horizontal Stacks

1. OpenAI Service Initialization: Initialize the OpenAI service using your API key. Set model parameters such as choosing between GPT-3.5-Turbo, WhisperV1, or DALL·E 3 based on the task.
2. Processing the Podcast: Use a file picker to let users select an audio file. Transcribe the file by sending its bytes to OpenAI’s transcription API. Update the shared conversation context with the transcript. Generate a summary, recommend similar podcasts, identify hosts, and create album art — all by making successive API calls that leverage the updated conversation context.
3. UI Updates: After each API call, update UI elements like text blocks and image controls to display the latest transcript, summary, and album art. This dynamic updating ensures that the user sees the conversation flow and the output of each agentic task in real time.

Below is a simplified method that ties these tasks together:

async Task ProcessPodcastAsync(StorageFile podcastFile)
{
    // Transcribe the audio file
    string transcript = await TranscribeAudioAsync(podcastFile);
    conversationContext.Add(ChatMessage.FromUser(transcript));

    // Summarize the transcript and update UI
    await SummarizePodcastAsync(transcript);

    // Identify hosts and generate album art
    await IdentifyHostsAsync();
    await GenerateAlbumArtAsync();
}

Building Podkast was a fun and rewarding project for me that helped explore the ease of WinUI 3 with the powerful capabilities of OpenAI models.

You can refer to my complete source code here: https://github.com/ItsAkilesh/Podkast-inator

Thanks for Reading! See You Around! Akilesh S