Creating Custom MSI Actions in .NET Using WiX Toolset

WiX (Windows Installer XML) is a powerful toolset for creating MSI installers. While its declarative nature simplifies everyday tasks, specific installation requirements may demand dynamic behavior — this is where custom actions shine. Custom actions allow you to run scripts or code during the installation process. For .NET developers, this means leveraging the power of C# to handle tasks beyond what WiX natively supports.

With practical examples and best practices, this article explores creating, integrating, and debugging custom MSI actions in .NET using WiX Toolset.

What Are Custom Actions?

Custom actions are user-defined operations executed during an MSI package’s installation, uninstallation, or maintenance. They extend the Windows Installer’s capabilities by enabling tasks that go beyond its usual capabilities.

Custom actions can perform a variety of tasks, such as:

• Modifying configuration files based on user input.
• Setting up system services.
• Verifying or modifying system prerequisites (e.g. checking for specific registry entries).
• Interacting with external systems, such as databases or web services.

They are typically implemented as:

1. Scripts (e.g., PowerShell or VBScript).
2. DLLs (e.g., .NET assemblies or native C++ libraries).
3. Executables (.exe files).

In the context of WiX, custom actions are often written in C# using Microsoft.Deployment.WindowsInstaller library to leverage the power and flexibility of the .NET platform.

Step 1: Setting Up Your WiX Project

Ensure you have the WiX Toolset installed. If not, download it from the official WiX website.

1. Create a new WiX project in Visual Studio: File > New Project > WiX Setup Project.
2. Define the installer’s structure in your WiX .**wxs** file. For example:

<Product Id="*" Name="MyApp" Version="1.0.0" Manufacturer="MyCompany" Language="1033">
  <Package InstallerVersion="500" Compressed="yes" InstallScope="perMachine"/>

  <Directory Id="TARGETDIR" Name="SourceDir">
    <Directory Id="ProgramFilesFolder">
      <Directory Id="INSTALLFOLDER" Name="MyApp" />
    </Directory>
  </Directory>

  <Feature Id="MainFeature" Title="Main Feature" Level="1">
    <ComponentGroupRef Id="ProductComponents" />
  </Feature>
</Product>

Step 2: Writing the Custom Action in C

WiX custom actions are typically implemented in C# as a separate library project.

1. Create a new Class Library project in Visual Studio: File > New Project > Class Library (.NET Framework or .NET Core/6/7).
2. Add references to WiX libraries: Add a NuGet package reference to **Microsoft.Deployment.WindowsInstaller** (included in WiX Toolset).
3. Write the custom action logic. Below is an example custom action that writes a value to the Windows registry:

using Microsoft.Deployment.WindowsInstaller;
using System;
using Microsoft.Win32;

namespace CustomActions
{
    public class CustomActions
    {
        [CustomAction]
        public static ActionResult WriteToRegistry(Session session)
        {
            try
            {
                session.Log("Begin WriteToRegistry Custom Action");

                string keyPath = "SOFTWARE\\MyApp";
                string valueName = "InstallPath";
                string valueData = session["INSTALLFOLDER"];

                using (RegistryKey key = Registry.LocalMachine.CreateSubKey(keyPath))
                {
                    if (key == null)
                    {
                        throw new Exception("Failed to open registry key.");
                    }
                    key.SetValue(valueName, valueData);
                }

                session.Log("Registry key written successfully.");
                return ActionResult.Success;
            }
            catch (Exception ex)
            {
                session.Log("Error in WriteToRegistry: " + ex.Message);
                return ActionResult.Failure;
            }
        }
    }
}

Step 3: Linking the Custom Action in WiX

After building the custom action project, link it to your WiX installer.

1. Add the DLL to your WiX installer project.** Include the compiled `.dll`** in the installer package:

<Binary Id="CustomActions" SourceFile="$(var.CustomActions.TargetDir)CustomActions.dll" />

2. Define the Custom Action:

<CustomAction Id="WriteToRegistry" BinaryKey="CustomActions" DllEntry="WriteToRegistry" Execute="deferred" Return="check" />

3. Schedule the CA. Add the custom action to the install sequence. For example, to execute it after files are installed:

<InstallExecuteSequence>
  <Custom Action="WriteToRegistry" After="InstallFiles">NOT REMOVE</Custom>
</InstallExecuteSequence>

Step 4: Testing and Debugging

Enable MSI Logging

Enable logging during installation to debug custom actions:

msiexec /i MyApp.msi /L*V install.log

Look for your custom action log messages to diagnose issues.

Common Debugging Techniques

1. Session Logging: Use session.Log(“message”) in your custom action to trace execution.
2. Handle Errors Gracefully: Wrap code in try-catch blocks to prevent crashes and provide meaningful error messages.
3. Dependency-Check: Use tools like Dependency Walker to ensure all DLLs referenced by the custom action are included in the MSI.

Best Practices

1. Use Deferred Execution: For system-level changes like registry updates or file modifications, use Execute="deferred" in your custom action.
2. Minimize Dependencies: Ensure the custom action logic is self-contained and includes all required libraries.
3. Test Extensively: Test custom actions in clean environments to simulate real-world installation scenarios.
4. Fail Gracefully: Provide detailed logs and ensure the installer rolls back cleanly on failure.

Conclusion

Custom actions in WiX provide the flexibility to handle advanced installation scenarios for .NET applications. Combining WiX’s declarative model with the dynamic capabilities of .NET allows you to create robust and reliable installers tailored to your application’s needs. Follow the steps and best practices outlined here to get started, and remember: clear logging and thorough testing are your best friends in this process.