Boosting Development Efficiency: Mastering WSL2 with Advanced Productivity Tips

The Windows Subsystem for Linux (WSL2) has significantly improved the workflow for developers working across Windows and Linux environments. It combines the robustness of Linux commands and applications with the convenience of Windows, streamlining development processes. However, to fully leverage the capabilities of WSL2, there are advanced configurations that can enhance productivity and usability further. This article serves as a consolidated resource, bringing together best-known methods for enhancing your WSL2 setup. This provides a comprehensive tutorial for the following items within the Linux subsystem, consolidated into a single document.

• Auto Mount Disk Partition
• Configuring Docker to store images on mapped partition
• Attaching USB Devices to WSL2
• Setting up VPN
• Port Forwarding (e.g ssh)

Incorporating these techniques during my transition from native Linux to WSL2 for robotics simulation work on ROS2 has resulted in a highly beneficial experience. This success serves to further validate WSL2’s practicality and effectiveness in real-world scenarios. These practices were verified on a system running Windows 11 and with Ubuntu 22.04 serving as the WSL2 guest environment.

Auto Mount Disk Partition

The WSL2 file system is stored within a single virtual disk file that can significantly increase in size with continuous use. To manage this effectively, it’s advisable to keep the Linux system’s core files within this vdisk file and allocate a raw disk partition for storing project files, docker images, and other large files used in development. Mapping a raw disk partition to WSL2 at startup offers a practical solution for managing additional content. An advantage of this approach is the ability to also map this partition to a native Linux system on the same hardware, ensuring immediate access to these files when booting into Linux directly.

This section outlines how to configure WSL2 with a dual-disk setup, where disk 1 is designated for Windows and includes the WSL2 virtual disk, while disk 2 is formatted for Linux and features a large secondary partition for both WSL2 and native Linux use. Partition sizes can be tailored as needed.

Dual Disk setup for WSL2

An illustrative setup is provided in the figure above, showing Partition 2 on disk 2, formatted with ext4 mapped into both WSL2 and native Linux.

For integration into WSL2, disk 2 must be taken offline within Windows, followed by utilizing a PowerShell WSL command to attach the disk to WSL2. It’s important to note that WSL2 requires attaching the entire disk, not just a partition. Once attached, partitions become accessible within the WSL2 environment and can be mounted using standard Linux commands.

The step described draw inspiration from an online discussion, to which the author has also contributed. Detailed instructions below will guide the automatic mapping of this partition upon starting WSL2. The steps assumes that user has two disks (eg NVME) attached to the system.

To get list of disks on system (In power shell)

C:\> wmic diskdrive list brief
Caption                DeviceID            Model                  Partitions  Size
KINGSTON SNV2S1000G    \\.\PHYSICALDRIVE0  KINGSTON SNV2S1000G    4           1000202273280
KINGSTON SNV2S1000G    \\.\PHYSICALDRIVE1  KINGSTON SNV2S1000G    3           1000202273280

In the Disk Management window, at the bottom, you’ll see a list of disks (Disk 0, Disk 1, and so on). These disk numbers correspond to the PHYSICALDRIVE numbers. For example, “Disk 0” in Disk Management corresponds to PHYSICALDRIVE0.

To get additional information about a disk (In power shell)

C:\> Get-Partition –DiskNumber 1

Let’s assume Disk 1 is the Linux disk which we want to mount to WSL2. Change disk number accordingly in target setup.

Automatic Mounting of partition in WSL2:

1. Take disk offline

Use Disk Management Console to bring disk offline. Also the command line version (from elevated power shell) for the same is given.

C:\> set-disk 1 -isOffline $true

2. Create disk mount Windows Task

• Using Windows Task Scheduler, create a task named mount-wsl-disks. This helps mount the drive with elevated permissions without triggering UAC prompts.
• Set the task to “Run with highest privileges”.
• Add mount command wsl — mount \.\PHYSICALDRIVE1 — bare to the task’s actions.

Command line version (from elevated powers hell)

schtasks /create /tn "mount-wsl-disks" /tr "C:\Windows\System32\wsl.exe   --mount \\.\PHYSICALDRIVE1 --bare" /sc once /sd 01/01/2100 /st 00:00 /it /ru <user> /rl highest /f

Replace <user> with windows username. Date is far in the future since this task will be run on demand.

3. Set WSL2 to Execute a Script on Startup:

This is available only in Windows 11. In WSL2, edit /etc/wsl.conf and add:

[boot]
command="bash /boot.sh"

4. Create the initial boot script /boot.sh:

This script will call another script (/wsl-boot.sh) and redirect its output to /wsl-boot.log for any debugging needs.

/boot.sh

#!/bin/bash
/bin/bash /wsl-boot.sh > /wsl-boot.log 2>&1

5. Create the main boot script /wsl-boot.sh:

The following script runs when WSL2 starts and triggers the mount-wsl-disks task. It then waits until the drive is ready for mounting. Modify target drive and mount point values accordingly.

/wsl-boot.sh

#!/bin/bash
# Start the mount task
/mnt/c/Windows/system32/schtasks.exe /run /tn "mount-wsl-disks"
# Wait for the drive (e.g /dev/sdd1) to be ready for mounting
target_drive="/dev/sdd1"
mount_point="/mnt/mapped"
timeout=10  # check for 10 seconds
interval=1  # every second
elapsed=0
while [ ! -e $target_drive ]; do
    if [ $elapsed -ge $timeout ]; then
        echo "Timed out waiting for $target_drive to be ready."
        exit 1
    fi
    echo "Waiting $target_drive to be ready..."
    sleep $interval
    elapsed=$((elapsed + interval))
done
# Mount the drive
mount $target_drive $mount_point
sleep 1
## Start docker service if docker persitent storage was on newly mapped partition.
# sudo service docker start

By following these five steps, the secondary partition on disk 2 will automatically mount when WSL2 starts, ensuring a seamless user experience.

Configuring Docker to store images on mapped partition

For those looking to optimize their Docker setup, directing Docker to store images and containers on a secondary partition can offer significant benefits in terms of storage management and performance. This can be particularly advantageous when running Docker within environments with limited primary storage space or when attempting to segregate system files from Docker images. This section outlines the steps to configure Docker to utilize a WSL2 mapped partition, which has been mounted at /mnt/mapped, for storing Docker images and containers.

Docker stores its data (including images, containers, and volumes) in a directory known as the Docker root directory. By default, this directory is located at /var/lib/docker on Linux systems. To change Docker’s storage location to the mapped partition:

1. Install dockers by following instructions. [Instructions]

2. Stop the Docker daemon if it’s currently running. This can typically be done with the command using systemd.

sudo systemctl stop docker

3. Open the Docker daemon configuration file, usually found at /etc/docker/daemon.json. If the file does not exist, create it.

4. Modify the data-root configuration option to point to the new location on the secondary partition. If the file was newly created or this option doesn’t exist, add the following line:

{
"data-root": "/mnt/mapped/docker"
}

Ensure the path /mnt/mapped/docker exists or modify the path according to your mounted location and desired directory structure. Save the changes and close the file.

5. Since we are configurating Docker to utilize a mapped partition, which might not be immediately available when the Docker service initiates at the start of WSL2, the Docker service is set to initiate manually.

sudo systemctl disable docker

Consequently, the command to start Docker is incorporated into the wsl-boot.sh script. See the comments towards the end of wsl-boot.sh file mentioned in previous segment.

By applying the specified adjustments to the wsl-boot.sh file as described, Docker images will be saved on the mapped partition. To access these downloaded images in native Linux, similar updates will have to be made to the daemon.json file within the native Linux environment.

Attaching USB Devices to WSL

Accessing USB devices directly in Windows Subsystem for Linux 2 (WSL2) is not inherently supported due to its architectural design. WSL2 operates a Linux kernel within a lightweight utility virtual machine (VM), which inherently isolates it from direct hardware access, including USB devices. Nevertheless, workarounds and third-party tools are available to facilitate this access:

Via USB/IP

USB over IP (USB/IP) is a technique for sharing USB devices over a network, enabling remote systems to access them. This approach can be utilized to make USB devices accessible in WSL2 by sharing them from the Windows host.

• Step 1: Installation of USB/IP software on Windows is required.
• Step 2: The desired USB device is shared from Windows.
• Step 3: The device is accessed from WSL2 by connecting to the shared device over the network.
• Step 4: Configure Linux kernel with relevant USB device driver.

Instructions to install usbip tool can be found at Link.

Steps to Attach USB devices to WSL.

• List available Devices using Windows PowerShell. Output will include busids.

C:\> usbipd list

• Bind device to WSL (Requires Elevated PowerShell)

C:\> usbipd bind --busid <busid>

• Attach Device to WSL

C:\> usbipd attach --wsl --busid <busid>

• In WSL, ensure the device is available

$ lsusb

Here is an excellent video tutorial to enable USB webcam:

Enabling USB Camera

Manual Mounting (Limited to Storage Devices):

USB storage devices can be manually mounted in WSL2 after being attached to Windows. This method is specific to storage devices and does not extend to other USB device types. The same method is used in dual disk setup tutorial above.

• Step 1: The USB storage device is attached to Windows.
• Step 2: The device is identified in Windows (via Disk Management or command-line tools).
• Step 3: The device is manually mounted in WSL2 using the mount command, requiring the correct file system type specification and ensuring WSL2 compatibility with necessary drivers.

Important Considerations:

• Limitations may exist based on the USB device type and the required drivers or software.
• Performance and compatibility could vary, with some devices potentially not functioning as anticipated.
• Security of USB devices should be a priority, especially when employing network-based sharing methods.

Given the complexities and potential security implications of enabling USB device access in WSL2, careful consideration is advised to determine the necessity and the most appropriate method, depending on specific requirements and the capabilities of the devices intended for use.

Setting Up VPN

Windows Subsystem for Linux version 2 (WSL2) operates within a lightweight virtual machine (VM), giving it a unique networking setup compared to its predecessor, WSL1. Due to this architecture, there are a few considerations regarding VPN access:

1. Separate Network Namespace: WSL2 instances run in their own network namespace. This means that even if a VPN is activated on the Windows host, WSL2 might not automatically utilize the VPN connection in the same manner as native Windows applications.

2. Consistency with Host: Some VPN clients, when activated on the Windows host, will seamlessly route WSL2 traffic through the VPN. However, this is not universal, and behavior can vary based on the VPN client or its configuration.

3. WSL2-Specific VPN Configuration: It’s possible to set up VPN connections directly within WSL2 using Linux VPN clients. This allows WSL2 to maintain its own separate VPN connection, independent of the host’s connection.

4. Potential Workarounds: If WSL2 isn’t recognizing or routing through the host’s VPN connection, users often employ workarounds, such as custom scripts or third-party tools, to forward necessary traffic or synchronize network configurations between the host and WSL2.

While WSL2 offers powerful integration between Windows and Linux, its unique network architecture can introduce complexities when it comes to VPN usage. Users may need to experiment or employ workarounds depending on their specific VPN setup and needs.

VPN Integration in WSL2:

By leveraging OpenConnect, it’s possible to establish a stable and secure connection to an organization’s internal network.

Setting up the necessary proxies within WSL2 allows for straightforward interactions with repositories, keeping the local development environment in sync with company resources. The standout attribute of this configuration is its reliability; VPN connectivity within WSL2 maintains consistent access without interruptions.

Install openconnect using apt-get.

sudo apt-get install openconnect

Obtain relevant files from VPN provider (e.g certificate, private key).

sudo openconnect --background -c “User.crt” -k private.key --cafile “CA 5A.crt” --authgroup=any --pid-file=/var/run/openconnect.pid --user <userid> vpn.xxx.com

After the VPN is connected, set appropriate environment variables related to proxy (e.g http_proxy) in WSL2 terminal before initiating any further commands (e.g git pull).

Port Forwarding

The Windows Subsystem for Linux 2 (WSL2) employs a virtual machine architecture, diverging from WSL1’s approach, thereby altering the behavior of networking. Services operating within WSL2 do not automatically become accessible to the Windows host or other devices on the local network. Below are the steps to establishing port forwarding for WSL2:

1- Get WSL2 IP Address

Make sure that WSL2 is active, then in an elevated command shell on Windows, perform the command to capture the IP address of the WSL2 instance currently running.

PS C:\> wsl -e ip -4 -o addr show eth0
2: eth0    inet 172.17.254.91/20 brd 172.17.255.255 scope global eth0\       valid_lft forever preferred_lft forever

Look for the inet address under the eth0 section. It’ll be something like 172.x.x.x. The IP address of WSL2 typically remains stable across reboots and rarely undergoes changes.

2- Forward Ports from Windows to WSL2

The netsh command in Windows, executed in elevated power shell, is utilized to establish port forwarding from the Windows host to the WSL2 instance. Below is an example for setting forwarding for port 22:

C:\> netsh interface portproxy add v4tov4 listenport=22 listenaddress=0.0.0.0 connectport=22 connectaddress=172.x.x.x

Replace 172.x.x.x with actual WSL2 IP address and adjust the port numbers as needed. The listen and connect ports can differ, offering the flexibility to expose a service on a different external port than the one it runs on within WSL2.

Verify that port forwarding is setup correctly.

PS C:\> netsh interface portproxy show all
Listen on ipv4:             Connect to ipv4:
Address         Port        Address         Port
--------------- ----------  --------------- ----------
0.0.0.0         22          172.29.220.198  22

(Optional) Automate the Process:

While the IP address in WSL is generally consistent across reboots, there are instances where the WSL2 IP address might change, necessitating an update in port forwarding. For convenience, here’s a script to automate the process.

Create a .bat file (e.g., wsl2-forward.bat) with the following content:

@echo off
setlocal enabledelayedexpansion

:: Get WSL IP address. Assuming single WSL running
for /f "tokens=4 delims= " %%i in ('wsl -e ip -4 -o addr show eth0 ^| findstr "inet"') do (
 set IP_WITH_CIDR=%%i
 for /f "tokens=1 delims=/" %%j in ("!IP_WITH_CIDR!") do set WSL2_IP=%%j
)

echo WSL2 IP Address: %WSL2_IP%
set PORT=22

:: Delete previous mapping
netsh interface portproxy delete v4tov4 listenport=%PORT% listenaddress=0.0.0.0

:: Add new mapping
netsh interface portproxy add v4tov4 listenport=%PORT% listenaddress=0.0.0.0 connectport=%PORT% connectaddress=%WSL2_IP%

echo Port forwarding set %WSL2_IP%:%PORT%
@echo on

This script fetches the WSL2 IP address and sets up the port forwarding. Run this script in elevated PowerShell each time when the WSL2 IP changes.

PS C:\> wsl2-forward.bat

Update Windows Firewall

For external access, it is critical to confirm that the Windows Firewall permits incoming connections on the ports being forwarded. This may necessitate creating a new inbound rule to enable connections on the designated port. To allow port 22 for SSH access from another machine, execute the following command from an elevated PowerShell on WSL2’s host Windows.

PS C:\> New-NetFirewallRule -DisplayName "Allow SSH" -Direction Inbound -LocalPort 22 -Protocol TCP -Action Allow

After this step, the ssh server running in WSL2 is accessible from other machines that have access to the host window PC.

TIP: In some cases, it’s crucial to ensure that the service running inside does not bind to a specific IP address. For instance, with JupyterLab, employing ‘-p 0.0.0.0’ as a parameter enables listening on all interfaces.

jupyter-lab --ip=0.0.0.0

Summary

With these advanced configurations, developers can make the most out of the Windows Subsystem for Linux, bridging the gap between Windows and Linux even further. Auto-mounting partitions enhance file management, setting up a VPN increases security and accessibility, and utilizing USB devices expands the hardware compatibility of WSL. These steps not only improve the overall functionality of WSL2 but also cater to specific use cases, enabling a more efficient and productive development environment.

While these configurations can significantly enhance your WSL setup, always proceed with caution and back up important data before making system changes.