Chapter-1 : First Baby Step, Connecting UART [RexOS Dev]

Github Link: https://github.com/Arpit-Mohapatra007/RexOS-riscv

To begin developing our OS from scratch, I took a step-by-step approach. I crafted each component bit by bit, assembled them, hit “Enter,” and began praying that my code would compile and boot.

1. Defining Memory Layout:

Before deciding on our own memory layout, let’s discuss the standard industry approaches:

(a) Embedded / Emulator : Hard-coded macros with addresses provided by silicon manufacturers.

(b) OS in PCs, Laptops, etc. :

• In ARM, RISC-V (like Raspberry Pi) hardware motherboard contains a read-only file “Device Tree (.dtb)” that is a database of addresses of each and every component.
• Intel and AMD use much more complex “ACPI (Advanced Configuration and Power Interface)”

For RexOS, I adopted a hybrid model. I hard-coded macros for standard hardware components because their addresses in the QEMU virt board are well-documented and static:

CLINT -> 0x02000000 PLIC -> 0x0c000000 UART0 -> 0x10000000 RAM -> 0x80000000

But what about the end of the RAM stick? We can’t hard-code it because we don’t know if the OS is running on an IoT device with minimal memory or a server with massive RAM. To solve this, I coded a DTB parser to hunt for RAM metadata. It fills a ram_metadata structure with the base address and size, allowing us to dynamically calculate ram_end = ram_base_address + ram_total_size.

This information will be crucial later in our journey.

2. Initialize the execution environment : When our OS boots, the entire system is full of garbage values. The RAM stick, registers, wires, and other hardware are completely uninitialized.

We need to set up a boot time stack to use to run our C functions like kmain , kalloc_init , etc. to set up page tables and all other essential setup before jumping into userspace.

For this we zeroed out 8 KB of memory and allocated it as boot time stack to our kernel. Actually we cleared out the whole .bss section which includes our stack in it.

Next, modern CPUs have multiple cores, and on boot all these cores wake up simultaneously, this will result in our start.S which sets up stack for our C functions being executed in all cores at the same time resulting in a classic race condition.

To tackle, I had two options: (a) Choose single core by just changing a flag in my Makefile to spin up a single core emulator. (b) Implementing a method known as “Hart Parking”. In Hart Parking, we park harts ( cores of CPU) that we don’t require right now, but we’ll seriously need them later on to implement multitask scheduling.

I chose the second tactic, because I want it to be a full-fledged OS not a simple educational OS. And now only Core 0 is allowed to call kmain and rest of the cores are parked using a simple RISC-V instruction wfi (wait for interrupt). These Cores are not executing any instruction like an infinite loop as that would waste CPU cycles and hardware life degradation nor is it hindering our boot sequence, it’s just sleeping, waiting for an interrupt.

3. Grouping Sections : Next, when in future we will open RAM for userspace, we need to protect our kernel code, also to smoothly work we need to align all our sections .text , .bss , .data , and .rodata of our kernel. To do this we took help from GNU linker Files .

4. UART Setup :

Why we need UART ?

UART ( Universal Asynchronous Receiver-Transmitter ) captures our keystrokes ( Receive ) and displays them on monitor ( Transmit ). Therefore, it is very crucial for our OS as it is the eye and ear of our RexOS.

And this is the bible that helped me to configure my OS to use UART: https://docs.freebsd.org/en/articles/serial-uart/#_82501645016550_registers

I chose NS16550A UART because it is widely used world-wide and it is also available in our QEMU “virt” board.

A brief summary of the article is here :

Instead of clock, UART uses : (i) Baud Rate Agreement : Baud stands for rate of data transmission , we configure our UART chip such that it could receive its data from extremely slow keyboard and transmit data to extremely fast CPU. (ii) Data Envelope : When there is no data in wire UART, voltage is high (1). When to send data (start) UART chip drops this voltage to 0 for one-bit width time, then data is transmitted, to signal stop, the chip drives the line high again.

NS16550A UART Layout

In UART, Offset 3’s 7th bit acts as DLAB switch.

How to set communication speed ? — It is very crucial as without it everything will become chaotic as no one will be able to communicate with each other. We use this formula to configure our UART :

Divisor = Clock Frequency / ( Desired Baud Rate * 16 )

For RexOS, the standard chosen configuration is a baud rate of 115,200 bits per second (bps). Inside QEMU’s standard RISC-V virt board hardware emulation, the system clock driving the emulated NS16550A UART chip is set to exactly 3,686,400 Hz. The number **16 represents the Oversampling Factor** hardwired into the silicon of the NS16550A chip.

With these values in place the formula yields us Divisor = 3 . This helps us to set a Baud Rate Agreement between our Input Device and UART chip, but what about UART chip and CPU, we can’t sign any Baud Rate Agreement with it as CPU is extremely fast compared to UART’s maximum possible speed. Hence, we adopt a strategy called Polling , Polling is when a CPU repeatedly checks hardware status instead of waiting for an interrupt.

Initialization of UART:

1. Disable Interrupt as we don’t want UART chip to interfere in our boot sequence. Write 0x00 to UART0_BASE + 1
2. To set communication speed, we need to set DLAB as we can see in the table. Write 0x80 [7th bit 1 rest are 0] to UART0_BASE + 3
3. For setting communication speed, we need to write our calculated divisor’s lower bytes into Offset 0 and higher bytes into Offset 1. Write 0x03 to UART0_BASE + 0 AND Write 0x00 to UART0_BASE + 1
4. Now we need to lock this speed and make Offset 0 and 1 perform their normal functioning. So we need to clear DLAB. But, we also need to set rules for our communication, I chose to:

• Disable Parity bit [ 3rd bit of Offset 3 ]
• Only one stop bit [ 2nd bit of Offset 3 == 0 ]
• 8 bit data to be transmitted and received [ Bit 1 and Bit 0 of Offset 3 == 11 ] Write 0x03 [00000011] to UART0_BASE + 3

5. Before allowing UART to receive and transmit data we need to reset [ Bit 2 -> Transmit FIFO Reset and Bit 1 -> Receiver FIFO Reset of Offset 2 ] both its wires and enable [ Bit 0 -> Transmit-Receiver FIFOs Enable Switch of Offset 2 ] them. Write 0x07 [111] to UART0_BASE + 2

Use of Polling to transmit data to Output Screen :

1. I deployed a loop that will check if Transmitter FIFO is empty or not. [ Bit 5 of Offset 5 ]. Check if Bit 5 of UART0_BASE + 5 is 0
2. If 5th bit is 0 that means there is data present in Transmitter FIFO wire, we need to wait until it becomes 1 which means wire is idle. Then, we need to send that data to Offset 0 whose purpose is to transmit and receive data. Write DATA to UART0_BASE + 0

We use this same function to print one character to display, to write our wrapper functions to display a string, a function to display a string with fixed size to prevent stack bleeding , a function to display hex representation of incoming binary code that we will use in our trap handlers.

Use of Polling to receive data and display it on Output Screen :

1. We again deployed a loop to check if data is waiting in Receiver FIFO or not. [ Bit 0 of Offset 5]. Check if Bit 0 of UART0_BASE + 5 is 0
2. If Bit 0 is 0 that means there is no data waiting in the Receive FIFO wire, we need to wait until some data comes in and Bit 0 becomes 1, we will send this received data to Offset 0 to display it on screen. Write DATA to UART0_BASE + 0

Once we configured our UART, I added some aesthetics to my OS by adding a RETRO -Style Banner and a Prompt ( RexOS> )to start every new line in our console. And also used ANSI Escape Sequences to make it look beautiful on boot.

5. Setup of Trap Handlers : This chain of functions will help us in future to debug our messy code ! To setup trap handlers , RISC-V provides us with a very handy CSR called mtvec , I loaded the address of my trap handler in this mtvec .

Inside our trap handler label, I stored current values of all 31 GPR excluding x0 because it is hard-coded zero, into the stack and passed mcause [ CSR that contains binary representing cause of trap ] and mepc [ CSR that contains address where trap occurred ] to our custom C function kpanic , where using switch cases and UART we displayed a custom diagnostic message to the console about the cause and place of trap occurrence.

I referred to this Medium article for writing my kpanic function logic : https://medium.com/@wadixtech/techniques-to-use-to-analyze-software-faults-exceptions-on-riscv-processors-1d7a14fe494c

6. Command Buffer and Command Parser : Next, to visually see our code in action I declared a command buffer that stores our input characters and compares them with a predefined set of commands using our own crafted strcmp() function because till now in bare metal, we don’t have standard libraries of C. Similarly, for future use I crafted strlen() function.

Right now our OS supports 2 commands :

1. help : It displays a beautiful menu of commands and their functionality.
2. clear : That runs a series of ANSI Escape Sequences to clear screen and reposition our cursor to the left corner of console.

7. Finally Waking our OS by Compiling all codes : To do this I created a Makefile using knowledge from my college. Only for Compilation Flags (Flags passed to GCC) and Build Flags ( Flags passed to QEMU ) I had to do online research :

Compilation Flags :

1. -nostdlib : Because we don’t have any standard library until now in our bare metal environment, hence we have to stop our stupid compiler from doing stdlib optimizations.
2. -march=rv64gc : march allows us to decide our architecture, rv64 stands for RISC-V 64 bit, g includes instruction set of I-type, M-type, A-type, F-type and D-type of RISC-V , c includes instruction from Compressed Instruction Set of RISC-V.
3. -mabi=lp64d : mabi allows us to decide our execution binary interface, l states compiler that all integers are of long data type by default ( 8 bytes or 64 bits), p64 states compiler that all pointers are also 64 bits wide, d allows binary to use dedicated floating point registers for floating point math.
4. -fno-builtin : It prohibits compiler from using std functions like memcpy ,memset etc. because they also don’t exist in our environment.
5. -ffreestanding : Tells compiler that this code is running without any OS environment and hence non existence of main function must not be flagged as an error.
6. -mcmodel=medany : mcmodel stands for Memory Code Model that are used by compiler to predict where code lives in RAM in order to compile code faster. Default is medlow which assumes everything lives in lowest 2GB of memory. And medany tells compiler that code maybe sitting anywhere in RAM but next line of code is definitely within 4GB window of current line of code. Because RexOS is linked at 0x80000000, medlow triggers instant linker relocation crashes. medany allows our kernel to sit anywhere by using program-counter-relative addressing.
7. -Wall and -Wextra : Tells compiler to show warnings and extra stuff useful for debugging.
8. -g : Tells compiler to add debug symbols to binary which will protect us from a debugging nightmare.

Flags for QEMU :

1. -machine virt : Tells QEMU to simulate a virt board for our kernel.
2. -cpu rv64 : Tells QEMU to simulate a RISC-V 64 bit CPU.
3. -smp 8 : Tells QEMU to simulate an 8 core ( hart ) machine. SMP stands for Symmetric MultiProcessing.
4. -m 128M : Tells QEMU to simulate a 128MB RAM stick in our machine.
5. -bios none : Instructs QEMU to not use its pre-built bios to run our code as our code is written in a way that it can boot on absolute bare metal.
6. -kernel : Tells QEMU to run our code as kernel in the simulated machine.

Reference: https://www.qemu.org/docs/master/system/riscv/virt.html

Final Image of RexOS after Waking Up :

Problem faced afterwards :

1. After Boot I realized that on pressing “backspace” key, my cursor only moves back by one position but does not delete the character at that position due to some historical reason, in UART backspace doesn’t work as it works in modern systems. Therefore I implemented a sequence of instructions to make it visually look like a normal backspace. On pressing backspace :

• I transmit “\b” (escaped backspace character) to UART, which results in cursor moving back by one position.
• Next I transmitted a white space to UART, that makes it visually look like the character got deleted but due to this cursor moved ahead by one position.
• Next I again transmitted another backspace character to UART, this resulted in a clean-looking modern-day backspace.

2. Due to similar historical implications, UART doesn’t transmit “newline character” (“\n”) on an “enter” key press. Therefore, I implemented a sequence of instructions to make it visually look like a normal enter press. On pressing enter key:

• I transmitted “\r” ( represents a Carriage Return). It moves the cursor back to the beginning of the current line without advancing to the next line.
• Next I transmitted “\n” to move cursor to next line. We can’t simply transmit a single “\n” character alone because in UART “\n” simply makes the cursor jump next line on the same position unlike modern day where it also sends cursor to start of the next line.

This wraps up our Phase -1 of RexOS Dev next, we will move to implement British policy of “Divide and Rule” on RAM sticks.