Go Scheduler

Why we need a scheduler?

Goroutines are user-space threads managed by the Go runtime. They are lighter-weight and cheaper than kernel threads. They have a smaller memory footprint (2 KB) and faster creation, destruction, and context switches. Goroutine switches take ~tens of nanoseconds, but thread switches take ~a microsecond.

A scheduler is create and assign goroutine to work on kernel threads.

First of all, we will start with the scheduler design with:

1. A Global Run Queue
2. A Thread

You may see that the first design has a bottleneck at the thread, because a thread can only perform one task at a time. If we need to do several tasks at the same time, we need to do context switching between tasks, and it cannot achieve parallelism.

Next, we will explore how to improve the scheduler to achieve better performance.

First Optimization: Max Threads

If we want to optimize the Go scheduler, we should increase the number of threads to equal the CPU’s cores so it can utilize the resource to perform parallel tasks. However, this could cause contention when threads try to access a global run queue, which will degrade performance even further.

Second Optimization: A Distributed Run Queue

Instead of all threads trying to access the same queue and dealing with races, we can use a distributed queue for each thread to avoid high contention.

Third Optimization: Work Stealing

What if one queue is overloaded and another is free? We can solve this by using the work stealing concept — letting a free queue steal work from a busy thread and execute it.

Fourth Optimization: Preemption (CPU-bound Work and Ghost Scheduler)

What if there is a task with high CPU consumption that blocks the thread (>20 microseconds)? We can improve this by pausing that task, putting it into a global run queue used to park long-running tasks, and letting the thread pick up a new task to execute.

Thank you the resource from GopherCon 2018: The Scheduler Saga — Kavya Joshi