Runtime Interaction with Shared Objects in Go

Building a Go Wrapper for C Shared Objects

Citation: LLM Generated by Author

Overview

Interfacing Go with C libraries is common in systems programming, but things become more interesting when those libraries are loaded dynamically at runtime rather than linked at build time.

In this post, we’ll walk through how to:

• Design a modular C library using context objects and function pointers
• Wrap that library safely in Go using cgo
• Dynamically load C shared objects at runtime using dlopen and dlsym
• Work around cgo’s function pointer limitations

The result is a small but realistic example of a plugin-style architecture in Go.

Background

In systems programming, drivers, protocol handlers, and other modular components often require a clean separation between what a system does and how it implements that functionality.

For example, a networking stack should always be able to send and receive packets, but whether those packets come from Ethernet, Wi-Fi, or a virtual interface is an implementation detail that is hidden behind a module abstraction.

Common Shared Object Architecture Patterns:

In C, this separation is commonly achieved with:

• A context structure that holds shared state
• A vtable (structure of function pointers) that defines behavior

A classic example is the Linux kernel, where file system drivers populate a file_operations structure with function pointers that define how the kernel interacts with a specific filesystem.

Tutorial

To demonstrate runtime loading, we’ll use a small “pets” library with two modules: dog and cat.

The Pet Library

First, let’s briefly walk through the pets library that was created for this tutorial. This library consists of three shared objects:

• pets.so defines common functions across all modules
• dog.so and cat.so are modules to load at runtime

Pet Context:

Below is the header file for the pets library. This file defines the Pet structure which will serve as a context object for the library. In this case, the vtable is intentionally minimal and consists of a single speak function pointer.

// c/include/pets.h
#ifndef PETS_H
#define PETS_H

typedef struct Pet {
    char name[100];
    void (*speak)(struct Pet*);
} Pet;

Pet* new_pet(const char*);
void free_pet(Pet* pet);

#endif // PETS_H

For the library itself, a few basic functions to create/free a pet context were defined. Since the pets library is used across all modules, the library can be dynamically linked during program load time.

// c/src/pets.c
#include <stdlib.h>
#include <stdio.h>
#include "pets.h"

Pet* new_pet(const char* name) {
    if (!name) {
        return NULL;
    }

    Pet* pet = malloc(sizeof(Pet));
    if (!pet) {
        return NULL;
    }

    snprintf(pet->name, sizeof(pet->name), "%s", name);
    pet->speak = NULL;
    return pet;
}

void free_pet(Pet* pet) {
    if (pet) {
        free(pet);
    }
}

Pet Modules:

I created two modules for this library, dog and cat. Each of these modules has a pet_init function to fill the pet context structure with function pointers for the module (just the speak function), and the modules were compiled as individual shared objects to be loaded at runtime.

// c/src/dog.c
#include "pets.h"
#include <stdio.h>

void dog_speak(Pet* pet) {
    printf("%s says: Woof! Woof!\n", pet->name);
}

void pet_init(Pet* pet) {
    if (pet) {
        pet->speak = dog_speak;
    }
}

// c/src/cat.c
#include "pets.h"
#include <stdio.h>

void cat_speak(Pet* pet) {
    printf("%s says: Meow! Meow!\n", pet->name);
}

void pet_init(Pet* pet) {
    if (pet) {
        pet->speak = cat_speak;
    }
}

The Go Wrapper

The goal of the Go wrapper is to:

• Isolate all unsafe C interactions
• Expose a clean, idiomatic Go API

Wrapping the Pets Library

First, lets use cgo to access the functions in pets.so. According to the cgo documentation, comments immediately preceding animport “C” statement are treated as C preamble code and compiled as part of the package’s C translation unit. This includes the compile flags that would be specified in a Makefileor CMakeLists.txt in a traditional C project. I used the following flags to import the pets library:

/* 
// the path to the directory that stores "pets.h" 
#cgo CFLAGS: -I../../c/include 

// -L for the path to the directory that contains "libpets.so" 
// -l to tell the loader to look for "libpets.so" in the directory 
#cgo LDFLAGS: -L../../lib -lpets 

// include the pets library like in C/C++ 
#include <pets.h> 
*/ 
import "C"

Now, lets create a thin wrapper around the C Pet structure and the associated functions.

package pets 
/* 
#cgo CFLAGS: -I../../c/include 
#cgo LDFLAGS: -L../../lib -lpets 
#include <stdlib.h> 
#include <pets.h> 
*/ 
import "C" 

import ( 
 "fmt", 
 "unsafe" 
) 

// create a golang structure that privately stores 
// the C Pet structure type Pet struct { pet *C.Pet } 
// safely wrap the creation of a C Pet structure 
func New(name string) (*Pet, error) { 
 // create the c pet structure 
 petName := C.CString(name)
 defer C.free(unsafe.Pointer(petName))
 pet := C.new_pet(petName)
 if pet == nil {
  return nil, fmt.Errorf("failed to create pet")
 }
 // return the golang pet with the C pet privatly stored 
 return &Pet{ pet: pet, }, nil 
} 

// wrap the free call 
func (p *Pet) Delete() {
 if p.pet != nil {
 C.free_pet(p.pet) 
 }
} 
// wrap the speak function 
func (p *Pet) Speak() { 
 // needs to call p->speak(p.Pet) 
} 

// getter wrapper to access the c pet->name 
func (p *Pet) Name() string { 
 return C.GoString(&p.pet.name[0]) 
}

Calling pet->speakrequires invoking a C function pointer directly. cgo does not allow calling C function pointers from Go, so an additional C helper wrapper is required. Fortunately, we can do this directly in the cgo comment like below.

/* 
// A generic helper function that invokes a function pointer 
// taking a Pet* as its argument 
void call_pet_func(void* f, Pet* pet) { 
 ((void (*)(Pet*))f)(pet); 
} 
*/ 
import "C" 
import ( 
 "unsafe" 
) 

type Pet struct { 
 pet *C.Pet 
} 

func (p *Pet) Speak() {
 C.call_pet_func(unsafe.Pointer(p.pet.speak), p.pet) 
}

Wrapping the Pets Modules

Now that we have setup the pet library, it is time to load the dog and cat modules. This follows a pretty similar pattern to the C usage; the cgo flags, New(), and Delete() functions are modified as follows to load a module at runtime.

package pets

/*
#cgo CFLAGS: -I../../c/include
#cgo LDFLAGS: -ldl -L../../lib -lpets

#include <dlfcn.h>
#include <stdlib.h>
#include <pets.h>

void call_pet_func(void* f, Pet* pet) {
 ((void (*)(Pet*))f)(pet);
}

*/
import "C"
import (
 "fmt"
 "unsafe"
)

type Pet struct {
 // create an unsafe pointer to save 
 // the handle to the pet module library
 petLib unsafe.Pointer
 pet    *C.Pet
}

func New(name string, petType string) (*Pet, error) {
 // create the c pet context structure
 petName := C.CString(name)
 defer C.free(unsafe.Pointer(petName))
 pet := C.new_pet(petName)
 if pet == nil {
  return nil, fmt.Errorf("failed to create pet")
 }

 // load the the appropriate pet library based on a parameter 
 // hard coded paths are bad practice, but this is just a tutorial 
 var petLibPath *C.char
 switch petType {
 case "dog":
  petLibPath = C.CString("../lib/libdog.so")
 case "cat":
  petLibPath = C.CString("../lib/libcat.so")
 default:
  return nil, fmt.Errorf("unknown pet type: %s", petType)
 }
 defer C.free(unsafe.Pointer(petLibPath))

 // load the shared library
 petLib := C.dlopen(petLibPath, C.RTLD_LAZY)
 if petLib == nil {
  return nil, fmt.Errorf("failed to load %s library", petType)
 }

 // since we are loading this module at runtime, 
 // we must get a pointer to the pet_init symbol defined 
 // in the module's c code
 petInitSymName := C.CString("pet_init")
 defer C.free(unsafe.Pointer(petInitSymName))
 petInitSym := C.dlsym(petLib, petInitSymName)
 if petInitSym == nil {
  C.dlclose(petLib)
  C.free_pet(pet)
  return nil, fmt.Errorf("failed to find pet_init symbol")
 }

 // call the initialization function to setup the pet's speak function 
 // We can reuse the same call_pet_func helper because both 
 // pet_init and pet->speak  share the same function signature: 
 //   void (*)(Pet*)
 C.call_pet_func(petInitSym, pet)

 return &Pet{
  petLib: petLib,
  pet:    pet,
 }, nil
}

func (p *Pet) Delete() {
 if p.pet != nil {
  C.free_pet(p.pet)
 }
 // make sure to close the library for proper cleanup
 if p.petLib != nil {
  C.dlclose(p.petLib)
 }
}

Note: In production systems, hard-coded shared object paths and unchecked dlopenflags should be avoided. Consider configurable search paths, versioned shared objects, and stricter error handling.

Full Wrapper

That’s it! The following code consolidates the earlier wrapper snippets into a single code block.

package pets

/*
#cgo CFLAGS: -I../../c/include
#cgo LDFLAGS: -ldl -L../../lib -lpets

#include <dlfcn.h>
#include <stdlib.h>
#include <pets.h>

void call_pet_func(void* f, Pet* pet) {
 ((void (*)(Pet*))f)(pet);
}

*/
import "C"
import (
 "fmt"
 "unsafe"
)

type Pet struct {
 petLib unsafe.Pointer
 pet    *C.Pet
}

func New(name string, petType string) (*Pet, error) {
 // create the c pet context structure
 petName := C.CString(name)
 defer C.free(unsafe.Pointer(petName))
 pet := C.new_pet(petName)
 if pet == nil {
  return nil, fmt.Errorf("failed to create pet")
 }

 // get the appropriate pet library
 var petLibPath *C.char
 switch petType {
 case "dog":
  petLibPath = C.CString("../lib/libdog.so")
 case "cat":
  petLibPath = C.CString("../lib/libcat.so")
 default:
  return nil, fmt.Errorf("unknown pet type: %s", petType)
 }
 defer C.free(unsafe.Pointer(petLibPath))

 // load the shared library
 petLib := C.dlopen(petLibPath, C.RTLD_LAZY)
 if petLib == nil {
  return nil, fmt.Errorf("failed to load %s library", petType)
 }

 // lookup the proper initialization function symbol
 petInitSymName := C.CString("pet_init")
 defer C.free(unsafe.Pointer(petInitSymName))
 petInitSym := C.dlsym(petLib, petInitSymName)
 if petInitSym == nil {
  C.dlclose(petLib)
  C.free_pet(pet)
  return nil, fmt.Errorf("failed to find pet_init symbol")
 }

 // call the initialization function to setup the pet's function pointers
 C.call_pet_func(petInitSym, pet)

 return &Pet{
  petLib: petLib,
  pet:    pet,
 }, nil
}

func (p *Pet) Delete() {
 if p.pet != nil {
  C.free_pet(p.pet)
 }
 if p.petLib != nil {
  C.dlclose(p.petLib)
 }
}

func (p *Pet) Speak() {
 C.call_pet_func(unsafe.Pointer(p.pet.speak), p.pet)
}

func (p *Pet) Name() string {
 return C.GoString(&p.pet.name[0])
}

Putting it All Together

Now that we have a complete wrapper, we can use it like any other Go package. Below is a minimal main demonstrating runtime module loading.

package main

import (
 "fmt"
 "loading-pets/pets"
)

func main() {
 dog, err := pets.New("Buddy", "dog")
 if err != nil {
  fmt.Println("Error creating dog:", err)
  return
 }
 defer dog.Delete()

 cat, err := pets.New("Whiskers", "cat")
 if err != nil {
  fmt.Println("Error creating cat:", err)
  return
 }
 defer cat.Delete()

 fmt.Printf("Your pet's name is %s \n", dog.Name())
 dog.Speak()

 fmt.Printf("Your pet's name is %s \n", cat.Name())
 cat.Speak()
}

$ go run main.go 
Your pet's name is Buddy 
Buddy says: Woof! Woof! 
Your pet's name is 
Whiskers Whiskers says: Meow! Meow!

Final Takeaway

This pattern allows Go programs to safely interact with dynamically loaded C modules while preserving strong abstractions and minimizing unsafe code. Although cgo has limitations, particularly around function pointers, small C helper shims make it possible to build flexible, modular architectures in Go.

Full Source

The repository for this project can be found on my GitHub here:

[embed]GitHub - g-hurst/loading-pets: Runtime Interaction with Shared Objects in Golang Runtime Interaction with Shared Objects in Golang. Contribute to g-hurst/loading-pets development by creating an…github.com