Extrinsic Camera Calibration Part 1: COLMAP Rigs

Today I was working with COLMAP’s rig support because I eventually want to integrate my own multi-camera extrinsic calibration into COLMAP. I have written my own Bundle Adjustment Code for scale aware extrinsics calibration (More about that in Part 2). But before blindly feeding those poses into COLMAP, I wanted to properly understand how COLMAP represents camera rigs internally.

So in this first part, I decided to follow the COLMAP rig workflow using the ETH rig dataset, but without using the provided ground-truth calibration.

COLMAP Setup and Dataset collection

I am using COLMAP 4.1.0.dev0 (Commit c03cb50e on 2026–05–20 with CUDA) built from source. Please make sure you have latest version of COLMAP to use rig support.

• I downloaded ETH rig dataset using wget [https://www.eth3d.net/data/terrains_rig_undistorted.7z](https://www.eth3d.net/data/terrains_rig_undistorted.7z) 7zz x terrains_rig_undistorted.7z
• After downloading the ETH terrain rig dataset, my directory looked like this:

~/colmap/experiments/terrains$ ls
images  rig_calibration_undistorted

Inside images/, there are four folders, one for each camera:

~/colmap/experiments/terrains$ ls images/
images_rig_cam4_undistorted
images_rig_cam5_undistorted
images_rig_cam6_undistorted
images_rig_cam7_undistorted

The dataset also provides a calibration/reconstruction folder:

~/colmap/experiments/terrains$ ls rig_calibration_undistorted/
cameras.txt  images.txt  points3D.txt

But for this experiment, I did not use rig_calibration_undistorted.COLMAP’s rig model assumes that sensors mounted in a rig have fixed relative poses, one reference sensor defines the rig origin, and a frame corresponds to one exposure instance of the rig where all or some sensors are exposed together.

What is a rig in COLMAP?

A normal COLMAP reconstruction treats every image pose independently. For example, if I have four cameras and 165 timestamps, then I have: *4 × 165 = 660 images*

Without rig modeling, COLMAP estimates one independent pose per image:

COLMAP is treating each image as an independent camera instance

But when you define a multi-camera rig, the relative positions of the cameras are locked. COLMAP factors the absolute pose into a fixed relative component and a moving world component.

Component Breakdown:

• T_rig->cami (Sensor Pose Inside Rig): The rigid transformation from the rig’s baseline origin to camera i. This is static across all timestamps. COLMAP optimizes this once per camera.
• T_world->rig^{t} (Rig Pose in World): The rigid transformation from the world origin to the rig’s baseline origin at timestamp t. This changes at every frame as the rig moves through space.

Step 1 — Feature extraction

From the dataset root cd ~/colmap/experiments/terrains , I first created a new database for the no-rig experiment:

rm -f database_no_rig.db
mkdir -p sparse_no_rig

Then I extracted features:

colmap feature_extractor \
    --database_path database_no_rig.db \
    --image_path images \
    --ImageReader.single_camera_per_folder 1

The important option here is--ImageReader.single_camera_per_folder 1 .

Since my image folders are:

images_rig_cam4_undistorted/
images_rig_cam5_undistorted/
images_rig_cam6_undistorted/
images_rig_cam7_undistorted/

this tells COLMAP that All images inside one folder belong to the same camera/intrinsic model.

So COLMAP creates one camera model per folder. At this stage, this is still not rig modeling. It only means one camera/intrinsic model per image folder . The relative poses between these cameras are not known yet.

5. Step 2 — Sequential matching

Next, I ran sequential matching:

colmap sequential_matcher \
    --database_path database_no_rig.db

This dataset is sequence-like, so sequential matching is a reasonable starting point. At this point, COLMAP still does not know that:

images_rig_cam4_undistorted/frame_k
images_rig_cam5_undistorted/frame_k
images_rig_cam6_undistorted/frame_k
images_rig_cam7_undistorted/frame_k

belong to the same physical rig frame. It is still just doing standard image matching.

6. Step 3 — Ordinary no-rig reconstruction

Then I ran the normal mapper:

colmap mapper \
    --database_path database_no_rig.db \
    --image_path images \
    --output_path sparse_no_rig

This produced sparse_no_rig/0 .

This is the important baseline. At this point, COLMAP has reconstructed the scene, but it has treated all images as independent camera poses.

Conceptually:

sparse_no_rig/
    660 images
    660 independent image poses
    3D points
    camera intrinsics

This is the model that will later be used to infer the rig layout.

7. Step 4 — Creating an unknown-pose rig configuration

Now I created a rig configuration file:

$ cat > rig_config_unknown.json << 'EOF'
[
    {
        "cameras": [
            {
                "image_prefix": "images_rig_cam4_undistorted/",
                "ref_sensor": true
            },
            {
                "image_prefix": "images_rig_cam5_undistorted/"
            },
            {
                "image_prefix": "images_rig_cam6_undistorted/"
            },
            {
                "image_prefix": "images_rig_cam7_undistorted/"
            }
        ]
    }
]
EOF

This file says: These four image folders are four cameras belonging to one rig. Use images_rig_cam4_undistorted/ as the reference sensor. But I do not provide relative poses.

But what if you know the relative poses already?

If you already know the geometric calibration of your camera rig beforehand, you can skip the estimation phase by providing the exact spatial relationships directly. In COLMAP, a known-pose configuration file utilizes the precise mathematical layout shown below:

Example:

[
  {
    "cameras": [
      {
        "image_prefix": "rig1/camera1/",
        "ref_sensor": true
      },
      {
        "image_prefix": "rig1/camera2/",
        "cam_from_rig_rotation": [
            0.7071067811865475,
            0.0,
            0.7071067811865476,
            0.0
        ],
        "cam_from_rig_translation": [
            0,
            0,
            0
        ]
      }
    ]
  }
]

• cam_from_rig_rotation: Expects a 4-element array defining a unit quaternion formatted strictly as [qw, qx, qy, qz].
• cam_from_rig_translation: Expects a 3-element vector [tx, ty, tz]

NOTE: The translation vector t represents the origin (center) of the rig coordinate frame expressed in the camera’s coordinate frame. Because COLMAP uses a passive “world-to-camera” (or in this case, “rig-to-camera”) transformation convention, the vector *t* points from the camera sensor to the rig origin, measured along the camera’s internal coordinate axes.

8. Step 5 — Inferring the rig from the no-rig reconstruction

Now I ran:

$ mkdir -p sparse_inferred_rig
$ colmap rig_configurator \
    --database_path database_no_rig.db \
    --input_path sparse_no_rig/0 \
    --rig_config_path rig_config_unknown.json \
    --output_path sparse_inferred_rig

From the no-rig reconstruction, COLMAP already has independent poses for many images at timestamp t:

For each timestamp t, it can estimate relative transforms between cameras. For example, if camera 4 is chosen as the reference sensor (the rig origin), the relative transform to camera 5 at time t is:

Because the cameras are rigidly attached, this relative transform should be approximately the same across frames.

By calculating this across all available timestamps, COLMAP can average and infer the static sensor configurations relative to the rig base. (Note: If camera 4 is the reference, then Tworld->cam4 is simply the identity matrix I).

This produced: sparse_inferred_rig/ — your very first rig-aware model.

9. Step 6 — Convert the inferred rig model to text

To inspect what COLMAP produced, I converted the model to text:

$ mkdir -p sparse_inferred_rig_txt
$ colmap model_converter \
    --input_path sparse_inferred_rig \
    --output_path sparse_inferred_rig_txt \
    --output_type TXT

10. Inspecting rigs.txt

$ cd sparse_inferred_rig_txt && cat rigs.txt

The output was:

# Rig calib list with one line of data per calib:
#   RIG_ID, NUM_SENSORS, REF_SENSOR_TYPE, REF_SENSOR_ID, SENSORS[] as (SENSOR_TYPE, SENSOR_ID, HAS_POSE, [QW, QX, QY, QZ, TX, TY, TZ])
# Number of rigs: 1
5 4 CAMERA 1 CAMERA 2 1 0.99998123663127991 -0.00098065457948955798 0.0057296984204197382 0.0019326815523717854 -0.2850523688677073 0.14458305557808537 0.053821346432722135 CAMERA 3 1 0.99962714521434048 -0.02337040373316826 -0.013289933823573028 -0.0047720476674999795 -0.091748405935830363 0.044933971091494378 0.090088073108902073 CAMERA 4 1 0.99960587788845023 -0.015486746807735626 -0.02340004858121203 -0.00082902989701457645 -0.26318586917758019 -0.0099468495980400105 0.55931028732108579

The important beginning is:

5 4 CAMERA 1

This means:

RIG_ID = 5
NUM_SENSORS = 4
REFERENCE_SENSOR = CAMERA 1

So COLMAP has created one rig:

Rig 5
CAMERA 1: reference sensor
CAMERA 2: estimated fixed pose relative to rig
CAMERA 3: estimated fixed pose relative to rig
CAMERA 4: estimated fixed pose relative to rig

Each sensor pose is represented by: QW QX QY QZ TX TY TZ

11. Inspecting frames.txt

Then I inspected: head -30 frames.txt

# Frame list with one line of data per frame:
#   FRAME_ID, RIG_ID, RIG_FROM_WORLD[QW, QX, QY, QZ, TX, TY, TZ], NUM_DATA_IDS, DATA_IDS[] as (SENSOR_TYPE, SENSOR_ID, DATA_ID)
# Number of frames: 165
661 5 0.99895221673873424 -0.011798653933931053 -0.0050389743014514028 -0.04393027630354393 -5.1748469285411636 -0.19796698159591189 0.11293139847855183 4 CAMERA 1 1 CAMERA 2 166 CAMERA 3 331 CAMERA 4 496
662 5 0.9990715053390965 -0.01429855963775855 -0.0042871911794993851 -0.040414086696940944 -5.1146668902207626 -0.22966193657762393 0.13529958268892595 4 CAMERA 1 2 CAMERA 2 167 CAMERA 3 332 CAMERA 4 497
663 5 0.99898311893574665 -0.016404244941156807 -0.0057023490831047182 -0.041606634618204644 -5.0623932166083598 -0.21883785144499784 0.13108716427638623 4 CAMERA 1 3 CAMERA 2 168 CAMERA 3 333 CAMERA 4 498

The first data line is:

661 5 ... 4 CAMERA 1 1 CAMERA 2 166 CAMERA 3 331 CAMERA 4 496

This means:

FRAME_ID = 661
RIG_ID = 5
NUM_DATA_IDS = 4

CAMERA 1 -> IMAGE_ID 1
CAMERA 2 -> IMAGE_ID 166
CAMERA 3 -> IMAGE_ID 331
CAMERA 4 -> IMAGE_ID 496

A frame is not just an image. A frame is one physical capture instance of the rig:

Frame 661
├── one pose of the full rig in the world
├── image from CAMERA 1
├── image from CAMERA 2
├── image from CAMERA 3
└── image from CAMERA 4

The output also says: # Number of frames: 165

So the final structure is:

165 rig frames
4 camera observations per frame
660 total images

Before rig modeling, COLMAP had 660 image poses. After rig modeling, the reconstruction is organized as:

165 rig poses + fixed sensor poses inside the rig

12. Step 7 — Rig-aware bundle adjustment

After COLMAP inferred the rig structure, I ran bundle adjustment on the rig-aware model:

$ cd ~/colmap/experiments/terrains
$ mkdir -p sparse_inferred_rig_ba
$ colmap bundle_adjuster \
    --input_path sparse_inferred_rig \
    --output_path sparse_inferred_rig_ba

The output was:

I0525 16:53:50.659735 94592 bundle_adjustment.cc:70] === Global bundle adjustment ===
I0525 16:54:16.131942 94592 bundle_adjustment_ceres.cc:1137] Bundle adjustment report
    Residuals : 1982170
   Parameters : 240097
   Iterations : 47
         Time : 24.6994 [s]
 Initial cost : 2.98097 [px]
   Final cost : 0.434825 [px]
  Termination : CONVERGENCE

I0525 16:54:16.201613 94592 timer.cc:90] Elapsed time: 0.426 [minutes]

This was a good result. The reprojection error dropped from: Initial cost: 2.98097 pxto Final cost: 0.434825 px

One thing I noticed in the COLMAP GUI was that the camera positions looked almost the same as the original no-rig reconstruction.

Hope you found this article useful. In next part, I will discuss how I found extrinsic poses between cameras in my rig.