|
1 | 1 | # Caliscope |
2 | 2 |
|
3 | | -Caliscope is an open-source multicamera calibration tool. It produces the intrinsic and extrinsic camera parameters that downstream tools need for 3D reconstruction, whether you are tracking animal behavior with [SLEAP](https://sleap.ai/) or [DeepLabCut](https://www.mackenziemathislab.org/deeplabcut), feeding calibration into an [anipose](https://anipose.readthedocs.io/) pipeline, running biomechanical analysis with [Pose2Sim](https://github.com/perfanalytics/pose2sim), or building something else entirely. If you need help fitting Caliscope into your pipeline, please [open a discussion](https://github.com/mprib/caliscope/discussions). |
| 3 | +Caliscope is a permissively licensed multicamera calibration tool. |
| 4 | +It produces the intrinsic and extrinsic camera parameters that downstream tools need for 3D reconstruction. |
| 5 | +The objective of this project is to **Do One Thing Well**, and that one thing is estimating camera parameters. |
| 6 | + |
| 7 | +If you sense that there *must* be a good way to calibrate your setup but existing tools don't quite cut it, please [open a discussion](https://github.com/mprib/caliscope/discussions). |
| 8 | +I (mprib) am interested in exploring whether Caliscope can be upgraded to accommodate your use case. |
4 | 9 |
|
5 | 10 | ## The calibration problem |
6 | 11 |
|
7 | | -Multicamera 3D reconstruction requires knowing each camera's optical properties (intrinsic calibration) and its position and orientation in space (extrinsic calibration). Getting these parameters right is the foundation of accurate triangulation. Getting them wrong produces errors that propagate silently through every downstream analysis. |
| 12 | +Multicamera 3D reconstruction requires knowing each camera's optical properties (intrinsic calibration) and its position and orientation in space (extrinsic calibration). |
| 13 | +Getting these parameters right is the foundation of accurate triangulation. |
| 14 | +Getting them wrong produces errors that propagate silently through every downstream analysis. |
| 15 | +Calibrating more than two cameras requires bundle adjustment, a nonlinear optimization that simultaneously refines all camera positions and 3D point estimates. |
| 16 | +Bundle adjustment is powerful but sensitive to its starting point: a poor initial estimate can cause it to converge to a poor solution or fail entirely. |
| 17 | +When the initial estimate is good, bundle adjustment converges quickly and reliably. |
| 18 | +Caliscope is designed to produce that good initial estimate and rapidly solve for a quality calibration. Some important strategies that facilitate this: |
| 19 | + |
| 20 | +- **Pairwise PnP initialization:** |
| 21 | +Caliscope builds an initial estimate of camera positions from pairwise relationships using PnP (Perspective-n-Point). |
| 22 | +For each pair of cameras that both see the calibration target in the same frame, it estimates their relative position. |
| 23 | +It then chains these pairwise estimates transitively: if the relationship between cameras A and B is known, and B to C is known, A to C can be inferred. |
| 24 | +This produces a reliable starting point, even when no single position of the target is visible to all cameras at once. |
8 | 25 |
|
9 | | -OpenCV provides functions for stereo calibration between two cameras, but most practical setups benefit from three or more. Additional viewpoints reduce occlusion and improve triangulation accuracy. Calibrating more than two cameras requires bundle adjustment, a nonlinear optimization that simultaneously refines all camera positions and 3D point estimates. Bundle adjustment is powerful but sensitive to its starting point: a poor initial estimate can cause it to converge to a wrong solution or fail entirely. |
| 26 | +- **Flexible targets:** |
| 27 | +The PnP approach allows a single ArUco marker printed on a standard sheet of paper to serve as the extrinsic calibration target. |
| 28 | +Large-format prints are visible from greater distances, making it practical to calibrate wide capture volumes. |
10 | 29 |
|
11 | | -Caliscope addresses this by building an initial estimate from pairwise relationships. For each pair of cameras that both see the calibration target in the same frame, it estimates their relative position using PnP (Perspective-n-Point) estimation, a general technique that does not depend on OpenCV's stereo calibration. It then chains these pairwise estimates together transitively: if the relationship between cameras A and B is known, and B to C is known, A to C can be inferred. This produces a rough but reliable starting point for bundle adjustment, even when no single position of the target is visible to all cameras at once. |
| 30 | +- **Mirror boards:** |
| 31 | +For setups where cameras face inward from all directions (common in animal behavior rigs and dense multicamera arrays), finding board positions visible to every camera can be difficult. |
| 32 | +Caliscope supports a charuco board printed on both sides of a rigid surface (mirror image on back), so cameras viewing opposite sides identify the same physical point from either direction. |
| 33 | +**This allows cameras to be linked via PnP even if they never share a common view of the board.** |
12 | 34 |
|
13 | | -This pairwise approach also opens up flexibility in calibration targets. A single ArUco marker printed on a standard sheet of paper can serve as the extrinsic target. Large-format prints are visible from greater distances, making it practical to calibrate wide capture volumes. For setups where cameras face inward from all directions (common in animal behavior rigs and dense multicamera arrays), finding board positions visible to every camera can be difficult. Caliscope supports mirror boards: a calibration pattern printed on both sides of a rigid surface, so cameras viewing opposite sides identify the same physical point from either direction. This enables calibration of camera arrangements that would be difficult to handle with existing tools. |
| 35 | +- **Visual feedback:** |
| 36 | +During intrinsic calibration, you can inspect the fitted distortion model to catch problems early. |
| 37 | +During extrinsic calibration, you can see a 3D representation of the cameras along with the calibration points moving through space. |
| 38 | +World scale accuracy is visualized across frames based on the distances between target calibration points at each frame. |
14 | 39 |
|
15 | | -The GUI provides feedback throughout the process. During intrinsic calibration, you can inspect the fitted distortion model to catch problems early. During extrinsic calibration, you can see reprojection errors after bundle adjustment, filter outliers, and verify scale accuracy against the known geometry of your calibration target. |
16 | 40 |
|
17 | 41 | ## Integration with other tools |
18 | 42 |
|
19 | | -Caliscope's primary output is a calibrated camera array: intrinsic parameters (focal length, distortion) and extrinsic parameters (position and orientation) for every camera in your system. Alongside the native `camera_array.toml`, Caliscope automatically exports `camera_array_aniposelib.toml` in the format used by [aniposelib](https://github.com/lambdaloop/aniposelib). Tools that consume aniposelib calibrations can use this file directly. |
| 43 | +Caliscope's primary output is a calibrated camera array: intrinsic parameters (focal length, distortion) and extrinsic parameters (position and orientation) for every camera in your system. |
| 44 | +Alongside the native `camera_array.toml`, Caliscope automatically exports `camera_array_aniposelib.toml` in the format used by [aniposelib](https://github.com/lambdaloop/aniposelib). |
| 45 | +Tools that consume aniposelib calibrations can use this file directly. |
20 | 46 |
|
21 | 47 | ## Tracking and triangulation |
22 | 48 |
|
23 | | -Caliscope includes a basic reconstruction pipeline that tracks 2D landmarks and triangulates them into 3D trajectories. You can load custom ONNX pose estimation models exported from SLEAP, DeepLabCut, RTMPose, or other frameworks. Built-in MediaPipe trackers are included for convenience. Output is available in CSV and TRC (OpenSim) formats. Signal processing (gap filling, smoothing, filtering) and biomechanical modeling are outside the current scope. |
24 | | - |
25 | | -## Acknowledgments |
26 | | - |
27 | | -Caliscope was inspired by [anipose](https://anipose.readthedocs.io/), which demonstrated the value of accessible multicamera calibration for the research community. Caliscope focuses on providing GUI feedback to identify problems in the calibration process and on using granular parameter initialization to improve the likelihood of calibration success. |
| 49 | +Caliscope includes a basic reconstruction pipeline that tracks 2D landmarks and triangulates them into 3D trajectories. |
| 50 | +You can load custom ONNX pose estimation models exported from SLEAP, DeepLabCut, RTMPose, or other frameworks. |
| 51 | +Output is available in CSV and TRC (OpenSim) formats. |
| 52 | +For more complete reconstruction workflows, tools like [anipose](https://anipose.readthedocs.io/) and [Pose2Sim](https://github.com/perfanalytics/pose2sim) will serve you better. |
| 53 | +Caliscope's aniposelib-compatible export (see above) makes it straightforward to use Caliscope for calibration and hand off to these tools for downstream processing. |
28 | 54 |
|
29 | 55 | ## Getting started |
30 | 56 |
|
31 | | -The [Installation](installation.md) guide covers setup. [Project Setup](project_setup.md) explains the workspace directory structure and file naming conventions. A [sample project](sample_project.md) with downloadable data demonstrates the full pipeline. |
| 57 | +The [Installation](installation.md) guide covers setup. |
| 58 | +[Project Setup](project_setup.md) explains the workspace directory structure and file naming conventions. |
| 59 | +A [sample project](sample_project.md) with downloadable data demonstrates the full pipeline. |
32 | 60 |
|
33 | | -If you encounter a bug or have a feature request, please [open an issue](https://github.com/mprib/caliscope/issues). For questions, post in [Discussions](https://github.com/mprib/caliscope/discussions). |
| 61 | +If you encounter a bug or have a feature request, please [open an issue](https://github.com/mprib/caliscope/issues). |
| 62 | +For questions, post in [Discussions](https://github.com/mprib/caliscope/discussions). |
0 commit comments