Description

Aim:

To calibrate the camera by obtaining it's calibration matrix and undistort already existing images using classical Computer Vision techniques

​

Key concepts:

• Camera Extrinsic Matrix: Transforms points from the world coordinate system to the camera coordinate system.

• Camera Intrinsic Matrix: Transforms points from the camera coordinate system to the pixel coordinate system.

• Homography Matrix: Relates the 2D points of a plane in the image to their corresponding 3D points in the world. Each image has its unique Homography because of the specific rotation and translation.

​

Methodology:

​

1. a. Initialization:

   1. Start by reading the images which are to be calibrated. This usually includes images of a known calibration object (like a chessboard).

   2. Initialize a V matrix filled with ones. This will be used later for singular value decomposition (SVD).

   3. Prepare empty lists to store computed homography matrices and detected chessboard corners.

2. For Each Image

   1. Convert the image to grayscale. This is usually done to simplify the process of corner detection.

   2. Detect the chessboard corners in the image using OpenCV’s cv2.findChessboardCorners function.

   3. Save only the four outermost corners of the detected chessboard.

   4. Specify corresponding 3D world coordinates for these corners manually.

   5. Compute the Homography matrix using these 2D-3D point correspondences.

   6. Use the computed homography to get two V vectors, which are then stacked vertically to update the V matrix.

3. Deriving Intrinsic Parameters:

   1. Perform singular value decomposition on the V matrix (excluding the initialized row of ones) to derive the intrinsic camera parameters. These parameters include the focal lengths (f_x, f_y), principal points (c_x, c_y), and skew coefficient (gamma).

4. Refinement with Least Squares

   1. Obtain an initial estimate of the camera's intrinsic parameters, and set initial distortion parameters (k1, k2) to zero.

   2. Utilize a non-linear optimization method (least_squares function) to refine these estimates. The optimization is done based on minimizing the reprojection error between observed 2D points and reprojected 2D points from the 3D world.

5. Deriving Extrinsic Parameters

   1. For each computed homography matrix, derive the extrinsic parameters, i.e., the rotation and translation matrices that define the camera's pose with respect to the calibration object.

6. Computing Reprojection Errors

   1. After obtaining the refined intrinsic and extrinsic parameters, compute the reprojection error for each image. This error quantifies how accurately the 3D world points are projected onto the 2D image plane given the estimated camera parameters.

7. Final Image Undistortion and Visualization

   1. Using the derived camera matrix and distortion coefficients, undistort each image using the cv2.undistort function.

   2. Calculate the final error across all images, which gives an idea of the calibration's accuracy.

   3. For visualization, draw detected chessboard corners on the images and save them.

8. Results

   1. The calibrated intrinsic matrix, distortion coefficients, and the reprojection error are the outputs of this calibration process. They are vital for tasks like 3D reconstruction, image rectification, and object pose estimation

​

​