Kinect Stereo Calibration


After looking at camera calibration and stereo calibration theoretically.. this post will introduce you to the world of calibrating the kinect. Kincet has an IR camera pair and an RGB camera. Working of IR camera pair has been discussed and explained in detail in lot of posts by others and also by me. Thus getting into the topic, kinect calibration literary means to get a transformation between the co-ordinates of depth camera and RGB camera. This can also be said as Kinect Stereo Calibration as we are calibrating two camera against each other. In short, the output of this calibration will be an RGB-D image, or the Image will be an overlap of RGB and Depth Streams.

To have this transformation For this it is necessary to get the internal characteristics of  both the IR and RGB camera. You can either use the Calibration API from OpenCv or use the Matlab Calibration ToolKit to find the internal Parameters. I have used Matlab Calibration Toolkit since it is easy to use and more reliable. Following are the steps for kinect stereo calibration

  • RGB with the chess board as the tool box uses Tsai calibration technique
  • IR images that is obtained using kinect contains granules i.e the receiver image that is contains a pattern of IR Dots that makes the image not suitable for calibration. It is better to close the kinect emitter have an IR projector/emitter that does not emit a pattern but emit clear IR rays so that the image obtained at the receiver is clean. But since I do not have clean emitter I have used the same images and the results are satisfactory
  • Calibrate both the cameras with the obtained images

External R and T matrices are found the same way but this time keeping the ches board constant for RGB and Depth-IR images.

p_rgb = point in rgb image P_rgb = rgb 3D point
p_ir = point in ir image P_ir = ir 3D point

P_ir.x = (p_rgb.x-Cx)*D/FOCALX 
P_ir.y = (p_rgb.y-Cy)*D/FOCALY
P_ir.z = D

P_rgb = R*P_ir + T
p_rgb.x = P_rgb.x*FOCALX/P_rgb.z
p_rgb.y = P_rgb.y*FOCALY/P_rgb.z

Thus p_rgb is the point corresponding to p_ir point on the IR image. P_rgb.z is the distance of p_rgb from the RGB camera.

The results that I have obtained are

The last image is the image obtained by overlapping the depth(2 nd image) and the RGB(1st image)

5 thoughts on “Kinect Stereo Calibration

  1. hi, there:
    I have a question about how to get the external matrix R and T.
    as you said, “keeping the chessboard constant for RGB and Depth-IR images”,
    does that mean you have to capture first a rgb image and then capture an IR image.
    And then change the postion of the chessboard, do the same thing above?
    If that ‘s the case, how many pairs of IR and RGB images have you used to obtain the R and T matrix?

    Thanks a lot

    • To get the relative R and T Matrix, the reference co-ordinate system must be same for both the images IR and RGB. This can be obtained by keeping the ” chessboard constant”, I mean the cess board should not be disturbed when the R and T matrices are being calculated this becomes the constant reference to both the cameras. For R T mats 5 RGB and 5 IR images should be good, as you would not be seeing any appreciable change in the corresponding R T mats for large number of images. The mean of the Rrgb,Rir and Trgb,Tir mats can be taken as the final Rrgb Tir to calculate the relative R T mats

    • HI,
      By final result, do you mean the RGB-depth calibrated image? If it is, then, I got it by saving the images from the video recorded with kinect with OpenNI. The R and T matrix were calculated beforehand an hard-coded into the code.

  2. Hi There;
    Do both the RGB and IR camera be capturing at the same resolution? Or can they be different from each other? like 640X480 and 320X240?

    Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s