After looking at camera calibration and stereo calibration theoretically.. this post will introduce you to the world of calibrating the kinect. Kincet has an IR camera pair and an RGB camera. Working of IR camera pair has been discussed and explained in detail in lot of posts by others and also by me. Thus getting into the topic, kinect calibration literary means to get a transformation between the co-ordinates of depth camera and RGB camera. This can also be said as Kinect Stereo Calibration as we are calibrating two camera against each other. In short, the output of this calibration will be an RGB-D image, or the Image will be an overlap of RGB and Depth Streams.
To have this transformation For this it is necessary to get the internal characteristics of both the IR and RGB camera. You can either use the Calibration API from OpenCv or use the Matlab Calibration ToolKit to find the internal Parameters. I have used Matlab Calibration Toolkit since it is easy to use and more reliable. Following are the steps for kinect stereo calibration
- RGB with the chess board as the tool box uses Tsai calibration technique
- IR images that is obtained using kinect contains granules i.e the receiver image that is contains a pattern of IR Dots that makes the image not suitable for calibration. It is better to close the kinect emitter have an IR projector/emitter that does not emit a pattern but emit clear IR rays so that the image obtained at the receiver is clean. But since I do not have clean emitter I have used the same images and the results are satisfactory
- Calibrate both the cameras with the obtained images
External R and T matrices are found the same way but this time keeping the ches board constant for RGB and Depth-IR images.
p_rgb = point in rgb image P_rgb = rgb 3D point
p_ir = point in ir image P_ir = ir 3D point
P_ir.x = (p_rgb.x-Cx)*D/FOCALX P_ir.y = (p_rgb.y-Cy)*D/FOCALY P_ir.z = D P_rgb = R*P_ir + T p_rgb.x = P_rgb.x*FOCALX/P_rgb.z p_rgb.y = P_rgb.y*FOCALY/P_rgb.z
Thus p_rgb is the point corresponding to p_ir point on the IR image. P_rgb.z is the distance of p_rgb from the RGB camera.
The results that I have obtained are