A Self-Calibration Method of Zooming Camera

— In this article we proposed a novel approach to self-calibrate a camera with variable focal length. We show that the estimation of camera’s intrinsic parameters is possible from only two points of an unknown planar scene. The projection of these points by using the projection matrices in two images only permit us to obtain a system of equations according to the camera’s intrinsic parameters . From this system we formulated a nonlinear cost function which its minimization allows us to estimate the camera’s intrinsic parameters in each view. The results on synthetic and real data justify the robustness of our method in term of reliability and convergence.


I. Introduction
T he self-calibration of camera [1,2,3,4,5,6,7,8,9,10,11,12,13,1   4,15,16] is a fundamental area in computer vision, it consist of estimation of camera's parameters with no knowledge about the observed scene.The principle of self-calibration consist to formulate equations based on the camera's intrinsic parameters by using invariant primitives (circular points, absolute conic...) in the scene space.These primitives describe the euclidean structure of the scene.
Our self-calibration method presented in this paper consist of calibrate a camera from a unknown planar scene and by considering that the camera's intrinsic parameters are constants except to the focal length which may be variable from one view to another which is suitable with the current applications of computer vision.The highlight of our idea lies partly in the use of only two points from an unknown plananar scene and the projection of this scene in only two image planes and secondly in the implementation of a relationship between images of the absolute conic in the both views and the projection matrices, to determinate, of the two points used in the scene.The procedure of our method of self-calibration described in this work can be presented in the form of the following steps: • Acquisition of two images of an unknown planar scene.
• Detection of interest points in the two images by using the Harris detector.
• The matching of the interest points by using the ZNCC correlation measure • Estimation of the projection matrices of the two points of the scene • Development of a relationship between the projection matrices estimated in the previous step and the images of the absolute conic in the two images of the scene.
• Formulation of a non-linear cost function on the basis of the relation found in the previous step.
• Optimization of the cost function by the Levenberg-Marquardt [17] through two phases: • Initialization phase providing an initial solution by applying constraints on the two images and some intrinsic parameters of the camera.
• Minimizing of the cost function to find the optimum solution according to the intrinsic parameters of our camera in the two images of the scene.
The present article is organized as follows: the second part has a review of previous works dealing with the problem of self-calibration.The self-calibration tools used in this article are discussed in the third section then our method of self-calibration is addressed in the fourth section.Our experimental results are presented in the fifth section and the last part contains a conclusion of our work.

II. Related works
In this section we present with a non-exhaustive manner some works done in the field of the camera self-calibration.In [18] the singular value decomposition of the fundamental matrix gives a simple form of Kruppa equations that solves the problem of self-calibration of cameras with varying intrinsic parameters.A new method for camera self-calibration performing a sequence of critical motions is processed in [19].In this method, the intrinsic parameters are assumed to be known except to the focal length which may vary freely from one view to another.The determination of the intrinsic parameters is based on the use of the image of the absolute conic in the different views.In [20], the authors are proposed a new method of self-calibration from a sequence of images of an unknown 3D scene.The projection of the scene points in the image planes is used with the fundamental matrix to determine the projection matrices of scene points.These projection matrices are used to formulate a nonlinear cost function which resolution can estimate the intrinsic parameters in the different views.In [21], the resolution of three linear equations obtained from circles and their centers respectively determine the vanishing line.This vanishing line and the circular points are used to estimate the camera's intrinsic parameters.The problem addressed in [22] is the self-calibration of the cameras with varying intrinsic parameters, This method is based on a quasi-affine reconstruction (by localization the plane in infinity ) which provides a system of equations based on the intrinsic parameters of the camera.The resolution of this system gives the intrinsic parameters in each view.In [23] the authors have proposed a linear approach to self-calibrate a camera performing a small rotation and general translation with the possibility of variation of the focal length.This method is based on using the homography induced by the plane at infinity and the absolute quadric.In [24], a method of camera self-calibration with varying intrinsic parameters is presented.It's based on the quasi-affine reconstruction, after thissreconstruction the homography of the plane at infinity is determined.This homography is used with constraints on the image of the absolute conic to estimate the intrinsic parameters of the camera [29].

III. Camera model and image of absolute conic
With R is the rotation matrix and t the translation vector of the camera in space and ξ is the matrix of the intrinsic camera parameters defined by: l : Focal length.
Considering the space of our scene is a projective space.In this space the points satisfying 0 w = are called the points at infinity, these points form the plane at infinity.In the plane at infinity the points satisfying 0 T M M = form the absolute conic denoted ∞ Ω .The absolute conic is a complex conic of the plane at infinity, the image of absolute conic ω is given by: 1 ( ) From ( 3) we deduce that the image of the absolute conic depends only on the intrinsic parameters of the camera.Once ω is estimated we can easily estimate the intrinsic parameters of the camera using the Cholesky factorization.

IV. Proposed method
The main idea of our new self-calibration method is based on the projection of two points of an unknown planar scene in only two image planes using two projection matrices which must be estimated.for k = i or j its projections in the two images (see Fig. 1).From (1) we have for the image i :

Assuming that the plane containing the segment [ ]
with qi r is the column number q of i R .
The equation ( 4) can be rewritten as follows: ) ) i H is a 3x3 matrix called the homography matrix scene to image i.
The equation (5) becomes: Let ( ) r and 2i r are orthogonal which permit us to write from (6) :  ( )

B x y =
From (7) we have for image i: For image j: From ( 9) and ( 10) we can write: And: Let: The equations (11) and (12) become: And: The matrices i P and j P are the projection matrices of the points A and B in the both images i and j.
From (13) we can write: is the homography matrix between the images i and j.From ( 14), (15) and ( 16) we can write: The matrix i P consists of eight unknowns and the above system gives us twelve equations that allows us to estimate the matrix i P .The matrix j P can be estimated from (16).

P p p p =
, with the ki p are the column vectors of i P .From (13) we can deduce: The system of equations (18) gives us: 2 cos( ) 2 sin( ) By replacing in (9)   , we obtain for image i : And for image j : From ( 20) and ( 21) we can deduce that: From ( 16) we can deduce: By replacing 1 j p and 2 j p by its expressions in (22) we obtain: By developing the first equation in (13) we obtain for image i: ) is the image of absolute conic in image i and For the image j: Let: From ( 25) and ( 26) we can deduced that the first two rows and columns of the matrices  20), ( 21), ( 22), ( 24) and ( 29) we can deduce the following system:

p p p p p p p p p p p p h p h p h p h p p p e e e e
e e e e ω ω ω ω ω ω In the present article we are assumed in the one hand that the intrinsic camera parameters are unknown and constant except for the focal length which can be vary from one view to another and in the second hand we are used two images only of an unknown planar scene which gives us six unknown, the system of equations (30) contains six equations where the possibility to estimate the intrinsic parameters of camera in the both views.
The equations of the system (30) are nonlinear therefore to solve it we minimize the cost function given by the following expression: with:

p p p p p p p p p p p p p h p h p h p h p p p e e e e e e e e
β ω β ω And ij ∆ is the vector of the intrinsic parameters of the camera in the two images i and j and n is the number of images used.
The minimization of (31) is given by the Levenberg-Marquardt algorithm which requires an initialization step, for this purpose we assume the following assumptions: • The main point is in the center of the image, so By substituting these parameters in the system of equations (30) we can determine the focal lengths i l and j l for the both images i and j .
To test the effectiveness of our method two types of data are used synthetic and real.

A. Simulation
In this simulation a sequence of ten 512x512 images of a calibration target is used to test the effectiveness of our self-calibration method presented in this paper.After the detection of interest points by the Harris detector [25] the corresponding points between images are determined using the correlation measure ZNCC [26].The images are perturbed by adding a Gaussian noise of deviation σ to the pixels of images.The projection of the scene in image planes allows us to obtain linear equations whose its solution allows to determine the projection matrices.The self-calibration equations are then built to formulate a nonlinear cost function which its minimization by the Levenberg-Marquardt algorithm helps us to estimate the intrinsic parameters of our camera in all views.To evaluate the performance of our approach we compared the results obtained with those given by a well-known calibration method [6] published by Z. Zhang and those given by [27].The figures ( 2) and (3) consecutively present the relative error of the focal length estimated by our method and those of Z. Zhang and C-R.Huang according to the number of images used and added noise.The both figures above show that the method developed in this paper and that of Z.Zhang gives the approximate results.By taking into account the robustness of the Z. Zhang's method we can say that our method is effective and efficient.On the other hand, the previous figures prove the preference of our method compared with that presented in [27].

B. Real images
In this second experiment, two 512 × 512 images of a real planar scene are acquired by a CCD camera having constant intrinsic parameters except to the focal length.(See Fig. 4).The interest points are determined as well as their matching and calculation of projection matrices.

Detection of interest points
The interest points in the two images are detected by using the Harris detector (Fig. 5).

Matching of interest points
The interest points previously identified are matched between the two images by using the ZNCC algorithm (Fig. 6).

Estimation of intrinsic parameters
This step consists of estimating the intrinsic parameters of the camera in each view by applying our method.The following table shows the results obtained.The above table presents the results obtained by our method of self-calibration discussed in this article.To verify the effeteness of the above results, we rectified the two used images by using a robust approach of image rectification [28] based on a calibrated camera.The following figure shows the rectified images obtained by applying [28] and using the values of table 1.The goal of image rectification is to obtain the epipolar lines horizontal and parallel to the x-axis, according to the Fig. 7 we can deduce that our approach presented in this paper gives satisfactory results and this by taking into account that our method is able to selfcalibrate a camera with a variable focal length from two images only based on two points only of an unknown planar scene.

VI. Conclusion
In this work we have proposed a new method of self-calibration of a CCD camera with constant intrinsic parameters except to the focal length which can vary freely from one view to another from an unknown planar scene.We have shown that the determination of the intrinsic parameters is possible by using only two points of the scene.The projections of the two points in the two image planes permit us to determine their projection matrices, these matrices are used to formulate a nonlinear cost function.The minimization of the cost function obtained allowed us to estimate the intrinsic parameters of the camera.The experimental results on the synthetic and real data prove the robustness of our method.
Our camera is modeled by a model called pinhole.Denote by z w = an image point.The projection of the M in its image m is denoted by: Let (S) an unknown planar scene and ( , , ,1)

Fig. 2 .
Fig. 2. Relative error on the focal length according to the number of images used.

Fig. 3 .
Fig. 3. Relative error on the focal length according to the added noise

Fig. 4 .
Fig. 4. The two real images used