Avaliação de Diferentes Métodos para Calibração de Câmera Não-Métrica Evaluation of Different Methods for Non-Metric Camera Calibration

Calibration of a non-metric digital camera is a procedure that aims the modeling of systematic errors caused by lens distortion due to manufacturing and assembly process. This procedure must be carried out in order to improve the accuracy of a project. In addition, in photogrammetric measurements it is essential to comprehend the interior orientation parameters to model the distortions and generate trustful cartographic products. The camera calibration is needed more often for non-metric camera due to its low geometric stability. In the case of a commercial off-the-shelf digital camera, its interior orientation parameters are sensitive to external exposure and other factors, these characteristics creates the necessity of calibrating the sensor before any data acquisition. There are difference on the calibration methods, where some approaches requires more time, more elaborated data and sophisticated algorithms, such as the calibration using ground control points, on the other hand, there are faster and automated approaches that applies computer vision to reduce any human interaction. In this paper, the quality of these two different approaches for camera calibration was investigated. The first calibration is called “GCP-based” and it is based on georeferenced data processed with commercial software, and the second calibration is called `̀ Chessboard-based” and applies computer vision algorithms to estimate the parameters using a planar chessboard with black and white pattern and known dimensions. As result, the planimetric RMSE were compared with the reference coordinates, better accuracy was obtained with Agisoft PhotoScan software, with a RMSE of 1.4 cm.


Introduction
The process of generating tri-dimensional (3D) information from a sequence of bi-dimensional (2D) images has made Photogrammetry very popular mainly due to the use of Unmanned Aerial Vehicles (UAV). Nowadays, it is possible to generate precise and reliable information from data acquired with low-cost digital sensors carried by UAV and processed with photogrammetric software (Neitzel & Klonowski, 2012). Considering the 3D information product, the Digital Terrain Model (DTM) is one of the most used products that are applied, for example, to civil engineering projects.
The application that uses a DTM requires a large set of precise and reliable information. The process of generating photogrammetric products using non-metric cameras can reproduce the systematic errors caused by the lens distortion, if the calibration is not considered. The difference in cost between a metric camera and a non-metric camera can express the geometric stability of the digital sensor. Metric cameras are expensive if compared to a non--metric, and in order to extract information derived from a low-cost camera it is necessary to calibrate the sensor.
Camera calibration is the process of estimating the Interior Orientation Parameters (IOP) of a camera, the aim is to use the parameters to correct the systematic errors caused by the low geometrical stability of the materials that are part of the lens system. Distortion is a physical phenomenon that in certain situations may greatly impact the images geometry without impairing quality nor reducing the semantic information present in the image (Drap & Lefèvre, 2016). Accurate IOP are prerequisite to apply the correction of lens distortion and generate precise and reliable 3D data from a set of different 2D images for survey applications, such as aerial mapping and mobile mapping applications (Remondino & Fraser, 2006).
Nowadays, there are efficient methods to estimate accurate IOP from different types of cameras. Where some calibration methods, such as the materialization of a 3D calibration field with Ground Control Points (GCP) and processed with photogrammetric software (Hamid & Ahmad, 2014;Hastedt & Luhmann, 2015a, Campos et al., 2015 requires a specific calibration apparatus, on the other hand, the calibration with a planar chessboard with black and white pattern can achieve satisfactory results with a more practical and faster approach as presented by Zhang, (2000), Douterloigne et al. (2009) e De la Escalera & Armingol (2010.
With the popularization of low-cost UAV platforms, off-the-shelf cameras and automated photogrammetric software, some professionals do not pay attention to the proper photogrammetric procedures and sometimes do not even apply the calibrated IOP on the photogrammetric pipeline. The aim of this paper is to evaluate the IOP impact on 3D extraction, based on two different camera calibration: using a chessboard and using a calibration field with GCP.

Lens Distortion
Camera lenses causes optical aberrations that generates a point displacement on the image space as a result of the design and manufacturing process (Galo & Tommaselli, 2011), however, these aberrations do not have necessary impact on the image quality (McGlone et al., 2004). The lens distortion is different for each camera. In general, digital cameras presents two main lens distortion, the radial and the tangential (Brown, 1971), also, it is important to notice that a digital camera usually exhibits significant radial distortion (Zhang, 2000). According to Mc-Glone et al. (2004), if a square grid is photographed in object space, the recorded image is affected by pincushion distortion, illustrated in Figure 1a, barrel distortion, illustrated in Figure 1b and tangential distortion, illustrated in Figure 2.

Non-metric Digital Camera
The camera used in this paper is a Canon EOS Rebel T3, illustrated on Figure 3. Commercial and off-the-shelf are one of the reasons for this equipment selection. The camera main characteristics are inserted on the images as additional information in the form of Exchangeable Image File Format (EXIF) and are used as initial input on the camera calibration process, see Table 1.

Traditional Mathematical Model
According to Kraus (2007) and Mikhail (2001), the main mathematical model for traditional Photogrammetry is the collinearity equation. It is based on the pinhole camera model, where a non-distorted image point coordinate (x,y) can be projected as a 3D point (and vice-versa). For this, it is necessary to know the camera Exterior Orientation Parameters (EOP) and the IOP (Equation 1; Equation 2).   (1)

Camera Model Canon EOS REBEL T3
(2) The model presents the systematic errors in the X-axis and the Y-axis as ∆x and ∆y respectively, xp and yp are the principal point displacement in millimeters and c represents the focal length in millimeters. The six EOP parameters specifies the perspective projection in object space as Xo, Yo and Zo, and its orientation, ω, φ and κ, are expressed in a 3x3 rotation matrix (r ij ). The point in object space that correspond to x and y is presented as Xp, Yp and Xp. According to Brown (1966Brown ( , 1971, the correction of the image coordinate can be modeled by applying the Equation 3 and Equation 4: Where k1, k2 and k3 are coefficients of radial distortion, p1 and p2 are coefficients of tangential distortion, x and y are the principal point coordinates with distortion and x̄ and ȳ are the point coordinates with distortion. One of the major issues found in this paper is the particularity of each software. There is a necessity of converting the IOP derived from Pix4D, Pho-toScan, OpenCVand Agisoft Lens in order to apply them on the traditional mathematical model presen-In which:  Luhmann (2015a, 2015b), the IOP conversion, Table 2, is based on the direct correlation between the focal length, in millimeters, and the distortion coefficients obtained from the calibration. W), and the calibration field was materialized as shown on Figure 4a, the image acquisition was planned aiming high overlapping images with redundant control points over the wall, resulting in 12 images, 33 ground control points and 14 checkpoints. The image acquisition used the photogrammetric principle called "Convergent Camera Arrangement'', shown on Figure 4b, and according to Kang and Ho (2012), in this camera arrangement, the captured image has clear convergent points represented in the objects in the scene and few vertical pixel mismatches between the corresponding points in each image plane. The GCP on the wall are printed targets and its local coordinates were acquired using a total station.
Photogrammetric software can simultaneously recover the IOP and EOP, this procedure is called phototriangulation or Bundle Block Adjustment (BBA). According to Remondino et al. (2017), the procedure is composed by two steps: a preliminary phase where 2D features are automatically detected and matched among images and then the execution of a BBA. The BBA provides a simultaneous determination of all system parameters along with the precision estimation and reliability of the extracted calibration parameters (Remondino & Fraser, 2006). For this method, two commercial photogrammetric software were used, Pix4D and PhotoScan, to process both images and GCP. However, the method is not fully automated, it means that part of the task is performed by a human operator and errors can be inserted during GCP identification.

Chessboard-based Calibration
This method is based on computer vision algorithms that can recognize black and white patter-

Calibration Methodology
This paper relies upon two different camera calibration methods to estimate the IOP of a Canon EOS Rebel T3. The first method is called "GCP-based" and is a calibration based on GCP in a local coordinate system, and the second method, called "Chessboard-based", where a computer vision algorithm can recognize black and white patterns on images, such as a 2D planar chessboard with fixed and known cells size. In order to apply the IOP on the mathematical model and obtain the 3D position of a point, it is necessary the EOP, in this paper was used the Agisoft PhotoScan EOP.

GCP-based Calibration
The GCP-based calibration was carried out on the University of Campinas (22.8184ºS, 47.0647º  Figure 5a, on planar images. Within this pattern, a series of characteristic points are located, which give vital information for the calibration procedure (cells camera coordinates) (De la Escalera & Armingol, 2010). The procedure consists of a closed-form solution, followed by a nonlinear refinement based on the maximum likelihood criterion (Zhang, 2000). For this method, two different calibration using a chessboard were performed to estimate the IOP by considering 17 images. The first calibration was done by Open-CV, one of the advantages is the simultaneous estimation of the IOP and EOP, Figure 5b. The second calibration was done using the Agisoft Lens software, developed by Agisoft, and it can estimate more than 5 distortion parameters.

Root Mean Squared Error (RMSE)
No measurement is ever exact (Ghilani & Wolf, 2006). According to Li (2010), Root Mean Squared Error (RMSE) measures the differences between values predicted by a hypothetical model and the observed values. Therefore, it will be used to measures the quality of the fit between the reference data, acquired with total station, and the predicted model. RMSE is one of the most frequently used measurement index for generalized regression models.
The mathematical model, in which the coordinates can be expressed in the X, Y and Z-axis, is presented on Equation 5.

Results and Analysis
The set of experiments refers to the Canon T3 calibration. It was done using 4 calibration tools, described in the calibration methodology section. The image acquisition was performed under the same circumstances of temperature and light exposure. In order to maintain the same geometric properties, such as constant focal length and distortion parameters, the image acquisition of the 3D calibration field and the 2D chessboard was done at the same day and close to each other.

Interior Orientation Parameters
An important aspect related to processing consistencies in the camera calibration process is to use the same computer to carry out all the calibration. Within the calibration results from each method, the acceptance of the estimated IOP was based on its standard deviation, the parameters were analyzed, converted using the Table 2 and then compiled as illustrated on Table 3.
Comparing the IOP stated on the Cannon T3 user manual to the estimated parameters from the presented methods, the recovered parameters from PhotoScan and Pix4D were quite similar, with approximately a difference of 1.21 mm on the focal  Marcos Vinicius Yodono Garcia; Henrique Cândido de Oliveira; Rafael Francisco Fernandes & Diógenes Cortijo Costa length, followed by Agisoft Lens, with approximately a difference of 1.00 mm, and OpenCV, with a difference of 0.52 mm.

Comparison Considering 3D Points on Object Space
For the proposed method, a comparison is presented on Table 4, with the following information: Point ID (check points); Reference coordinates of check points (from total station); coordinates from photogrammetric intersection without considering IOPs (using only datasheet information); coordinates obtained via intersection considering the   IOP from GCP method (PhotoScan and Pix4D) and Chessboard method (OpenCV and Agisoft Lens).
The table shows the reference coordinates, the triangulated coordinates without lens distortion and the computed coordinates triangulated with the parameters derived from the methods. To determine the coordinates of a point on object space, were used the EOP obtained from the Agisoft Pho-toScan software.
The misinformation about camera calibration can lead professionals to generate products with systematic errors. In the case of a Canon T3 EOS camera, using the parameters stated on the camera datasheet could generate a planimetric RMSE of 22.2 cm. Analyzing the Table 4, the results regarding planimetric RMSE presented the following values: 1.4 cm (PhotoScan), 1.7 cm (Pix4D), 23.8 cm (Open-CV), and 2.2 cm (Agisoft Lens).
Although the method that provided better accuracy is the GCP-based calibration executed by commercial software, the Chessboard-based calibration presented acceptable results with Agisof Lens. An interesting fact found on this research is the Agisoft Lens capacity of delivering coordinates with near accuracy from Pix4D Mapper, a software that  has complex feature recognition algorithms and uses GCP as known points on object space. The Agisoft Lens and OpenCV are fully automated, it means that there is no systematic errors designed by a human operator placing a GCP in the software, besides the acceptable accuracy, in these experiments Agisoft Lens is the faster solution for camera calibration. The results presented by OpenCV may be improved by inserting more images in the process (experiment not covered in this paper).

Conclusion
This paper has reviewed two different methods for digital camera calibration. As the use of low-cost digital cameras for photogrammetric applications is becoming more popular, there is a necessity to adopt the appropriate calibration procedure.
The estimated IOPs of 4 software are compared with respect to a photogrammetric approach. For the results of a Canon T3 camera it can be summarized that using a GCP-based calibration instead of a Chessboard-based, accurate parameters are estimated. For the Canon T3 itself, a high variation between the standard configuration and the calibrated parameters can be observed. The impact of the IOP causes a loss in accuracy of cartographic products, depending on the set of estimated parameters.
Furthermore, the calibration of a Canon T3 allowed a experiment of two methods, the GCP--based and the Chessboard-based. The GCP-based calibration estimated accurate objects in all three coordinate directions but requires more time and powerful software. On the other hand, the Chessboard-based calibration delivered a relatively reliable parameters with a faster and easiest solution. With respect to camera calibration, if off-the-shelf camera is used for photogrammetric purposes, it is critical to be aware of the camera calibration impact on estimating accurate 3D points.
For future researches, it will be relevant to study the impact of a high number of processed images using OpenCV and performing the Chessboard--based method.

Acknowledgement
The researchers would like to thank the São Paulo Research Foundation (FAPESP) for the project funding (grant number 2017/17003-3), the Graduate Program in Civil Engineering (PPGEC) at University of Campinas (UNICAMP) for the facility and equipment and the UNICAMP Support Fund for Education, Research and Extension (FAEPEX), grant number 2859/18 given to the first author.