IMAGE Team - The Pantheon project
The goal of this application is the digitalization of whiteboard contents capture. This application locates the boundary of the whiteboard, rectifies the geometry distortion and corrects the non-uniform illumination.
 |  |
| Whiteboard capture. | Result image. |
The method is composed of 5 steps:
The detection is done with the Shen-Castan operator which provides a good detection and a good localization of the edges and allows a binarization without parameter (actually the threshold is set to 2 for all images).
pany2pan whiteboard.png whiteboard.pan prgb2gray 0.299 0.587 0.114 whiteboard.pan tmp3.pan pgaussianfiltering 1 tmp3.pan tmp18.pan pshen 1.3 tmp18.pan tmp6.pan pbinarization 2 255 tmp6.pan whiteboard1.pan |  |
| Edge image |
The detection is based on the Hough transform which is used to locate the nearly straight lines among the edges. The detection is done separately for the horizontal (between -20° et 20°) and the vertical lines (between 70° et 110°). For each, an initial closing by reconstruction is used to remove the small lines that are not in the expected direction. Next, the Hough transform is used to detect the longest lines (the 6 longest vertical lines and the 6 longest horizontal lines).
# Horizontal plinearerosion 0 0 1 whiteboard1.pan tmp7.pan pdilatationreconstruction 8 tmp7.pan whiteboard1.pan tmp9.pan phoughlines 6 -20 20 10 tmp9.pan tmp19.pan pdilatation 0 1 tmp19.pan tmp25.pan # Vertical plinearerosion 90 0 1 whiteboard1.pan tmp10.pan pdilatationreconstruction 8 tmp10.pan whiteboard1.pan tmp11.pan phoughlines 6 70 110 4 tmp11.pan tmp20.pan pdilatation 0 1 tmp20.pan tmp26.pan por tmp25.pan tmp26.pan whiteboard2.pan |  |
| Superimposition of the detected lines on the initial image. |
Detection of the whiteboard borders is done from the selection of the all the quadrangles that satisfied the following constraints:
The selected quadrangle is the one with the best ratio of perimeter to number of points under the boundary (3 pixels thickness).
The selection of the best quadrangle is done by the operator pgetquandragle.cpp, which yields an image with the four corners as four dots.
pand whiteboard2.pan whiteboard1.pan tmp41.pan pgetquadrangle tmp41.pan whiteboard2.pan whiteboard3.pan |  |
| The selected quadrangle. |
Perspective correction needs the calculation of a 3x3 matrix that transforms input image coordinates to corrected image coordinates. The matrix is calculated from the estimated focal length knowing that a whiteboard is a pure rectangle, whereas it appears as quadrangle in the input image.
The correction is done by the operator pquandragle2rectangle.cpp.
pquadrangle2rectangle whiteboard.pan whiteboard3.pan whiteboard4.pan |  |
| The rectified image. |
The illumination correction consists in background subtraction. The background corresponds to low-frequencies whereas the writing corresponds to high-frequencies. So, the background is approximated by a closing with a large structuring element, and then the result is subtracted to the image, and the mean value of the closing is added to yield the corrected image.
This operation is applied only on the lightness channel of the HSL color space (ie, the channel #2 of the HSL image).
prgb2hsl whiteboard4.pan tmp34.pan pgetband 0 tmp34.pan tmp44.pan pgetband 1 tmp34.pan tmp45.pan pgetband 2 tmp34.pan tmp36.pan pdilatation 1 5 tmp36.pan tmp37.pan perosion 1 5 tmp37.pan tmp38.pan pmeanvalue tmp38.pan tmp41.pan psub tmp36.pan tmp38.pan tmp40.pan paddval tmp41.pan tmp40.pan tmp42.pan pimg2imc 4 tmp44.pan tmp45.pan tmp42.pan tmp43.pan phsl2rgb tmp43.pan whiteboard5.pan$ ppan2png whiteboard5.pan result.png |  |
| The corrected image. |
This script needs 2 operators (pgetquadrangle, pquadrangle2rectangle) that are not available in the standard Pandore distribution (see Resources on top of the page).
Code