IMAGE Team - The Pantheon project

Tutorial: Scanning Whiteboard Contents



I. Objective

The goal of this application is the digitalization of whiteboard contents capture. This application locates the boundary of the whiteboard, rectifies the geometry distortion and corrects the non-uniform illumination.

Whiteboard capture.Result image.

II. Method

The method is composed of 5 steps:

  1. Detection of the edges.
  2. Detection of the nearly horizontal and the vertical lines.
  3. Detection of the whiteboard border among the previously extracted lines.
  4. Correction of the geometry distortion.
  5. Correction of the illumination.

II.1 Detection of the Edges

The detection is done with the Shen-Castan operator which provides a good detection and a good localization of the edges and allows a binarization without parameter (actually the threshold is set to 2 for all images).

pany2pan whiteboard.png whiteboard.pan
prgb2gray 0.299 0.587 0.114 whiteboard.pan tmp3.pan
pgaussianfiltering 1 tmp3.pan tmp18.pan
pshen 1.3 tmp18.pan tmp6.pan
pbinarization 2 255 tmp6.pan whiteboard1.pan
Edge image

II.2 Detection of the Nearly Horizontal and Vertical Lines

The detection is based on the Hough transform which is used to locate the nearly straight lines among the edges. The detection is done separately for the horizontal (between -20° et 20°) and the vertical lines (between 70° et 110°). For each, an initial closing by reconstruction is used to remove the small lines that are not in the expected direction. Next, the Hough transform is used to detect the longest lines (the 6 longest vertical lines and the 6 longest horizontal lines).
# Horizontal
plinearerosion 0 0 1 whiteboard1.pan tmp7.pan
pdilatationreconstruction 8 tmp7.pan whiteboard1.pan tmp9.pan
phoughlines 6 -20 20 10 tmp9.pan tmp19.pan
pdilatation 0 1 tmp19.pan tmp25.pan
# Vertical
plinearerosion 90 0 1 whiteboard1.pan tmp10.pan
pdilatationreconstruction 8 tmp10.pan whiteboard1.pan tmp11.pan
phoughlines 6 70 110 4 tmp11.pan tmp20.pan
pdilatation 0 1 tmp20.pan tmp26.pan

por tmp25.pan tmp26.pan whiteboard2.pan
Superimposition of the detected lines on the initial image.

II.3 Detection of the Whiteboard Borders

Detection of the whiteboard borders is done from the selection of the all the quadrangles that satisfied the following constraints:

The selected quadrangle is the one with the best ratio of perimeter to number of points under the boundary (3 pixels thickness).

The selection of the best quadrangle is done by the operator pgetquandragle.cpp, which yields an image with the four corners as four dots.

pand whiteboard2.pan whiteboard1.pan tmp41.pan
pgetquadrangle tmp41.pan whiteboard2.pan whiteboard3.pan
The selected quadrangle.

II.4 Correction of the Distortion

Perspective correction needs the calculation of a 3x3 matrix that transforms input image coordinates to corrected image coordinates. The matrix is calculated from the estimated focal length knowing that a whiteboard is a pure rectangle, whereas it appears as quadrangle in the input image.

The correction is done by the operator pquandragle2rectangle.cpp.

pquadrangle2rectangle whiteboard.pan whiteboard3.pan whiteboard4.pan
The rectified image.

II.5 Correction of the Illumination

The illumination correction consists in background subtraction. The background corresponds to low-frequencies whereas the writing corresponds to high-frequencies. So, the background is approximated by a closing with a large structuring element, and then the result is subtracted to the image, and the mean value of the closing is added to yield the corrected image.

This operation is applied only on the lightness channel of the HSL color space (ie, the channel #2 of the HSL image).

prgb2hsl whiteboard4.pan tmp34.pan
pgetband 0 tmp34.pan tmp44.pan
pgetband 1 tmp34.pan tmp45.pan
pgetband 2 tmp34.pan tmp36.pan

pdilatation 1 5 tmp36.pan tmp37.pan
perosion 1 5 tmp37.pan tmp38.pan
pmeanvalue tmp38.pan tmp41.pan
psub tmp36.pan tmp38.pan tmp40.pan
paddval tmp41.pan tmp40.pan tmp42.pan

pimg2imc 4 tmp44.pan tmp45.pan tmp42.pan tmp43.pan
phsl2rgb tmp43.pan whiteboard5.pan$
ppan2png whiteboard5.pan result.png
The corrected image.

III. The Complete Pandore Script (bash)

This script needs 2 operators (pgetquadrangle, pquadrangle2rectangle) that are not available in the standard Pandore distribution (see Resources on top of the page).

The Pantheon project
Image Team GREYC Laboratory
UMR CNRS 6072 - ENSICAEN - University of Caen, France
This page was last modified on 31 October 2013