licenseplates/docs/plan.tex

\documentclass[a4paper]{article}

\usepackage{hyperref}

\title{Using local binary patterns to read license plates in photographs}
\date{November 17th, 2011}

% Paragraph indentation
\setlength{\parindent}{0pt}
\setlength{\parskip}{1ex plus 0.5ex minus 0.2ex}

\begin{document}
\maketitle

\section*{Project members}
Gijs van der Voort\\
Richard Torenvliet\\
Jayke Meijer\\
Tadde\"us Kroes\\
Fabi\'en Tesselaar

\tableofcontents
\setcounter{secnumdepth}{1}

\section{Problem description}

License plates are used for uniquely identifying motorized vehicles and are
made to be read by humans from great distances and in all kinds of weather
conditions.

Reading license plates with a computer is much more difficult. Our dataset
contains photographs of license plates from various angles and distances. This
means that not only do we have to implement a method to read the actual
characters, but also have to determine the location of the license plate and
its transformation due to different angles.

We will focus our research on reading the transformed characters on the
license plate, of which we know where the letters are located. This is because
Microsoft recently published a new and effective method to find the location of
text in an image.

In short our program must be able to do the following:

\begin{enumerate}
    \item Use perspective transformation to obtain an upfront view of license
          plate.
    \item Reduce noise where possible.
    \item Extract each character using the location points in the info file.
    \item Transform character to a normal form.
    \item Create a local binary pattern histogram vector.
    \item Match the found vector with a learning set.
\end{enumerate}

\section{Solution}

Now that we know the problem we can start with stating our solution. This will
come in a few steps as well.

\subsection{Transformation}

A simple perspective transformation will be sufficient to transform and resize
the plate to a normalized format. The corner positions of license plates in the
dataset are supplied together with the dataset.

\subsection{Reducing noise}

Small amounts of noise will probably be suppressed by usage of a Gaussian
filter. A real problem occurs in very dirty license plates, where branches and
dirt over a letter could radically change the local binary pattern. A question
we can ask ourselves here, is whether we want to concentrate ourselves on these
exceptional cases. By law, license plates have to be readable. Therefore, we
will first direct our attention at getting a higher score in the 'regular' test
set before addressing these cases. Considered the fact that the LBP algorithm
divides a letter into a lot of cells, there is a good change that a great
number of cells will still match the learning set, and thus still return the
correct character as a best match. Therefore, we expect the algorithm to be
very robust when dealing with noisy images.

\subsection{Extracting a letter}

Because we are already given the locations of the characters, we only need to
transform those locations using the same perspective transformation used to
create a front facing license plate.

\begin{enumerate}
    \item Crop the image in such a way that the character precisely fits the
          image.
    \item Scale the image to a standard height.
    \item Extend the image on either the left or right side to a certain width.
\end{enumerate}

The resulting image will always have the same size, the character contained
will always be of the same height, and the character will alway be positioned
at either the left of right side of the image.

\subsection{Local binary patterns}

Once we have separate digits and characters, we intend to use Local Binary
Patterns to determine what character or digit we are dealing with. Local Binary
Patters are a way to classify a texture, because it can create a histogram
which describes the distribution of line directions in the image. Since letters
on a license plate are mainly build up of straight lines and simple curves, it
should theoretically be possible to identify these using Local Binary Patterns.

This will actually be the first thing we implement, since it is not known if it
will give the desired results. Our first goal is therefore a proof of concept
that using LBP's is a good way to determine which character we are dealing
with.

Important to note is that by now, we have transformed this letter to a standard
size, which eliminates the need to normalize the histograms generated by the
algorithm.

Once we have a Local Binary Pattern of the character, we use a Support Vector
Machine to determine what letter we are dealing with. For this, the feature
vector of the image will be a concatenation of the histograms of the cells in
the image.

\subsection{Matching the database}

In order to recognize what character we are dealing with, we use a Support
Vector Machine. The SVM can be trained with a subsection of the given dataset
called the ''Learning set''. Once trained, the entire classifier can be saved
as a Pickle object\footnote{See
\url{http://docs.python.org/library/pickle.html}} for later usage.

\end{document}