licenseplates/docs/plan.tex

\documentclass[a4paper]{article}

\title{Using local binary patterns to read license plates in photographs}
\date{November 17th, 2011}

% Paragraph indentation
\setlength{\parindent}{0pt}
\setlength{\parskip}{1ex plus 0.5ex minus 0.2ex}

\begin{document}
\maketitle

\section*{Project members}
Gijs van der Voort\\
Richard Torenvliet\\
Jayke Meijer\\
Tadde\"us Kroes\\
Fabi\'en Tesselaar

\tableofcontents
\setcounter{secnumdepth}{1}

\section{Problem description}

license plates are used for uniquely identifying motorized vehicles and are 
made to be read by humans from great distances and in all kinds of weather 
conditions.

Reading license plates with a computer is much more difficult. Our dataset
contains photographs from license plates from all sorts of angles and distance. 
Meaning that not only do we have to implement a method to read the actual 
characters, but also have to determine the location of the license plate and its
transformation due to different angles.

We will focus our research in reading the transformed characters of the 
lisence plate, on which we know where the letters are located. This is because
Microsoft recently published a new and effective method to find the location of 
text in an image.

In short our program must be able to do the following:

\begin{enumerate}
\item Use perspective transformation to obtain an upfront view of license plate.
\item Reduce noise where possible.
\item Extract each character using the location points in the info file.
\item Transform character to a normal form. 
\item Create a local binary pattern histogram vector.
\item Match the found vector with the learning set.
\end{enumerate}

\section{Solution}

Now that we know the problem we can start with stating our solution. This will
come in a few steps as well.

\subsection{Transformation}

A simple perspective transformation will be sufficient to transform and resize 
the plate to a normalized format. Corners of license plates can be found in the
data files.

\subsection{Reducing noise}

Small amounts of noise will probably be suppressed by usage of a Gaussian
filter. A real problem occurs in very dirty license plates, where branches and
dirt over a letter could radically change the local binary pattern. A question
we can ask ourselves here, is whether we want to concentrate ourselves on
these exceptional cases. By law, license plates have to be readable. Therefore, 
we will first direct our attention at getting a higher score in the 'regular' 
test set before addressing these cases. Looking at how LBP work, there is a good 
change that our features are, to a certain degree, indifferent to noise on the 
plates.

\subsection{Extracting a letter}

Because we are already given the locations of the characters, we only need to 
transform those locations using the same perspective transform used to to
create a front facing license plate.

\begin{enumerate}
\item Crop the image in such a way that the character precisely fits the image.
\item Scale the image to a standard height.
\item Extend the image on either the left or right side to a certain width. 
\end{enumerate}

The resulting image will always have the same size, the character contained will
always be of the same height, and the character will alway be positioned at 
either the left of right side of the image.

\subsection{Local binary patterns}

Once we have separate digits and characters, we intend to use Local Binary 
Patterns to determine what character or digit we are dealing with. Local Binary 
Patters are a way to classify a texture, because it can create a histogram which
describes the distribution of line directions in the image. Since letters on a 
license plate are mainly build up of straight lines and simple curves, it should 
theoretically be possible to identify these using Local Binary Patterns.

This will actually be the first thing we implement, since it is not known if it 
will give the desired results. Our first goal is therefore a proof of concept 
that using LBP's is a good way to determine which character we are dealing with.

Important to note is that by now, we have transformed this letter to a standard 
size, which eliminates the need to normalize the histograms generated by the 
algorithm.

Once we have a Local Binary Pattern of the character, we use a Support Vector 
Machine to determine what letter we are dealing with. For this, the feature 
vector of the image will be a concatenation of the histograms of the cells in 
the image.

\subsection{Matching the database}

In order to determine what character we are dealing with, we use a SVM, as said 
before. To prevent us from having to teach this SVM each time we start the 
program, we are going to save the SVM to a pickle object, which packs an object 
in Python to a certain data format, so it can be unpacked somewhere else, or, in 
our case, when executing the program to identify a character.

\end{document}