|
|
@@ -1,5 +1,7 @@
|
|
|
\documentclass[a4paper]{article}
|
|
|
|
|
|
+\usepackage{hyperref}
|
|
|
+
|
|
|
\title{Using local binary patterns to read license plates in photographs}
|
|
|
\date{November 17th, 2011}
|
|
|
|
|
|
@@ -22,30 +24,31 @@ Fabi\'en Tesselaar
|
|
|
|
|
|
\section{Problem description}
|
|
|
|
|
|
-license plates are used for uniquely identifying motorized vehicles and are
|
|
|
-made to be read by humans from great distances and in all kinds of weather
|
|
|
+License plates are used for uniquely identifying motorized vehicles and are
|
|
|
+made to be read by humans from great distances and in all kinds of weather
|
|
|
conditions.
|
|
|
|
|
|
Reading license plates with a computer is much more difficult. Our dataset
|
|
|
-contains photographs from license plates from all sorts of angles and distance.
|
|
|
-Meaning that not only do we have to implement a method to read the actual
|
|
|
-characters, but also have to determine the location of the license plate and its
|
|
|
-transformation due to different angles.
|
|
|
-
|
|
|
-We will focus our research in reading the transformed characters of the
|
|
|
-lisence plate, on which we know where the letters are located. This is because
|
|
|
-Microsoft recently published a new and effective method to find the location of
|
|
|
+contains photographs of license plates from various angles and distances. This
|
|
|
+means that not only do we have to implement a method to read the actual
|
|
|
+characters, but also have to determine the location of the license plate and
|
|
|
+its transformation due to different angles.
|
|
|
+
|
|
|
+We will focus our research on reading the transformed characters on the
|
|
|
+license plate, of which we know where the letters are located. This is because
|
|
|
+Microsoft recently published a new and effective method to find the location of
|
|
|
text in an image.
|
|
|
|
|
|
In short our program must be able to do the following:
|
|
|
|
|
|
\begin{enumerate}
|
|
|
-\item Use perspective transformation to obtain an upfront view of license plate.
|
|
|
-\item Reduce noise where possible.
|
|
|
-\item Extract each character using the location points in the info file.
|
|
|
-\item Transform character to a normal form.
|
|
|
-\item Create a local binary pattern histogram vector.
|
|
|
-\item Match the found vector with the learning set.
|
|
|
+ \item Use perspective transformation to obtain an upfront view of license
|
|
|
+ plate.
|
|
|
+ \item Reduce noise where possible.
|
|
|
+ \item Extract each character using the location points in the info file.
|
|
|
+ \item Transform character to a normal form.
|
|
|
+ \item Create a local binary pattern histogram vector.
|
|
|
+ \item Match the found vector with a learning set.
|
|
|
\end{enumerate}
|
|
|
|
|
|
\section{Solution}
|
|
|
@@ -55,66 +58,70 @@ come in a few steps as well.
|
|
|
|
|
|
\subsection{Transformation}
|
|
|
|
|
|
-A simple perspective transformation will be sufficient to transform and resize
|
|
|
-the plate to a normalized format. Corners of license plates can be found in the
|
|
|
-data files.
|
|
|
+A simple perspective transformation will be sufficient to transform and resize
|
|
|
+the plate to a normalized format. The corner positions of license plates in the
|
|
|
+dataset are supplied together with the dataset.
|
|
|
|
|
|
\subsection{Reducing noise}
|
|
|
|
|
|
Small amounts of noise will probably be suppressed by usage of a Gaussian
|
|
|
filter. A real problem occurs in very dirty license plates, where branches and
|
|
|
dirt over a letter could radically change the local binary pattern. A question
|
|
|
-we can ask ourselves here, is whether we want to concentrate ourselves on
|
|
|
-these exceptional cases. By law, license plates have to be readable. Therefore,
|
|
|
-we will first direct our attention at getting a higher score in the 'regular'
|
|
|
-test set before addressing these cases. Looking at how LBP work, there is a good
|
|
|
-change that our features are, to a certain degree, indifferent to noise on the
|
|
|
-plates.
|
|
|
+we can ask ourselves here, is whether we want to concentrate ourselves on these
|
|
|
+exceptional cases. By law, license plates have to be readable. Therefore, we
|
|
|
+will first direct our attention at getting a higher score in the 'regular' test
|
|
|
+set before addressing these cases. Considered the fact that the LBP algorithm
|
|
|
+divides a letter into a lot of cells, there is a good change that a great
|
|
|
+number of cells will still match the learning set, and thus still return the
|
|
|
+correct character as a best match. Therefore, we expect the algorithm to be
|
|
|
+very robust when dealing with noisy images.
|
|
|
|
|
|
\subsection{Extracting a letter}
|
|
|
|
|
|
-Because we are already given the locations of the characters, we only need to
|
|
|
-transform those locations using the same perspective transform used to to
|
|
|
+Because we are already given the locations of the characters, we only need to
|
|
|
+transform those locations using the same perspective transformation used to
|
|
|
create a front facing license plate.
|
|
|
|
|
|
\begin{enumerate}
|
|
|
-\item Crop the image in such a way that the character precisely fits the image.
|
|
|
-\item Scale the image to a standard height.
|
|
|
-\item Extend the image on either the left or right side to a certain width.
|
|
|
+ \item Crop the image in such a way that the character precisely fits the
|
|
|
+ image.
|
|
|
+ \item Scale the image to a standard height.
|
|
|
+ \item Extend the image on either the left or right side to a certain width.
|
|
|
\end{enumerate}
|
|
|
|
|
|
-The resulting image will always have the same size, the character contained will
|
|
|
-always be of the same height, and the character will alway be positioned at
|
|
|
-either the left of right side of the image.
|
|
|
+The resulting image will always have the same size, the character contained
|
|
|
+will always be of the same height, and the character will alway be positioned
|
|
|
+at either the left of right side of the image.
|
|
|
|
|
|
\subsection{Local binary patterns}
|
|
|
|
|
|
-Once we have separate digits and characters, we intend to use Local Binary
|
|
|
-Patterns to determine what character or digit we are dealing with. Local Binary
|
|
|
-Patters are a way to classify a texture, because it can create a histogram which
|
|
|
-describes the distribution of line directions in the image. Since letters on a
|
|
|
-license plate are mainly build up of straight lines and simple curves, it should
|
|
|
-theoretically be possible to identify these using Local Binary Patterns.
|
|
|
+Once we have separate digits and characters, we intend to use Local Binary
|
|
|
+Patterns to determine what character or digit we are dealing with. Local Binary
|
|
|
+Patters are a way to classify a texture, because it can create a histogram
|
|
|
+which describes the distribution of line directions in the image. Since letters
|
|
|
+on a license plate are mainly build up of straight lines and simple curves, it
|
|
|
+should theoretically be possible to identify these using Local Binary Patterns.
|
|
|
|
|
|
-This will actually be the first thing we implement, since it is not known if it
|
|
|
-will give the desired results. Our first goal is therefore a proof of concept
|
|
|
-that using LBP's is a good way to determine which character we are dealing with.
|
|
|
+This will actually be the first thing we implement, since it is not known if it
|
|
|
+will give the desired results. Our first goal is therefore a proof of concept
|
|
|
+that using LBP's is a good way to determine which character we are dealing
|
|
|
+with.
|
|
|
|
|
|
-Important to note is that by now, we have transformed this letter to a standard
|
|
|
-size, which eliminates the need to normalize the histograms generated by the
|
|
|
+Important to note is that by now, we have transformed this letter to a standard
|
|
|
+size, which eliminates the need to normalize the histograms generated by the
|
|
|
algorithm.
|
|
|
|
|
|
-Once we have a Local Binary Pattern of the character, we use a Support Vector
|
|
|
-Machine to determine what letter we are dealing with. For this, the feature
|
|
|
-vector of the image will be a concatenation of the histograms of the cells in
|
|
|
+Once we have a Local Binary Pattern of the character, we use a Support Vector
|
|
|
+Machine to determine what letter we are dealing with. For this, the feature
|
|
|
+vector of the image will be a concatenation of the histograms of the cells in
|
|
|
the image.
|
|
|
|
|
|
\subsection{Matching the database}
|
|
|
|
|
|
-In order to determine what character we are dealing with, we use a SVM, as said
|
|
|
-before. To prevent us from having to teach this SVM each time we start the
|
|
|
-program, we are going to save the SVM to a pickle object, which packs an object
|
|
|
-in Python to a certain data format, so it can be unpacked somewhere else, or, in
|
|
|
-our case, when executing the program to identify a character.
|
|
|
+In order to recognize what character we are dealing with, we use a Support
|
|
|
+Vector Machine. The SVM can be trained with a subsection of the given dataset
|
|
|
+called the ''Learning set''. Once trained, the entire classifier can be saved
|
|
|
+as a Pickle object\footnote{See
|
|
|
+\url{http://docs.python.org/library/pickle.html}} for later usage.
|
|
|
|
|
|
\end{document}
|