\documentclass[10pt,a4paper]{article}
\usepackage{float,url}

% Load code highlighter color scheme
\input{colors}

\title{Modelleren, Simuleren \& Contin\"ue Wiskunde \\
       Assignment 1: Floating point arithmetic}
\author{Tadde\"us Kroes (6054129) \and Sander van Veen (6167969)}

\begin{document}

\maketitle

\section{Representation} % {{{
\label{sec:Representation}

We wrote a small C program to determine the properties of floating point numbers
(float, double and long double) on our working machine\footnote{Machine
info...}. To determine the size of the various data types, we used the
\texttt{sizeof} operator. The range of the mentioned data types can derived from
glibc's constants, like \texttt{FLT\_MAX}. Glibc also defines the machine precision
(epsilon) of each data type. \\
\\
The values we found are summarized in the table below:

\begin{table}[H]
    \begin{tabular}{l|lll}
        Data type            & Bytes & Range & Epsilon \\
        \hline
        \texttt{float}       & 4     & $[1.175494 \cdot 10^{38}, 3.402823 \cdot 10^{38}]$
        & $1.192093 \cdot 10^{7}$ \\
        \texttt{double}      & 8     & $[2.225074 \cdot 10^{308}, 1.797693 \cdot 10^{308}]$
        & $2.220446 \cdot 10^{16}$ \\
        \texttt{long double} & 12    & $[3.362103 \cdot 10^{4932}, 1.189731 \cdot 10^{4932}]$
        & $1.084202 \cdot 10^{19}$ \\
    \end{tabular}
    \caption{Floating point characteristics.}
\end{table}

We will explain the $\epsilon$ we found for the precision of the \texttt{float}
data type. First, we state that epsilon is the smallest representable number
greater than one (thus $a + \epsilon \neq a$, where $|a| \ge 1$). Given the
representation as defined in the lecture slides, we know that the 8-bit exponent
of $1$ is $01111111_2 = 127_{10}$, so $e = 127 - bias = 127 - 127 = 0$. The
mantissa are all zero except for the ``hidden bit'', which is 1. This gives the
exact number $1 \cdot 10^0 = 1.0$. The number closest to one can be made by
making the least significant mantissa `1'. If we apply the given formula, we get
the following decimal value:

$$ (-1)^{sign}(1 + \sum_{i=1}^{23} \ b_{i}2^{-i} )\cdot 2^{(e-127)}
   = 1(1 + 1 \cdot 10^{-22}) \cdot 2^0 =  1.000000119209 = 1 + \epsilon $$

We noticed that the precision of numbers between -1 and 1 is much higher, as we
will show later in this report. We thought that the precision would be the same
as the $\epsilon$ which we calculated above, because the exponent is
$00000000_2$ which gives us $e = 0 - bias = -127$. There is no more hidden bit,
but since $2^{-126} = 2 \cdot 2^{-127}$ the precision should be the same. We
think that the higher precision is due to extra precision in the floating point
registers of our computer. Optimization is possible, because numbers between -1
and 1 are ``denormalized'', and therefore contain redundant representations.

% }}}

\section{Calculation speed} % {{{
\label{sec:Calculation speed}

We created one base source file, the executable benchmark files are generated
using the Makefile (which will substitute the variables). The benchmark can be
started using \texttt{./benchmark.bash}.

\begin{table}[H]
\begin{tabular}{l|ll}
Type                 & Operator & Million ops/sec \\
\hline
\texttt{float}       & ADD      & 311  \\
\texttt{double}      & ADD      & 296  \\
\texttt{long double} & ADD      & 235  \\
\texttt{float}       & DIV      & 213  \\
\texttt{double}      & DIV      & 213  \\
\texttt{long double} & DIV      & 190  \\
\texttt{float}       & MULT     & 9.57 \\
\texttt{double}      & MULT     & 9.58 \\
\texttt{long double} & MULT     & 12.8 \\
\texttt{float}       & SQRT     & 190  \\
\texttt{double}      & SQRT     & 222  \\
\texttt{long double} & SQRT     & 121  \\
\end{tabular}
\caption{Calculation speed of various mathematical operations.}
\end{table}

\noindent \textbf{Observations}
\begin{itemize}
    \item We see that when the data type has a larger storage size, the addition
    operation takes increasingly longer.
    \item Division and multiplication performance are the same for the data
    types \texttt{float} and \texttt{double}. However, division and
    multiplication for the \texttt{long double} data type does take longer to
    execute.
    \item We notice that the square root operation is slower for the
    \texttt{float} than for the \texttt{double} data type. Therefore, we think
    that the \texttt{sqrt} function of glibc is optimised for the
    \texttt{double} data type.
\end{itemize}

% }}}

\section{Summation} % {{{
\label{sec:Summation}

We've calculated $\sum_{i=1}^{N}\frac{1}{i}$ for $N = 10^8$ and $N = 2 \cdot
10^8$ using a forward and backward summation approach, with data types
\texttt{float} and \texttt{double}. The results of this are in the table below.

\begin{table}[H]
\begin{tabular}{l|llll}
Type                   & N              & Forward     & Backward    & Kahan\\
\hline
\texttt{float}         & $10^8$         & $15.403683$ & $18.807919$ & $18.997896$ \\
\texttt{float}         & $2 \cdot 10^8$ & $15.403683$ & $18.807919$ & $19.691044$ \\
\texttt{double}        & $10^8$         & $18.997896$ & $18.997896$ & \\
\texttt{double}        & $2 \cdot 10^8$ & $19.691044$ & $19.691044$ & \\
\end{tabular}
\caption{Results of various summation approaches on floats and doubles.}
\end{table}

\noindent \textbf{Observations}
\begin{itemize}
    \item Since the results for the \texttt{double} data type are equal for both
    the forward and backward summation approach, we can say that these are the
    correct results.
    \item For the \texttt{float} data type, we observe that the backward approach
    yields a higher result than the forward approach. This can be explained as
    follows. When using the forward approach, we start with a small $i$, thus
    with a large $1/i$. This means that the initial value of \texttt{sum} is
    large. The value will keep growing until the significance of $1/i$ is too
    small to add to the result. From this point, no more $1/i$ will be added to
    the result because the significance of the individual numbers is too small.
    However, the sum of the ignored numbers would be a large enough number to
    add to the result. This is why the backward approach yields a higher number:
    the sum of the ignored numbers is computed and later the larger numbers
    are added. The remaining imprecision is probably due to rounding problems and
    the fact that $1/10^8$ and a range of larger numbers are represented as
    zeroes in \texttt{float} representation and therefore not added to the
    result. This problem does not occur when using the \texttt{double} data type
    which has a higher precision, therefore yielding the (correct) higher
    result.
    \item We can see that both approaches yield the same result for $N = 10^8$
    and $N = 2 \cdot 10^8$ when using the \texttt{float} data type. This is due
    to the same problem as described above: all numbers in $[\frac{1}{10^8},
    \frac{1}{2 \cdot 10^8}]$ are also represented as zero and therefore not added
    to the result.
    \item To improve the precision of the \texttt{float} data type summation, we
    implemented the Kahan summation algorithm. This algorithm basically divides the
    summation in a higher- and lower-order part. The lower-order part is used to
    compensate for the error at each summation. See
    \url{http://en.wikipedia.org/wiki/Kahan_summation_algorithm} for more
    information about this algorithm. The results are in the last column as the
    table, we can see that the algorithm yields the correct number. We know this
    because they are equal to the result of the \texttt{double} summations (but
    rounded to the precision of a \texttt{float}, of course).
\end{itemize}

% }}}

\section{Extra precision} % {{{
\label{sec:Extra precision}

Our machine has an Intel Core2 Duo cpu (cpu type is E6750) running at 2.66GHz.
Because Intel added the x87 instruction set (a subset of x86), the FPU
(floating point unit) has a more precise floating point register.

To demonstrate this, we created simple C program, which will approximate the
constant $e$ twice. The first time it will save the final result in a float, the
second time it will store the result in the floating point register. The second
approximation is compared to the first and if the first approximation is large
than the second, the processor has a more precise floating point register. The
simple C program is listed in the appendix \ref{sec:extra_precision.c} and
produces this output:

\begin{verbatim}
$ ./pr
more precision detected!
first: 2.6910297870635986328125000000000000000000000000000000000
last : 2.6910298253667153112189680541632696986198425292968750000
\end{verbatim}

% }}}

\appendix{}

\section{floating\_point.c} % {{{
\label{sec:floating_point.c}

\input{floating_point}

% }}}

\section{speed.c} % {{{
\label{sec:speed.c}

\input{speed}

% }}}

\section{sum.c} % {{{
\label{sec:sum.c}

\input{sum}

% }}}

\section{kahan\_sum.c} % {{{
\label{sec:kahan_sum.c}

\input{kahan_sum}

% }}}

\section{extra\_precision.c} % {{{
\label{sec:extra_precision.c}

\input{extra_precision}

% }}}

\section{Makefile} % {{{
\label{sec:Makefile}

\input{Makefile}

% }}}

\end{document}
