Commit cf82d409 authored by Taddeüs Kroes's avatar Taddeüs Kroes

Wrote 'Related work' chapter of report.

parent ed63f765
......@@ -28,11 +28,11 @@
howpublished = {\url{http://www.codeproject.com/Articles/56385/Gesture-Recognition-for-Touch-Devices-Touch-Gestur}}
}
@misc{processingMT,
@unpublished{processingMT,
author = {David Bouchard},
title = {A basic multi-touch gesture demo for Android},
year = {2011},
howpublished = {\url{https://forum.processing.org/topic/a-basic-multi-touch-gesture-demo-for-android}}
note = {\url{https://forum.processing.org/topic/a-basic-multi-touch-gesture-demo-for-android}}
}
@incollection{GART,
......
......@@ -8,7 +8,7 @@
\hypersetup{colorlinks=true,linkcolor=black,urlcolor=blue,citecolor=DarkGreen}
% Title Page
\title{Universal multi-touch event mechanism}
\title{A universal detection mechanism for multi-touch gestures}
\author{Taddeüs Kroes}
\supervisors{Dr. Robert G. Belleman (UvA)}
\signedby{Dr. Robert G. Belleman (UvA)}
......@@ -65,6 +65,7 @@ events.
% Deelvragen
To design such a mechanism properly, the following questions are relevant:
\begin{itemize}
\item What are the requirements of the mechanism to be universal?
\item What is the input of the mechanism? Different touch drivers have
different API's. To be able to support different drivers (which is
highly desirable), there should probably be a translation from the
......@@ -94,20 +95,44 @@ events.
\chapter{Related work}
\section{GART}
\section{Gesture and Activity Recognition Toolkit}
% TODO
\cite{GART}
The Gesture and Activity Recognition Toolkit (GART) \cite{GART} is a
toolkit for the development of gesture-based applications. The toolkit
states that the best way to classify gestures is to use machine learning.
The programmer trains a program to recognize using the machine learning
library from the toolkit. The toolkit contains a callback-mechanism that
the programmer uses to execute custom code when a gesture is recognized.
\section{Processing implementation of simple gestures in Android}
Though multi-touch input is not directly supported by the toolkit, the
level of abstraction does allow for it to be implemented in the form of a
``touch'' sensor.
% TODO
\cite{processingMT}
The reason to use machine learning is the statement that gesture detection
``is likely to become increasingly complex and unmanageable'' when using a
set of predefined rules to detect whether some sensor input can be seen as
a specific gesture. This statement is not necessarily true. If the
programmer is given a way to separate the detection of different types of
gestures and flexibility in rule definitions, over-complexity can be
avoided.
% oplossing: trackers. bijv. TapTracker, TransformationTracker gescheiden
\section{Gesture recognition software for Windows 7}
% TODO
\cite{win7touch}
The online article \cite{win7touch} presents a Windows 7 application,
written in Microsofts .NET. The application shows detected gestures in a
canvas. Gesture trackers keep track of stylus locations to detect specific
gestures. The event types required to track a touch stylus are ``stylus
down'', ``stylus move'' and ``stylus up'' events. A
\texttt{GestureTrackerManager} object dispatches these events to gesture
trackers. The application supports a limited number of pre-defined
gestures.
An important observation in this application is that different gestures are
detected by different gesture trackers, thus separating gesture detection
code into maintainable parts.
\section{The TUIO protocol}
......@@ -147,6 +172,10 @@ events.
calculated. For more complex objects such as fiducials, arguments like
rotational position is also included.
ALIVE and SET messages can be combined to create ``point down'', ``point
move'' and ``point up'' events (as used by the \cite[.NET
application]{win7touch}).
TUIO coordinates range from $0.0$ to $1.0$, with $(0.0, 0.0)$ being the
left top corner of the screen and $(1.0, 1.0)$ the right bottom corner. To
focus events within a window, a translation to window coordinates is
......@@ -159,6 +188,25 @@ events.
scale these values back to the actual screen dimension.
\end{quote}
In other words, the design of the gesture detection mechanism should
incorporate a translation from driver-specific coordinates to pixel
coordinates.
\section{Processing implementation of simple gestures in Android}
An implementation of a detection mechanism for some simple multi-touch
gestures (tap, double tap, rotation, pinch and drag) using
Processing\footnote{Processing is a Java-based development environment with
an export possibility for Android. See also \url{http://processing.org/.}}
can be found found in a forum on the Processing website
\cite{processingMT}. The implementation is fairly simple, but it yields
some very appealing results. The detection logic of all gestures is
combined in a single class. This does not allow for extendability, because
the complexity of this class would increase to an undesirable level (as
predicted by the GART article \cite{GART}). However, the detection logic
itself is partially re-used in the reference implementation of the
universal gesture detection mechanism.
% TODO
\chapter{Experiments}
......@@ -185,7 +233,13 @@ events.
\chapter{Design}
% TODO: link naar appendix met schema
\section{Requirements}
% TODO
% ondersteunen van meerdere drivers
% gesture detectie koppelen aan bepaald gedeelte van het scherm
% scheiden van detectiecode voor verschillende gesture types
% eventueel te gebruiken in meerdere talen
\section{Input server}
......@@ -195,18 +249,24 @@ events.
\section{Gesture server}
\subsection{Windows}
% TODO
% vertaling naar pixelcoordinaten
% toewijzing aan windows
% TODO
% toewijzen even aan deel v/h scherm:
% TUIO coördinaten zijn over het hele scherm en van 0.0 tot 1.0, dus moeten
% worden vertaald naar pixelcoördinaten binnen een ``window''
\section{Windows}
\subsection{Trackers}
% TODO
% toewijzen even aan deel v/h scherm:
% TUIO coördinaten zijn over het hele scherm en van 0.0 tot 1.0, dus moeten
% worden vertaald naar pixelcoördinaten binnen een ``window''
% TODO
% event binding/triggering
% extendability
\section{Trackers}
% TODO
% event binding/triggering
% extendability
% TODO: link naar appendix met schema
\chapter{Reference implementation}
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment