|
|
@@ -1,4 +1,4 @@
|
|
|
-\documentclass[10pt,a4paper]{article}
|
|
|
+\documentclass[a4paper]{article}
|
|
|
|
|
|
\usepackage[english]{babel}
|
|
|
\usepackage[utf8]{inputenc}
|
|
|
@@ -7,10 +7,6 @@
|
|
|
% Link colors
|
|
|
\hypersetup{colorlinks=true,linkcolor=black,urlcolor=blue}
|
|
|
|
|
|
-% Paragraph indentation
|
|
|
-\setlength{\parindent}{0pt}
|
|
|
-\setlength{\parskip}{1.5ex plus 0.5ex minus 0.2ex}
|
|
|
-
|
|
|
\title{Bachelor thesis\\Universal multi-touch event mechanism}
|
|
|
\author{\begin{tabular}{ll}
|
|
|
Name: & Taddeüs Kroes\\
|
|
|
@@ -23,10 +19,17 @@ Supervisor: & Dr. R.G. Belleman (UvA)\\
|
|
|
|
|
|
\begin{document}
|
|
|
|
|
|
+% Title page
|
|
|
\maketitle
|
|
|
\abstract{
|
|
|
% TODO
|
|
|
}
|
|
|
+
|
|
|
+% Set paragraph indentation
|
|
|
+\parindent 0pt
|
|
|
+\parskip 1.5ex plus 0.5ex minus 0.2ex
|
|
|
+
|
|
|
+% Table of contant on separate page
|
|
|
\pagebreak
|
|
|
\tableofcontents
|
|
|
\pagebreak
|
|
|
@@ -46,7 +49,7 @@ these frameworks have no access to their multi-touch events.
|
|
|
This problem was observed during an attempt to create a multi-touch
|
|
|
``interactor'' class for the Visualization Toolkit
|
|
|
(VTK\footnote{\url{http://www.vtk.org/}}). Because VTK provides the application
|
|
|
-framework here, it is undesirable to use an entire framework like QT
|
|
|
+framework here, it is undesirable to use an entire framework like Qt
|
|
|
simultaneously only for its multi-touch support.
|
|
|
|
|
|
% Ruw doel
|
|
|
@@ -54,18 +57,20 @@ The goal of this project is to define a universal multi-touch event triggering
|
|
|
mechanism. To test the definition, a reference implementation is written in
|
|
|
Python.
|
|
|
|
|
|
-% Setting/afbakening
|
|
|
+% Setting
|
|
|
To tests multi-touch interaction properly, a multi-touch device is required.
|
|
|
The University of Amsterdam (UvA) has provided access to a multi-touch table
|
|
|
from PQlabs. The table uses the TUIO
|
|
|
protocol\footnote{\url{http://www.tuio.org/}} to communicate touch events.
|
|
|
|
|
|
+% Afbakening
|
|
|
+% TODO: moet dit omlaag naar 'Definition of the problem'?
|
|
|
The scope of this thesis includes the design of an multi-touch triggering
|
|
|
mechanism, a reference implementation of this design, and its integration into
|
|
|
-a VTK interactor.
|
|
|
-To be successful, the design should allow for extensions to be added to any
|
|
|
-implementation. The reference implementation is a Proof of Concept with some
|
|
|
-simple touch gestures that are used by the VTK interactor.
|
|
|
+a VTK interactor. To be successful, the design should allow for extensions to
|
|
|
+be added to any implementation. The reference implementation is a Proof of
|
|
|
+Concept that translates TUIO events to some simple touch gestures that are used
|
|
|
+by a VTK interactor.
|
|
|
|
|
|
\subsection{Structure of this document}
|
|
|
|
|
|
@@ -90,16 +95,11 @@ To design such a mechanism properly, the following questions are relevant:
|
|
|
\item Can events be shared with multiple processes at the same time? For
|
|
|
example, a network implementation could run as a service instead of
|
|
|
within a single application, triggering events in any application that
|
|
|
- needs it.
|
|
|
+ needs them.
|
|
|
\item Is performance an issue? For example, an event loop with rotation
|
|
|
- detection could swallow up more processing resources than desirable.
|
|
|
+ detection could swallow up more processing resources than desired.
|
|
|
\end{itemize}
|
|
|
|
|
|
-\section{Hypothesis}
|
|
|
-
|
|
|
-To accomplish extendability, an object-oriented approach is an obvious choice.
|
|
|
-% TODO
|
|
|
-
|
|
|
\section{Related work}
|
|
|
|
|
|
% TODO
|
|
|
@@ -112,25 +112,72 @@ To accomplish extendability, an object-oriented approach is an obvious choice.
|
|
|
|
|
|
\subsubsection{The TUIO protocol}
|
|
|
|
|
|
- % TODO
|
|
|
- % Vertellen dat het met UDP werkt
|
|
|
- % wat komt er dan binnen?
|
|
|
- % wat doet de implementatie ermee?
|
|
|
+ The TUIO protocol\footnote{\url{http://tuio.org/?specification}}
|
|
|
+ defines a way to geometrically describe tangible objects, such as
|
|
|
+ fingers or fiducials on a multi-touch table. The table used for this
|
|
|
+ thesis uses the protocol in its driver. Object information is sent to
|
|
|
+ the TUIO UDP port (3333 by default).
|
|
|
+
|
|
|
+ For efficiency reasons, the TUIO protocol is encoded using the Open
|
|
|
+ Sound Control
|
|
|
+ (OSC)\footnote{\url{http://opensoundcontrol.org/specification}} format.
|
|
|
+ An OSC server/client implementation is available for Python:
|
|
|
+ pyOSC\footnote{\url{https://trac.v2.nl/wiki/pyOSC}}.
|
|
|
+
|
|
|
+ A Python implementation of the TUIO protocol also exists:
|
|
|
+ pyTUIO\footnote{\url{http://code.google.com/p/pytuio/}}. However, the
|
|
|
+ execution of an example script yields an error regarding Python's
|
|
|
+ built-in \texttt{socket} library. Therefore, the reference
|
|
|
+ implementation uses the pyOSC package to receive TUIO messages.
|
|
|
+
|
|
|
+ The two most important message types of the protocol are ALIVE and SET
|
|
|
+ messages. An ALIVE message contains the list of session id's that are
|
|
|
+ currently ``active'', which in the case of multi-touch a table means
|
|
|
+ that they are touching the screen. A SET message provides geometric
|
|
|
+ information of a session id, such as position, velocity and acceleration.
|
|
|
+
|
|
|
+ Each session id represents an object. The only type of objects on the
|
|
|
+ multi-touch table are what the TUIO protocol calls ``2DCur'', which is
|
|
|
+ a (x, y) position on the screen.
|
|
|
+
|
|
|
+ ALIVE messages can be used to determine when an object touches and
|
|
|
+ releases the screen. For example, if a session id was in the previous
|
|
|
+ message but not in the current, The object it represents has been
|
|
|
+ lifted from the screen.
|
|
|
+
|
|
|
+ SET provide information about movement. In the case of simple (x, y)
|
|
|
+ positions, only the movement vector of the position itself can be
|
|
|
+ calculated. For more complex objects such as fiducials, arguments like
|
|
|
+ rotational position is also included.
|
|
|
+
|
|
|
+ TUIO coordinates range from $0.0$ to $1.0$, with $(0.0, 0.0)$ being the
|
|
|
+ left top corner of the screen and $(1.0, 1.0)$ the right bottom corner.
|
|
|
+ To focus events within a window, a translation to window coordinates is
|
|
|
+ required in the client application, as stated by the
|
|
|
+ specification:
|
|
|
+ \begin{citation}
|
|
|
+ In order to compute the X and Y coordinates for the 2D profiles a
|
|
|
+ TUIO tracker implementation needs to divide these values by the
|
|
|
+ actual sensor dimension, while a TUIO client implementation
|
|
|
+ consequently can scale these values back to the actual screen
|
|
|
+ dimension.
|
|
|
+ \end{citation}
|
|
|
|
|
|
\subsection{Experiments}
|
|
|
|
|
|
% testimplementatie met taps, rotatie en pinch. Hieruit bleek:
|
|
|
- % - dat het gebruik van threads soms onvermijdelijk is
|
|
|
% - dat er verschillende manieren zijn om bijv. "rotatie" te
|
|
|
% detecteren, (en dat daartussen onderscheid moet kunnen worden
|
|
|
% gemaakt)
|
|
|
% - dat detectie van verschillende soorten gestures moet kunnen
|
|
|
% worden gescheiden, anders wordt het een chaos.
|
|
|
- % -
|
|
|
+ % - Er zijn een aantal keuzes gemaakt bij het ontwerpen van de gestures,
|
|
|
+ % bijv dat rotatie ALLE vingers gebruikt voor het centroid. Het is
|
|
|
+ % wellicht in een ander programma nodig om maar 1 hand te gebruiken, en
|
|
|
+ % dus punten dicht bij elkaar te kiezen (oplossing: windows).
|
|
|
|
|
|
- % Tekenprogramma dat de huidige state tekent en waarmee
|
|
|
- % transformatie kan worden getest
|
|
|
- % Link naar appendix "supported events"
|
|
|
+ % Tekenprogramma dat huidige points + centroid tekent en waarmee
|
|
|
+ % transformatie kan worden getest Link naar appendix "supported events"
|
|
|
|
|
|
% Proof of Concept: VTK interactor
|
|
|
|
|
|
@@ -140,27 +187,28 @@ To accomplish extendability, an object-oriented approach is an obvious choice.
|
|
|
|
|
|
\section{Server structure}
|
|
|
|
|
|
-\subsection{Input server}
|
|
|
+% TODO: link naar appendix met schema
|
|
|
|
|
|
-% TODO
|
|
|
-% vertaling driver naar point down, move, up
|
|
|
-% TUIO in reference implementation
|
|
|
+ \subsection{Input server}
|
|
|
|
|
|
-\subsection{Gesture server}
|
|
|
+ % TODO
|
|
|
+ % vertaling driver naar point down, move, up
|
|
|
+ % TUIO in reference implementation
|
|
|
|
|
|
- \subsubsection{Windows}
|
|
|
+ \subsection{Gesture server}
|
|
|
|
|
|
- % TODO
|
|
|
- % toewijzen even aan deel v/h scherm:
|
|
|
- % TUIO coördinaten zijn over het hele scherm en van 0.0 tot 1.0, dus moeten
|
|
|
- % worden vertaald naar pixelcoördinaten binnen een ``window''
|
|
|
+ \subsubsection{Windows}
|
|
|
|
|
|
- \subsubsection{Trackers}
|
|
|
+ % TODO
|
|
|
+ % toewijzen even aan deel v/h scherm:
|
|
|
+ % TUIO coördinaten zijn over het hele scherm en van 0.0 tot 1.0, dus moeten
|
|
|
+ % worden vertaald naar pixelcoördinaten binnen een ``window''
|
|
|
|
|
|
- % TODO
|
|
|
- % event binding/triggering
|
|
|
- % extendability
|
|
|
+ \subsubsection{Trackers}
|
|
|
|
|
|
+ % TODO
|
|
|
+ % event binding/triggering
|
|
|
+ % extendability
|
|
|
|
|
|
\section{Reference implementation}
|
|
|
|
|
|
@@ -175,9 +223,16 @@ To accomplish extendability, an object-oriented approach is an obvious choice.
|
|
|
% Trackers zijn een effectieve manier om gebaren te detecteren
|
|
|
% Trackers zijn uitbreidbaar door object-orientatie
|
|
|
|
|
|
+\section{Suggestions for future work}
|
|
|
+
|
|
|
+% TODO: Network protocol (ZeroMQ)
|
|
|
+
|
|
|
\section{References}
|
|
|
|
|
|
% TODO: BibTex
|
|
|
+\cite{TUIOspecification}
|
|
|
+\bibliography{report}{}
|
|
|
+\bibliographystyle{plain}
|
|
|
|
|
|
\appendix
|
|
|
|