|
|
@@ -409,47 +409,47 @@ goal is to test the effectiveness of the design and detect its shortcomings.
|
|
|
\section{Detecting gestures from events}
|
|
|
\label{sec:gesture-detection}
|
|
|
|
|
|
- The events that are grouped by areas must be translated to complex gestures
|
|
|
- in some way. Gestures such as a button tap or the dragging of an object
|
|
|
- using one finger are easy to detect by comparing the positions of
|
|
|
- sequential $point\_down$ and $point\_move$ events.
|
|
|
-
|
|
|
- A way to detect more complex gestures is based on a sequence of input
|
|
|
- features is with the use of machine learning methods, such as Hidden Markov
|
|
|
- Models \footnote{A Hidden Markov Model (HMM) is a statistical model without
|
|
|
- a memory, it can be used to detect gestures based on the current input
|
|
|
- state alone.} \cite{conf/gw/RigollKE97}. A sequence of input states can be
|
|
|
- mapped to a feature vector that is recognized as a particular gesture with
|
|
|
- some probability. This type of gesture recognition is often used in video
|
|
|
- processing, where large sets of data have to be processed. Using an
|
|
|
- imperative programming style to recognize each possible sign in sign
|
|
|
- language detection is near impossible, and certainly not desirable.
|
|
|
+ The low-level events that are grouped by an event area must be translated
|
|
|
+ to high-level gestures in some way. Simple gestures, such as a tap or the
|
|
|
+ dragging of an element using one finger, are easy to detect by comparing
|
|
|
+ the positions of sequential $point\_down$ and $point\_move$ events. More
|
|
|
+ complex gestures, like the writing of a character from the alphabet,
|
|
|
+ require more advanced detection algorithms.
|
|
|
+
|
|
|
+ A way to detect complex gestures based on a sequence of input features
|
|
|
+ is with the use of machine learning methods, such as Hidden Markov Models
|
|
|
+ \footnote{A Hidden Markov Model (HMM) is a statistical model without a
|
|
|
+ memory, it can be used to detect gestures based on the current input state
|
|
|
+ alone.} \cite{conf/gw/RigollKE97}. A sequence of input states can be mapped
|
|
|
+ to a feature vector that is recognized as a particular gesture with a
|
|
|
+ certain probability. An advantage of using machine learning with respect to
|
|
|
+ an imperative programming style is that complex gestures can be described
|
|
|
+ without the use of explicit detection logic. For example, the detection of
|
|
|
+ the character `A' being written on the screen is difficult to implement
|
|
|
+ using an imperative programming style, while a trained machine learning
|
|
|
+ system can produce a match with relative ease.
|
|
|
|
|
|
Sequences of events that are triggered by a multi-touch based surfaces are
|
|
|
often of a manageable complexity. An imperative programming style is
|
|
|
- sufficient to detect many common gestures. The imperative programming style
|
|
|
- is also familiar and understandable for a wide range of application
|
|
|
- developers. Therefore, the aim is to use this programming style in the
|
|
|
- architecture implementation that is developed during this project.
|
|
|
+ sufficient to detect many common gestures, like rotation and dragging. The
|
|
|
+ imperative programming style is also familiar and understandable for a wide
|
|
|
+ range of application developers. Therefore, the architecture should support
|
|
|
+ an imperative style of gesture detection.
|
|
|
|
|
|
- However, the architecture should not be limited to multi-touch surfaces
|
|
|
- alone. For example, the architecture should also be fit to be used in an
|
|
|
- application that detects hand gestures from video input.
|
|
|
-
|
|
|
- A problem with the imperative programming style is that the detection of
|
|
|
- different gestures requires different pieces of detection code. If this is
|
|
|
- not managed well, the detection logic is prone to become chaotic and
|
|
|
- over-complex.
|
|
|
+ A problem with the imperative programming style is that the explicit
|
|
|
+ detection of different gestures requires different gesture detection
|
|
|
+ components. If these components is not managed well, the detection logic is
|
|
|
+ prone to become chaotic and over-complex.
|
|
|
|
|
|
To manage complexity and support multiple methods of gesture detection, the
|
|
|
architecture has adopted the tracker-based design as described by
|
|
|
\cite{win7touch}. Different detection components are wrapped in separate
|
|
|
- gesture tracking units, or \emph{gesture trackers} The input of a gesture
|
|
|
- tracker is provided by an area in the form of events. When a gesture
|
|
|
+ gesture tracking units, or \emph{gesture trackers}. The input of a gesture
|
|
|
+ tracker is provided by an event area in the form of events. When a gesture
|
|
|
tracker detects a gesture, this gesture is triggered in the corresponding
|
|
|
- area. The area then calls the callbacks which are bound to the gesture
|
|
|
- type by the application. Figure \ref{fig:trackerdiagram} shows the position
|
|
|
- of gesture trackers in the architecture.
|
|
|
+ event area. The event area then calls the callbacks which are bound to the
|
|
|
+ gesture type by the application. Figure \ref{fig:trackerdiagram} shows the
|
|
|
+ position of gesture trackers in the architecture.
|
|
|
|
|
|
\trackerdiagram
|
|
|
|