Warping Methods for Air Writing Character Recognition

Related Work

Following reveals the knowledge obtained from the research sphere related to our Air Writing approach with Leap Motion Device. This article is focusing on Warping Methods for the recognition process.

• Kinect Sensor with DTW and SVM Approach

Chengzhang Qu et al, “Online Kinect Handwritten Digit Recognition Based on Dynamic Time Warping and Support Vector Machine”, Journal of Information & Computational Science, pp 413 - 422, January 2015

Summery of the Approach

First the movement of fingertip is tracked by the proposed tracking algorithm.

Then normalization and smoothing procedures were performed on the 3D trajectory of tracked points, which ensured the continuation of the trajectory and eliminated the jagged and noisy data.

After smoothing, the feature vectors in the form of time series were extracted from the smoothed data as online features, and these features were used to calculate the distance vector by an improved Dynamic Time Warping (DTW) algorithm.

The distance vector could be regarded as an input sample to recognize, and the samples represented by DTW distance vectors were finally trained and tested with binary Support Vector Machine (SVM) classifiers.

It should be noticed that the whole recognition for this natural interaction was a real-time process, so that it had potential capabilities in future application.

Detailed : Pre-processing

(Movement of fingertip tracking using a tracking algorithm is not important to us because it done by the Leap Motion Device itself)

After tracking procedure, they performed a normalization operation to transform all the raw 3D trajectories into a regular size. 

- First, they move the 3D trajectory to the original point(0, 0, 0). [Translate to Center]

- Second, they normalize each position such that it follows a normal Gaussian distribution N(0; 1) over all the 3D points in all recorded data. The normalized results (the blue curve) show raw 3D point which is highly jagged and noisy. 

- The trajectories is still highly jagged and noisy after normalization operation. To address this issue, they employed a standard Kalman filter algorithm to calculate a smoothed trajectory. The red

curve in shows the smoothing results.

Detailed : Feature Extraction

They extract 6 types of features from each point (each time frame) of 3D handwritten trajectory in x, y, z axis, respectively.. Thus, the feature of a trajectory is a temporal series data recording the feature in time order, which is considered to be better presentation for handwritten in space.

Following are the list of extracted features.

1. 3D positions and Derivatives
2. Velocities (the mean value of two adjoining distance between two trajectory points)
3. Acceleration
4. Slop Angle
5. Path Angle (degree between two adjoining connections in 3D trajectory, which means the degree in 3D space.)
6. Log radius of curvature

They also calculate 2D features for a comparison. The 2D features are projection of 3D trajectory in X-Y plane.

Detailed : Recognition

They proposed a joint recognition framework based on DTW and SVM where either same or different length of input vectors can be recognized as well.

DTW - Dynamic Time Warping

It is a well-known technique in time series data processing. DTW finds an optimal alignment

between two given (time-dependent) sequences under following restrictions.

1. Boundary Condition
2. Monotonicity Condition
3. Step size Condition

Intuitively, the sequences are warped in a nonlinear fashion to match each other. It has the advantage on measuring various data, despite of their time contraction and stretch problems.

Basically, the calculation of DTW distance is to find a warping path Q between vector M and N, where Warping path Q satisfies the restrictions mentioned above.

In 3D handwritten interaction, person writing in the air would have its own order or preference. This order would not affect for Monotonicity condition, but for other two. So they have did a slight modification for those DTW restrictions.

Detailed : Training

They replaced the data features with the improved DTW distance vectors, build binary SVM classifier for each class upon them for recognition problem. This got a high accuracy on a very small training size.