Air Writing with Neural Gas Network Approach

Related Work

Following reveals the knowledge obtained from the research sphere related to our Air Writing approach with Leap Motion Device. This article is focusing on Neural Gas Methods for the recognition process.

Kinect Sensor with Neural Gas Network Approach 

MRA Heidari et al, “Writing in the Air Using Kinect and Growing Neural Gas Network”, Jurnal Teknologi Universiti Teknologi Malaysia, vol. 72, no. 5, 2014

Summery of the Approach

The proposed character recognition method has three main steps: preprocessing, training and recognition. Noise tolerance and removal of more outliers was a important factor for the success. The character recognition was based on Growing Neural Gas and multi-layer perceptron networks which are types of Artificial Neural Networks.

Detail : Normalization

Noises are categorized into three levels as follows.

1. Mistaken strokes. (With purposefully or errors of tracking)
2. Extra and unwanted movements. (unintentional jitters)
3. Movements before the start point and after the end point are considered as useless data (Due to the continues streaming)

Eliminate such kind of useless  data from the obtained path has two steps.

1. Constructing the region of interest; 
2. Removing noisy data.

Constructing Region of Interest

A boundary is being determined so that the user can draw the character within this boundary. This boundary is a Tetrahedral with predefined measures.

Tetrahedral

Noise Reduction

For reducing the noisy data using this approach, a normalization technique with first derivation degree is used. First the first derivation of considered curve is being calculated which gives back the slope between the sampling points directly. The result is the amount of change in the direction of the curve in each moment. These results can help the system to delete the sampling points which have high amount of change in their location.

To omit the environment noises, the first derivation of the extracted path is being calculated to achieve the slope between sampling points. By use of the first deviation results, system computes the angle of changes.

In this technique, two thresholds are chosen then the calculated angles are being normalized and sit through 0 to 360. After that, all the sampling points which are located in defined thresholds will be eliminated.

By this normalization process, all the small environmental noises and breaks are eliminated.

In this technique, an interval will be defined by the user. If the angle between two lines be in the defined criteria, the sampling point, which connects these two lines, will be eliminated and the previous point will be connected to the current point.

In this normalization technique, the value of first deviation of extracted path which contains so many sampling points is calculated. So after calculating the first derivation, the slope between each two lines is available. In continue, the slopes will be investigated and the local angel for each sample point will be calculated.

In this technique, each achieved angel comparison with the previous angel. If the difference between them is lower than threshold, the calculated point will be eliminated and the previous point will connect to the next point.

After eliminating the noisy sample points, two steps need to be performed.

1. Extracting the key-features
2. Choosing limited numbers of key-features

Specifying the key-features helps the character recognition algorithm to detect the considered character and ignore processing the useless data. To find these key-features, a down-sampling algorithm based on growing neural gas technique is used.

The achieved sampling points from the noise reduction part are transferred as an input to down-sampling algorithm. Then the growing neural gas algorithm starts to add the key-features to the points and change their current position to the right position.

According to the congestion of the sampling points in different locations, the growing neural gas algorithm defines and locates the key-features.

Growing Neural Gas (GNG)

Growing neural gas is an unsupervised learning method. The growing neural gas also is able to recognize the number of neurons needed to describe an input dataset. In this algorithm, at first there are only two neurons in the system, then the input data are being feed to growing neural gas.

According to the location of input data, the location of neurons will be changed to reduce the errors of the system. After having specified number of input signals, a neuron will be added to system in order to reducing the average local errors of the system.

Character Recognition

After normalization part and gathering the key-features, now all the data to execute the character recognition part are available. In character recognition part, a multi-layer perceptron (MLP) neural network is used.

An MLP consist of at least 3 layers, an input layer, one or more hidden layer(s) and an output layer. 

The input layer is not considered a “true” layer because no computation is performed by it. It receives problem-specific inputs from the outside world.

An MLP contains one or more hidden layers, which receive inputs from preceding layers (input or hidden layers) and their outputs connect to the next layers (output or hidden layers). Each neuron in a hidden layer employs a nonlinear activation function that is differentiable.

Initially in the setup part, by the use of the defined dataset, the neural network is being trained to recognize the characters. There neural network receives the key-features as an input and defines the percentage of dependency of this set to each character. The character which has the highest dependency among other characters is chosen as the best candidate.