Interim Report - "Done" and here goes the Introduction

Yay!!!!! Completed and Submitted the Interim Report. Finally arrives with Literature Review.

Virtual “Valipilla” - Finger Gesture Based Learn to Write English Alphabet Tool

INTRODUCTION

The recent technologies in vision based sensors are competent enough to capture the finger positions and movements. This allows people to write something in the air even with the finger, which could be perfectly recognized by a computerized platform. So this research project was carried out considering the above two aspects in the domain of learning a language; more specifically writing the alphabet.

Motivation for the Project

Most of the educationists all over the world suggest that, for a kindergartner, the first tool that should be used to teach writing, more specifically the letters in the alphabet, must not be a pencil, pen, marker or a crayon. What they recommend is that the child should first use their fingers to move along the path of the letter. And that is why in Sri Lankan culture people had this tool called “valipilla” (වැලිපිල්ල) to teach how to write. It is an ideal multi sensory approach to teach the alphabet because the muscle movement helps the child to process what they are writing and make it more likely to stick in to their minds.

Traditional Sri Lankan ‘Valipilla’

With this traditional approach there need to be always an instructor behind the child to provide guidance how to write. Instructor has to show them how to write correctly, verify them whether they are writing correctly and show them the correct way if they are wrong. So his presence is always necessary to get an effective outcome.

Problem Statement

So, can we automate this process of verification that above mentioned? If so, then the instructor does not need to being always attended to the learning activity and the child could carry on by his own.  To answer that automation question we have to find solutions for two kinds of problems; How to accurately capture the finger movement, and how to verify whether it is the correct movement that we expected.

If a computer system could track the finger movement trajectory precisely with vision sensors then the first problem would be solved. Then we can use this trajectory input, to recognize as an alphabetical character when someone writes with a finger, and the second problem would be solved.

Application of the Research

So with this approach as mentioned above, we could easily replace the physical instructor and provide a finger gesture based virtual platform for the child to improve his writing ability all by his own. In other means it will add a higher value to the child education system by having an automated application to check whether the child has written the letters in the alphabet in correct way and by that provide the guidance to write it correctly. Not only for child education, but also this kind of a tool can be useful for an adult who learns a new language. Apart from the learning alphabet applications, this research can be further extended as a finger gesture based writing tool.

Aim and Objective

The main objective of this project is to research on a way to implement an accurate virtual platform to practice in writing English alphabetical letters using finger gesture and provide a developed tool as the concept of proof. Also the aim is to provide a prototype tool that can act as a supportive platform in the education system to improve writing ability when learning a new language.

Project Scope

The scope of the project is bound to the following areas mainly targeting on developing a finger gesture based learn-to-write tool for English alphabet.

• Capturing Finger Gesture

There should be a process to efficiently capture the index finger gesture movements and correctly identify gesture pauses in letter strokes. A suitable technique should be employed to mention the starting and ending point of the strokes.

• Recognition of Alphabetical Letters

There should be a process to match the finger gesture input trajectory data with the expected trajectories. Literature survey is conducted to find a suitable recognition technique to perform this task.

• Gesture Writing and User Interface

There should be an interactive user interface (UI) with a drawing area to write the letter. When designing the UI it is important to keep in mind that the interaction done with a finger, in the air and not with a mouse, on a surface. And the activities should be designed by applying auditory and visual aids.

• Display Feedback

There should be a method to output the results of the recognition process in a presentable way to the user. This should include whether the user has written the character correctly or not. 

Novelty and Contribution of the Research

There are lots of approaches to recognize handwritten characters. The challenging part of this is combining those approaches to recognize something written in the air with finger gesture, and also capturing those finger gesture trajectories by accurately identifying the start and end of the letter stroke. Still the related work carried out in this research space are lacking of its accuracy. A huge weight of the performance depends on the used recognition technique and letter strokes capturing mechanism. There are few related research projects which have used different recognition techniques and capturing mechanisms. The focus of almost all of those projects was to build up an air-writing tool, but not to make it as a supportive tool for improve writing ability. So our expectation of this research is to contribute for learning area.