Image Processing and Button Hole Problem

A large scale manufacturing company produces buttons which are exactly identical in shape except the number of holes.

What you have to do is to develop a Computer Vision system.

It should acquire an image of a single button.

Then it should determine whether the button is of 6, 4, 2, 0 holes. Number of no holes are defects.

After identifying the number of holes, button should direct to a separate box according to the number of holes.

Before move into implementation, let's watch the process of a button manufacturing company.

As you see, in a normal button process, production of each type of button happened in separately. Actually there is no need to clarify buttons depending on its holes, because creating holes for 2 holes, 4 holes, 6 holes are separate processes. Only the vision system used to identify defects. Anyway let's assume they are all mixed in the tray. 

Okay!! Now let's begin our implementation.

Here is our button tray that captured from the camera.

This is the system we propose.

Robotic arm can move across our button tray. Also it can move down and stick the button to its arm. It releases the button when it moves into correct button basket where button belongs to. When the button tray is empty, tray moves forward to identify next set of buttons

Let's begin our button type recognition process. To do this we're using OpenCV 2.4 with Java wrapper.

First we thought to use Hough Circle Transformation for identify the holes in buttons. But the results are not always 100% accurate for that approach. We can say it will give 85% best results. But what we expect is 99% accuracy. Therefore we went for another approach.

Step 1

Our first step is to load the image and convert it to gray scale.

Step 2

Then blur the image to reduce the noise effect. Here we're using Gaussian blur technique.

Step 3

Then detect the edges. Here we're using Canny edge detector.

 Step 4

Now we are ready with our edge detected image to segment the button regions. For that we're using Contours.

Robotic arm moves along each of the button region. As the boundary taken both as a square and circular region, robotic arm could easily attach the button into it.

Step 5

Next step is recognition of holes. For that we're again using counters within each of the identified segment.

By identifying which basket the button belongs to, robotic arm could easily release the attached button at the basket. When a basket is filled we could easily take another basket. 

Here is the log of the current button tray. As this button tray is fully checked, tray can be moved forward to check for the next step.