CN101799996B - Click-reading method of click-reading machine based on video image - Google Patents

Abstract

A kind of point reading machine point reading method based on video image, it is characterized in that method step is: (1) obtain the video image of point reading pen and book in real time with video camera; (2) utilize the nib of point reading pen to lightly touch switch, And this image is input in the microprocessor; (3) book edge outline is corrected; (4) set up book coordinate system; (5) adopt the rod-shaped characteristic of point reading pen to obtain the maximum connected area method; (6) according to The rod-shaped feature of the reading pen, using the line scanning method to obtain the position of the pen tip of the reading pen; (7) Use the length and width values of the known book and the pixel values of the length and width of the book in the image to calibrate the image coordinates; (8) Under the book coordinate system, the nib coordinates of the point reading pen are calculated by using the calibration results; (9) thus realizing the point reading of the point reader based on the video image. The invention has the advantages of: utilizing a video camera, a point reading pen and a microprocessor to realize the point reading function of the point reading machine; utilizing the maximum connected domain method to effectively eliminate the noise after image binarization and the interference of other non-target objects; The utility model has the advantages of being simple and practical, convenient to carry, good versatility and high reading accuracy.

Description

A Point Reading Method Based on Video Image

technical field

The invention relates to a point reading method of a point reader, in particular to a point reading method based on a video image.

Background technique

Point reader (also known as intelligent computer reading machine, English interactive electronic textbook, synchronous reading machine, English intelligent point reader, e-book, electronic textbook, etc.) is a sound interaction that perfectly combines modern digital information technology with the latest teaching needs. It is a learning product that can turn written textbooks into audio textbooks that can be arbitrarily produced according to learning needs.

In recent years, point readers have rapidly entered the domestic market and developed into one of the main auxiliary tools for children's learning at home and abroad. The current popular point reading machine adopts the pressure sensing method to obtain the point reading coordinates to realize the point reading function. The point reading principle is: place a dot-matrix pressure sensor in the pressure-sensitive panel, and use the method of scanning the abscissa and ordinate to obtain the position of the point-reading pen on the pressure-sensitive panel (that is, the x and y coordinate values). The microprocessor receives the point-reading instruction, converts the point-reading coordinates on the pressure-sensitive panel into corresponding book coordinates, reads out the corresponding voice file, and realizes the point-reading function. The point reader using the pressure sensing method has the advantages of high point reading accuracy, but because its pressure sensing panel needs to be protected by a plastic shell and accommodate books of various sizes, it is bulky, heavy, and inconvenient to carry; different manufacturers produce Point readers, even point readers of different models from the same manufacturer, all require a dedicated pressure-sensing panel, which is poor in versatility.

Contents of the invention

The object of the present invention is to provide a point-reading method based on video images, which is simple, practical, easy to carry, good in versatility, and has high point-reading accuracy.

The present invention is realized like this, it is characterized in that method step is:

(1) obtain the video image of point reading pen and book in real time with video camera;

(2) Use the nib of the point reading pen to lightly touch the switch to trigger the microprocessor to capture a single frame image, and input the image into the microprocessor;

(3) First use the edge detection operator and the threshold segmentation method to binarize the image, and then use image expansion, image erosion, image filtering and maximum connected domain method to eliminate the interference of non-target objects, accurately extract the edge contour of the book, and Book edge contour correction;

(4) Obtain the pixel value of the length and width of the book by using the edge contour of the book, and establish the book coordinate system;

(5) First, according to the color information of the point reading pen, the threshold segmentation method is used to binarize the image, and then the maximum connected domain method is used to obtain the rod-shaped feature of the point reading pen;

(6) according to the rod-shaped feature of point reading pen, utilize line scanning method to obtain the nib position of point reading pen;

(7) Use the length and width values of the known book and the pixel values of the length and width of the book in the image to calibrate the image coordinates;

(8) Under the book coordinate system, use the calibration result to calculate the pen tip coordinates of the point reading pen;

(9) According to the coordinate value of the nib in the book coordinate system, the corresponding voice information is called out, thereby realizing the point reading based on the video image.

The point reading function of the point reading machine is realized by using a video camera, a point reading pen and a microprocessor.

The point reading pen has a pen holder of a special color and a light touch switch at the tip of the pen. When the switch is triggered, a signal can be generated and can be recognized and received by the microprocessor.

The principle is: use the video camera to obtain the video images of the reading pen and the book in real time; use the image edge detection operator and the threshold segmentation method to binarize the image, so that the edge outline of the book can be separated from the background. In order to accurately extract the edge outline of the book, Use image expansion and erosion, image filtering and maximum connected domain method to eliminate background and noise interference; use least squares fitting method to get the upper edge straight line of the book, calculate the angle between the upper edge straight line of the book and the horizontal direction, and use the angle Correct the edge outline of the book; on this basis, use the least squares fitting method to obtain the edge straight line of the book, and establish the book coordinate system; according to the special color information of the reading pen, use the threshold segmentation method to binarize the image, and use The maximum connected domain method extracts the outline of the reading pen and determines the position of the pen tip; uses the length and width values of the known book and the pixel value of the book's length and width in the image to calibrate the image coordinates; uses the calibration results to calculate the pen tip of the reading pen Coordinates, according to the coordinate value of the pen tip of the point reading pen in the book coordinate system, the corresponding voice information is called out, and the point reading machine based on the video image is realized.

The advantages of the present invention are: the maximum connected domain method is used to effectively eliminate the noise after image binarization and the interference of other non-target objects; the edge straight line equation of the book is obtained by the least square fitting method, and the edge contour of the book is accurately detected; The rod-shaped feature of the point reading pen is accurately obtained by using the special color of the point reading pen; the point reading accuracy is improved by using the above method, and the point reading function of the point reading machine based on the video image is realized by using a video camera, a point reading pen and a microprocessor. Therefore, the point reader based on the video image has the advantages of being simple and practical, easy to carry, good versatility, and high point reading accuracy.

Description of drawings

Fig. 1 is the work flowchart of the present invention.

Detailed ways

The method of the present invention requires a simple bracket to support an ordinary wide-angle camera so that it can shoot vertically. There is only a single book in the scene of shooting, and it can completely contain the whole book. The reading pen is solid color and has a light touch switch on the tip of the pen. The microprocessor can receive the signal generated when the light touch switch on the pen tip is triggered, and control the camera to acquire images. At the same time, the actual length and width of the book should be known.

The present invention adopts the flow chart of the point-reading method based on the video image as shown in Figure 1 to accurately measure the point-reading position of the point-reading machine and read the corresponding voice, and its specific implementation steps are as follows:

1. Real-time acquisition of video images

Place the camera with the bracket at a suitable position directly in front of the book, and obtain video images of the reading pen and the book in real time.

2. Is the nib light touch switch triggered?

It is judged whether the development of the pen tip is triggered, and if it is triggered, the microprocessor executes the single-frame image acquisition of the point reading pen and the book, otherwise, it waits for point reading.

3. Single frame image acquisition

When the point reading pen reads, the light touch switch on the pen tip is triggered, and a signal is sent to the microprocessor to control the camera to collect single-frame image data and input it into the microprocessor.

4. Contour extraction and correction of books

Extract and correct the edge contour of the book on the single frame image obtained in step 3. The main steps are as follows:

1) Use the edge detection operator to extract the edge information of the book, where the edge detection operator can be Canny, Robert or Sobel.

2) The threshold method is used to binarize the image, and the image expansion, image erosion, image filtering and maximum connected domain method are used to eliminate the interference of non-target objects and accurately extract the edge contour of the book.

3) After obtaining the upper edge line of the book by vertical scanning method and twice least squares fitting method, calculate the angle α between the line and the horizontal direction, rotate the image clockwise by angle α, and complete the image correction.

The twice least squares fitting method is as follows:

Firstly, the first least squares fitting method is performed on the upper edge point set of the book to obtain the upper edge straight line equation; secondly, the distance from each point in the point set to the line is calculated and sorted from small to large, and the two points with the longest distance in the point set are removed. One-third of the points, the influence of noise points is excluded, and finally, the upper edge line is obtained by the second least square fitting method using the remaining points in the point set.

5. Establishment of Book Coordinate System

Using the book edge contour image corrected in step 4, the main steps to establish the book coordinate system are as follows:

1) Using the feature of the shortest distance between the vertices of the book and the vertices of the corresponding image, determine the positions of the upper left and upper right vertices of the book, and calculate the pixel value of the length of the upper edge of the book, which is the pixel value of the length of the book.

2) Use the position of the upper left vertex of the book and the straight line equation of the upper edge of the book obtained in step 4, and use the feature that the upper edge of the book is perpendicular to the left edge to calculate the straight line equation of the left edge of the book.

3) Knowing the length and width of the book and the obtained pixel value of the upper edge length of the book, calculate the pixel value of the position of the lower left vertex of the book and the length of the left edge, which is the pixel value of the book width.

4) Take the upper left vertex as the origin, and use the straight line on the upper edge of the book and the straight line on the left edge as the coordinate system x-axis and y-axis respectively to establish a book coordinate system.

6. Rod-shaped feature extraction of the reading pen and determination of the position of the pen tip

The present invention utilizes the special color of the point reading pen to extract the rod-shaped feature of the point reading pen, and obtains its nib position by scanning method, and its main steps are as follows:

1) Use the red color (or blue, green, etc.) of the reading pen, and use the threshold segmentation method to separate the edge contour of the reading pen from the image.

2) Use the maximum connected domain method to remove other interferences and extract the rod-shaped features of the reading pen.

3) Use the line scanning method to obtain the position of the pen tip. If the point reader is held by the right hand, scan from left to right to obtain the position of the point reader; if the left hand holds the point reader, scan from right to left to obtain the point reader the nib position.

7. Image coordinate calibration and point reading pen tip coordinate calculation

The image coordinates are calibrated by using the length and width values of the known book and the pixel values of the book length and width in the image, that is, the actual coordinate value corresponding to each pixel in the image is obtained. Use the calibration result and the tip position obtained in step 6 to calculate the tip coordinates of the point reading pen.

8. Realization of reading function

According to the coordinates of the pen tip of the point reading pen, the corresponding voice information is called out, and the point reading function of the point reader based on the video image is realized.

Claims (3)

1. a kind of point reading machine point-reading method based on video image, it is characterized in that method step is: (1) obtain the video image of point reading pen and book in real time with video camera; (2) Utilize the nib touch switch of the point reading pen, when the point reading pen points to read, the touch switch on the pen tip is triggered, and a signal is transmitted to the microprocessor to control the camera to collect single-frame image data and input the image into the microprocessor; (3) First use the edge detection operator and the threshold segmentation method to binarize the image, and then use image expansion, image erosion, image filtering and maximum connected domain method to eliminate the interference of non-target objects, accurately extract the edge contour of the book, and The book edge contour is corrected; the main steps of book contour extraction and correction are: firstly, use the edge detection operator to extract the edge information of the book, and the edge detection operator uses Canny, Robert or Sobel; value, and use image expansion, image erosion, image filtering and maximum connected domain method to eliminate the interference of non-target objects, and accurately extract the edge contour of the book; finally, use the vertical scanning method and the least square fitting method twice to get the upper part of the book After the edge is straight, calculate the angle α between the line and the horizontal direction, rotate the image clockwise by α angle, and complete the correction of the image; (4) Obtain the pixel value of the length and width of the book by using the edge contour of the book, and establish the book coordinate system; the steps for establishing the book coordinate system are: 1) Using the feature of the shortest distance between the vertices of the book and the vertices of the corresponding image, determine the positions of the upper left and upper right vertices of the book, and calculate the pixel value of the length of the upper edge of the book, which is the pixel value of the length of the book; 2) Using the position of the upper left vertex of the book and the obtained straight line equation of the upper edge of the book, and using the feature that the upper edge of the book is perpendicular to the left edge, calculate the straight line equation of the left edge of the book; 3) Knowing the length and width of the book and the obtained pixel value of the upper edge length of the book, calculate the pixel value of the position of the lower left vertex of the book and the length of the left edge, which is the pixel value of the book width; 4) Take the upper left vertex as the origin, and use the straight line on the upper edge of the book and the straight line on the left edge as the coordinate system x-axis and y-axis respectively to establish a book coordinate system; (5) First, according to the color information of the point reading pen, the threshold segmentation method is used to binarize the image, and then the maximum connected domain method is used to obtain the rod-shaped feature of the point reading pen; (6) according to the rod-shaped feature of point reading pen, utilize line scanning method to obtain the nib position of point reading pen; (7) Use the length and width values of the known book and the pixel values of the length and width of the book in the image to calibrate the image coordinates; (8) Under the book coordinate system, use the calibration result to calculate the pen tip coordinates of the point reading pen; (9) According to the coordinate value of the nib in the book coordinate system, the corresponding voice information is called out, thereby realizing the point reading based on the video image. 2. a kind of point reading machine point-reading method based on video image according to claim 1 is characterized in that: utilize video camera, point-reading pen and microprocessor to realize the point-reading function of point-reading machine. 3. A kind of point reading machine point-reading method based on video image according to claim 1, it is characterized in that described point-reading pen has the pen holder of special color, and nib has light touch switch, can generate signal when switch triggers and Can be recognized and received by the microprocessor.

Images