KR20160033329A - System and Method for Translating Sign Language - Google Patents

Abstract

The present invention relates to a sign language translation system and method, and more particularly, to a sign language translation system and method, which includes a key input unit for inputting a command of a user, a photographing unit for photographing a sign language, an operation recognition unit for converting the image information of sign language, A database storing character information and voice information corresponding to the sign language information value of the action and the sign language information corresponding to the sign language information of the operation, a database for comparing the sign language information value converted in the action recognition unit with the sign language information value stored in the database, An operation analyzing unit for retrieving information and an output unit for outputting character and / or voice information as a result of retrieval from the operation analyzing unit, whereby the listener can transmit his or her intention to the non-speaker.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sign language translation system,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sign language translation system and method, and more particularly, to a sign language translation system and method for outputting a translation character or voice information for a sign language sign language operation using motion recognition.

In modern society, the welfare policy for the disabled is getting bigger day by day, but the communication between the disabled and the non - disabled is not smooth and inconvenient.

Considering these points, we propose a sign language translation system that enables the handicapped to communicate the handicap even to the non-handicapped who are not familiar with sign language by recognizing the sign language of the handicapped.

Korean Patent Laid-Open Publication No. 10-2000-0017756 (published Apr. 2000, 2000)

SUMMARY OF THE INVENTION Accordingly, the present invention has been made to solve the above-mentioned problems occurring in the prior arts, and it is an object of the present invention to provide a handwriting recognition apparatus and a handwriting recognition apparatus which convert a handwriting operation into a number of hands, a number of fingers, And outputting a character and / or voice information about a sign language operation by matching the speech with the speech.

According to another aspect of the present invention, there is provided a sign language translation system including a key input unit for inputting a command of a user, a photographing unit for photographing a sign language operation, an operation for converting the image information of the sign language image captured by the photographing unit into sign language information A database for storing sign language information corresponding to the sign language information value of the sign language and the sign language information corresponding to the sign language information value, a database for comparing the sign language information converted in the action recognition unit with the sign language information value stored in the database, And / or audio information, and an output unit for outputting character and / or audio information as a search result from the operation analysis unit.

On the other hand, the sign language translation method of the present invention includes a step of receiving a sign language operation and a translation start function key from a key input unit, a step of photographing a sign language operation operation of a photographing unit, The operation analyzing unit compares the sign language information converted by the action recognition unit with the sign language information value stored in the database to search for the corresponding character and / or voice information, and the output unit searches the operation interpretation unit And outputting the resultant character and / or voice information.

As described above, according to the sign language translation system and method of the present invention, the sign language operation is converted into the number of hands, the number of fingers, the azimuth angle of the hand and finger movements, etc., It is possible for the receiver to transmit his / her own opinion to the non-receiver.

1 is a block diagram schematically showing a sign language translation system according to an embodiment of the present invention.
2 is an execution screen showing the sign language information recognized by the action recognition unit of the sign language translation system according to the embodiment of the present invention.
3 is an operational flowchart of a sign language translation system according to an embodiment of the present invention.
4 is an example of a display result for a sign language translation system according to an embodiment of the present invention.

Each block or step may represent a portion of a module, segment, or code that includes one or more executable instructions for executing the specified logical function (s). It should also be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially concurrently, or the blocks or steps may sometimes be performed in reverse order according to the corresponding function.

Hereinafter, a sign language translation system of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing a sign language translation system according to an embodiment of the present invention.

1, a sign language translation system 100 according to an embodiment of the present invention includes a control unit 110 that controls the overall operation of the sign language translation system 100, a key input unit 120 that receives a user's command, An operation recognition unit 140 for converting the image information of the sign language operation of the sign language operation into the sign language information such as coordinate value and azimuth angle and recognizing the sign language operation, A database 150 for storing information and character and voice information corresponding to the sign language information value, and a display unit for comparing the sign language information converted in the operation recognition unit with the sign language information value stored in the database, And an output unit 170 for outputting character or voice information as a search result from the operation analyzing unit.

The control unit 110 corresponds to a processor that controls overall control of the components of the sign language translation system 100, and may be configured to include some or all of the above components.

The key input unit 120 includes various function keys for selecting a sign language operation, a translation start / end, a translation information output unit, and the like.

In the embodiment of the present invention, two infrared cameras (or an infrared ray recognition module), three infrared ray LEDs A light source, and the like.

The motion recognition unit 140 converts a video signal output from the photographing unit 130 into sign language information such as coordinate values and azimuth angle, recognizes a sign language operation, and outputs the sign language sign to the operation analysis unit 160.

In one embodiment of the present invention, the motion recognizing unit 140 tracks the movement of the hand using the infrared camera of the photographing unit 130 and draws it in a 3D map to recognize the movement of the hand. That is, the image input through the photographing unit 130 is divided into x-axis and y-axis as well as z-axis information as depth information and converted into coordinate values to analyze the position of the hand in the image, And recognizes the movement of each finger in detail.

More specifically, the motion recognition unit 140 first receives the image information from the photographing unit 130, determines the x, y, and z axes, and then extracts specific points on the screen to continuously orbit the moving point After tracking, analyze the tracked points to recognize the motion. Then, the motion recognition unit 140 converts the image information of the hydration operation output from the photographing unit 130 into numerical values such as x, y, z coordinate values, azimuth angle, and the like.

FIG. 2 is an execution screen showing the sign language information recognized by the action recognition unit of the sign language translation system according to the embodiment of the present invention. For example, the number of hands is 1 when both hands of the hand- If two hands are used, it becomes 2. In addition, the number of fingers recognizes the number of fingers extended, the tools are used when a tool other than the hand of the listener is recognized, and the gesture is a movement of a hand, for example, Is defined as TYPECIRCLE, and TYPESWIPE for gestures that shake hands up and down.

In addition, the average finger tip position and palm position are represented by x, y, z axis values from the constructed 3D map, and the hand sphere radius is the value from the configured 3D map in mm .

Finally, the motion recognizing unit 140 rotates about an axis in a horizontal plane perpendicular to the moving direction of the hand based on the azimuth angle information in the image signal applied from the photographing unit 130, that is, the x-axis rotation azimuth ), A rotation around an axis in a parallel horizontal plane with respect to the direction of movement of the hand, that is, a y-axis rotation azimuth (hand roll), a rotation about an axis in a vertical plane perpendicular to the movement direction of the hand, hand yaw).

The database 150 is a structure for temporarily storing various data to be processed by each component, and includes a sign language information value such as a coordinate value and an azimuth for the sign language operation, a character and voice information corresponding to the sign language information value, For this purpose, the hydration operation is converted into a database.

In an embodiment of the present invention, the hand motion may be arbitrarily defined and converted into a database. For example, according to the number of fingers, it may be defined as "Hello" when the number of fingers is one, and "Would you tell me a way if you have two fingers", or "I love you" when you draw a circle with your fingers.

The operation analyzing unit 160 compares the sign language information about the sign language conversion operation performed by the action recognition unit 140 and the sign language information value stored in the database 150 to search for the same sign language information, And searches for character or audio information. Accordingly, the searched character or voice information can be displayed or reproduced on the output unit 170.

The output unit 170 outputs the character or voice information as a search result from the operation analysis unit 160. The character output unit 170 outputs character information Or a speaker type for outputting audio information.

On the other hand, the sign language translation system 100 may be configured to be portable so as to be portable, and may include a power source unit for supplying a predetermined operation power source Vcc to its internal components.

Hereinafter, the sign language translation method of the present invention using the system configured as above will be described.

3 is an operational flowchart of a sign language translation system according to an embodiment of the present invention.

First, in a state in which the sign language information such as the coordinate value and the azimuth for the sign language and the interpretation information such as various characters or voices corresponding thereto are stored in the database 150, the user turns on the operation state of the sign language translation system 100 When the start function key of the post-key input unit 120 is input (S201), the control unit 110 controls the respective components to supply the operation power from the power unit.

The control unit 110 determines whether the output type selection function key is input from the key input unit 120, that is, whether the character type information is output through the output unit 170, voice output, If the selected output type information is input from the key input unit 120, the selected output type information is stored in the database 150. [ On the other hand, if the output type information is not applied from the key input unit 120 for a predetermined time, the control unit 110 can automatically set the output type information to the character output set by default. At this time, the basic setting mode of the output type information can be arbitrarily set by the user or can be set by the manufacturer at the time of manufacturing the sign language translation system.

Meanwhile, the photographing unit 130 transmits the image signal applied from the first and / or the second camera to the motion recognizing unit 140 (S202).

Next, the motion recognition unit 140 generates sign language information corresponding to the sign language operation by converting the video signal of the sign language output from the photographing unit 130 into numerical values such as a coordinate value and an azimuth angle (S203) And sends the sign language information to the operation analysis unit 160.

Then, the operation analyzing unit 160 extracts corresponding information from the database 150 based on the sign language information applied from the action recognition unit 140, and performs an operation analysis (S204).

When the user inputs the end function key information of the key input unit 120, the character and / or voice is provided to the output unit 170 (S205).

On the other hand, when predetermined character and / or voice data is received from the operation analyzing unit 160, the output unit 170 outputs the corresponding character and / or voice data (S206).

Therefore, the receiver can easily communicate his / her wish to the non-receiver by using sign language in a free posture.

Meanwhile, the sign language translation system of the present invention can be applied to various communication systems to provide a receiver with convenient use of communication devices.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the invention.

100: Sign language translation system 110:
120: key input unit 130:
140: motion recognition unit 150: database
160: Operation analysis unit 170: Output unit

Claims (4)

A key input unit for inputting a user command,
A photographing unit for photographing a sign language operation,
An operation recognition unit for converting the image information of the sign language operation captured by the image capturing unit into sign language information,
A database storing sign language information values of sign language operations and character and voice information corresponding to such sign language information values,
An operation analyzer for comparing the sign language information converted by the action recognition unit with the sign language information value stored in the database and searching for the corresponding character and / or voice information;
And an output unit for outputting character and / or voice information which is a search result from the operation analysis unit.
The method according to claim 1,
Wherein the motion recognition unit comprises:
The x, y, and z axes are determined while receiving the image information from the photographing unit, and the number of hands, the number of fingers, the average position of the fingertip, and the palm position are determined as x, y, The rotational azimuth angle around the axis in the horizontal plane perpendicular to the direction of movement of the hand, the rotational azimuth angle about the axis in the parallel horizontal plane relative to the direction of movement of the hand, And recognizes the hydration operation by converting the azimuth angle into the sign language information.
Receiving a sign language operation photographing and a translation start function key from a key input unit,
A step of photographing a shooting operation of the shooting unit,
Wherein the operation recognition unit converts the image information of the sign language operation captured by the image capturing unit into sign language information,
Comparing the sign language information converted by the action recognition unit with the sign language information value stored in the database and searching for the corresponding character and / or voice information;
And outputting a character and / or voice information which is a search result from the operation analyzing unit.
4. The method of claim 3,
Wherein the step of converting the video information of the sign language operation into sign language information comprises:
Wherein the motion recognition unit recognizes the x, y, and z axes while receiving the image information from the photographing unit, and determines the number of hands, the number of fingers, the average position of the fingertips, and the palm position as x, y, z Coordinate value values, and includes a rotational azimuth angle around the axis in a horizontal plane perpendicular to the moving direction of the hand, a rotational azimuth angle around the axis in a parallel horizontal plane with respect to the hand moving direction, and a vertical plane perpendicular to the moving direction of the hand And converting the azimuth angle into sign language information.

Images