KR20220096280A - Braille keyboard system based on smartphone case - Google Patents

Abstract

The present invention relates to a smartphone case-based braille keyboard system, comprising: a braille keyboard installed on the back of a smartphone;
an input unit transmitting an input signal of the input key to a control unit;
text-to-speech conversion unit;
audio output unit;
Consists of a storage unit and a control unit for storing data, the present invention implements a braille keyboard with high accuracy and fast input speed, and can be used while moving because characters can be entered while holding a smartphone, and input characters is output through the application, and there is a remarkable effect that the user can recognize typos and corrections of characters input through screen or voice output.

Description

Smartphone case-based Braille keyboard system {Braille keyboard system based on smartphone case}

The present invention relates to a smartphone case-based braille keyboard system, and more particularly, a braille keyboard with high accuracy and fast input speed is implemented, and characters can be entered while holding a smartphone, which can be used on the go, The input text is output through the application, and the user relates to a smartphone case-based braille keyboard system capable of recognizing typos and corrections of input text through screen or voice output.

In general, according to a survey by the Ministry of Health and Welfare in 2017, it was confirmed that 55.9% of respondents (levels 1 to 4 with visual impairments) used smartphones. However, since smartphones mainly provide visual information, there are many limitations for visually impaired people to use smartphones. Keypad-related services are being supported to increase smartphone access for the visually impaired. Google supports the Talk Back service, which is a method of touching the screen in the form of Braille, and Samsung and Apple support Voice Assistant and Voice Over services that can recognize the characters to be input by outputting the touched text as a voice when the smartphone keypad is touched. do. However, in the case of the Talk Back service, since both hands touch the screen, there is a disadvantage that the smartphone must be placed on an object such as a desk. In the case of Voice Assistant and Voice Over service, since the keypad is input after confirming the voice, there is a disadvantage of memorizing the approximate location of the keypad and checking the voice for each phoneme input.

And as an example of the prior patent technology for the braille input, in the registered patent publication No. 10-1580316, in the braille input/output device for a smartphone for inputting and outputting braille in conjunction with a smartphone, the external covering the back and side surfaces of the smartphone It consists of a pad in the form of a case, the pad comprising:

a receiving space in which the smartphone is inserted and fixed to the front;

a braille output unit located on the side of the receiving space and outputting the characters input to the smart phone in braille;

a braille input unit formed on the rear surface and inputting one braille; and

Includes; a control device for controlling the braille output unit in conjunction with the smart phone, and processing the braille input through the braille input unit;

The pad further includes a wireless module interworking with a wireless unit included in the smartphone, the wireless module is controlled by the control device, the smartphone further includes a processing module, the processing module is a smartphone It further includes an output tab for instructing the output of the inputted characters and a silent input unit for processing the written characters through the input device, wherein the braille input unit includes a braille button consisting of 6 unit buttons, a delete button for deleting the input braille, and a delete button for deleting the inputted characters. and a completion button for completing the braille and transmitting it to the control device, wherein a recognizer capable of distinguishing each unit button is formed on the surface of the unit buttons of the braille input unit

Braille input/output device for smart phone is disclosed.

In addition, Patent Registration No. 10-1491039 discloses that in a braille input device for a smartphone for inputting braille into a smartphone, it consists of a pad in the form of an outer case surrounding the back of the smartphone, the pad includes:

a case including a seating part surrounding the smartphone;

a braille key formed on the upper surface of the case to input braille;

a plurality of function keys formed on the upper surface of the case; and a control device that detects the operation of the braille key and a plurality of function keys and transmits the input braille or smartphone operation function to the smartphone; including, wherein the function keys are a backspace key, a shift key, a space key, and enter It includes a key, a transmission key, and a mode change key, wherein the mode change key changes the functions of other keys to Korean, English and smartphone operation according to the operation, and the pad further includes a vibration element controlled by the control device. Including, when the vibrating element operates in a different mode according to the operation of the mode change key, a braille input device for a smartphone is disclosed, characterized in that it vibrates at a frequency differentiated for each mode.

However, the prior art has disadvantages in that it is inconvenient to use while moving, and it is difficult for a user to easily recognize typos and corrections of input characters.

Therefore, the present invention has been devised to solve the above problems, and a braille keyboard with high accuracy and fast input speed is implemented, and characters can be entered while holding a smartphone, so that it can be used while moving, and the input characters is output through the application, and the user intends to provide a smartphone case-based braille keyboard system that can recognize typos and corrections of characters input through screen or voice output.

The present invention relates to a smartphone case-based braille keyboard system, comprising: a braille keyboard installed on the back of a smartphone;

an input unit transmitting an input signal of the input key to a control unit;

text-to-speech conversion unit;

audio output unit;

storage to store data

and a control unit.

Therefore, the present invention implements a braille keyboard with high accuracy and fast input speed, and can input characters while holding a smartphone, so it can be used while moving, the input characters are output through the application, and the user can use the screen or There is a remarkable effect of recognizing typos and corrections of characters input by voice output.

1 is a block diagram of the present invention smart phone case-based braille keyboard system.
2 is a braille standard diagram of the present invention;
3 is an explanatory diagram for converting Braille into a signal according to the present invention
Figure 4 is a flow chart of the braille keyboard system of the present invention.
5 is an exemplary picture of a system in which the present invention is implemented;
6 is an input value experimental result diagram of the present invention;

The present invention relates to a smartphone case-based braille keyboard system, the braille keyboard installed on the back of the smartphone;

an input unit transmitting an input signal of the input key to a control unit;

text-to-speech conversion unit;

audio output unit;

storage to store data

and a control unit.

In addition, the input keys of the braille keyboard are composed of eight input keys in an array form, and six input keys are arranged in braille form and function to input characters, and the remaining two input keys are space, deletion, transmission and line. It is characterized in that it is a function key in charge of the change function and voice conversion of the input text.

In addition, the keyboard is characterized in that it is a type using a button type or FSR sensor.

The present invention will be described in detail with reference to the accompanying drawings as follows.

This system implements a braille keyboard in the form of a smartphone case, and consists of a total of 8 input keys in the form of an array. The 6 input keys are arranged in the form of Braille and function to input characters. In addition, the remaining two input keys are function keys responsible for space, deletion, transmission, and line change functions and voice conversion of input characters. The hardware and application transmit text input through wired communication using an OTG cable, and implement a text-to-speech conversion function in the application so that the output text can be checked as voice output. to be.

Braille of the present invention is an important tool for visually impaired people to acquire knowledge and skills, and for reading, education, daily life, and the like. Braille is a global standard that consists of a total of 6 dots by placing dots in 2 horizontal and 3 vertical lines. Each dot is assigned a unique number and is distinguished. Braille in Hangeul consists of six dots representing one phoneme, and is divided into a beginning, a vowel, and a final sound. As an example of Korean Braille notation, the word 'Braille' is written in five Braille characters, and read as 'ㄴ', 'ㅓ', 'ㅁ', 'ㄱ', and 'A'. This system attaches six buttons in the form of braille to the smartphone case so that characters can be entered by designating the same number in braille. Figure 2 is a standard in Braille form.

When the text-to-speech function of the present invention is described, text-to-speech (TTS) refers to a technology for converting text into voice by implementing a human voice. There is a disadvantage that pronunciation and intonation are not natural when converting text into speech, but pronunciation and intonation are developing naturally based on deep learning technology. This system was implemented by utilizing the TTS function supported by Android to check the contents and typos when the visually impaired input text. Through this, the text written in the application can be played back by voice, so that the user can recognize the written text.

The actual braille has a unique pattern for each dot as shown in FIG. 3, and six braille input keys of the hardware have the same pattern as the actual braille. In the present invention, reference was made to the standard Braille table to determine and convert Braille into characters, and the input key values (0=low, 1=high) were equally designated by dividing the Braille into protruding parts and non-braille parts. The values of each braille were listed in the order of the dots, and the characters corresponding to the listed values were stored and constructed with software. When the user presses the braille input key, a character corresponding to the input braille is determined through a braille determination algorithm. In order to increase the accuracy of phonemes that require input of two or more buttons, the Braille determination algorithm compensates for the time difference when the buttons are clicked simultaneously by transmitting the value of the input key after 0.3 seconds from the time the input key is pressed. After that, the values of the input keys are listed in the order of dots, and the characters that are compared and matched with the built software are transmitted to the application by referring to the standard Braille table. There are two function keys, and the corresponding function is transmitted by determining the click action of each button. The transmitted signal can be monitored by outputting it from the application. Figure 4 shows the flow chart of the whole system including the Braille decision algorithm.

The present invention implemented a braille keyboard consisting of six braille buttons and two function buttons at the grip position on the back of the smartphone case, and designed a braille keyboard with an FSR sensor attached in the same way. A character input key was implemented through a braille determination algorithm using a braille standard, and the function key implemented two functions as one input key according to the click operation. The left side plays the role of space and deletion, and the right side performs transmission and line breaks and text-to-speech output. The application can check the output text by voice by implementing the TTS function. Figure 5 shows the configuration of the implemented system and two types of hardware. As another embodiment of the present invention, a vibration element is installed in the smartphone, and when an incorrect word is input by an input key, the control unit operates the vibration element so that the user knows. At this time, the control unit examines the completeness of each syllable in comparison with the dictionary built in the storage unit. In addition, it is possible to inform that a typo has occurred through the voice output unit. The vibration function is also applied to spacing according to the setting. Vibration is turned off when the corresponding misspelling is corrected or deleted, and the control unit can also inform where the misspelling is located in the current location through the voice output unit.

Therefore, the input accuracy experiment of the implemented system is as follows. In order to evaluate the accuracy of the text input of the implemented system, an experiment was conducted with 5 experimenters, and the number of errors and ' Accuracy was derived from the number of errors that occurred by entering the word 'braille keyboard' 50 times. As a result of the experiment, it was confirmed that the button-type hardware had an accuracy of 98.6% for phoneme input and 95.6% for word input. In the case of FSR sensor type hardware, the accuracy of phoneme input 88.2% and word input 58% was confirmed. Figure 6 is an example of the results of phoneme input experiment and word input experiment of hardware. Tables 1 and 2 show the average value and system accuracy of the number of typos through each hardware of the Braille keyboard system.

Sub. phoneme Word input Error acc.(%) input Error acc.(%) One 100 2 98 50 4 92 2 100 0 100 50 One 98 3 100 0 100 50 2 96 4 100 4 96 50 0 100 5 100 One 99 50 4 92 Avg. (%) 100 1.4 98.6 50 4.4 95.6

Table 1. Button type hardware input accuracy

Sub. phoneme Word input Error acc.(%) input Error acc.(%) One 100 10 90 50 19 62 2 100 14 86 50 21 58 3 100 7 93 50 27 46 4 100 13 87 50 18 64 5 100 15 85 50 20 60 Avg. (%) 100 11.8 88.2 50 42 58

Table 2. FSR sensor type hardware input accuracy

Among the errors that occurred, the common error of the two hardware is that only some of the pressed buttons are recognized in the case of braille where multiple buttons must be pressed. Taking 'ㅚ' where 5 dots are input as an example, braille patterns such as 'a', 'ㄴ', 'ㅣ', 'ㅔ' are included in the braille pattern of 'ㅚ' by entering 1 to 4 dots. have. When 'ㅚ' is input, a character having a braille pattern included in the inputted braille pattern of 'ㅚ' has occurred. In the case of the word input experiment, the 'braille keyboard' consists of 11 phonemes, and it was judged that the word input was successful only when 11 phonemes were perfectly input. The word input experiment has lower accuracy than the phoneme input experiment. When comparing the accuracy test of each type hardware, the button type hardware has higher overall accuracy. The button has the advantage that the user can recognize it with a feeling and sound when pressed, so it shows high accuracy when inputting. On the other hand, the FSR sensor has no sound or feeling when input, making it difficult to check whether the input has been made, and it is difficult to measure the pressure strength, so relatively low accuracy was confirmed.

The implemented system input speed comparison experiment is as follows.

In order to compare the input speed of the button-type hardware and the FSR sensor-type hardware, the required time was measured in the experiment of inputting the 'braille keyboard' 50 times for the evaluation of the braille judgment algorithm. As a result of the input speed comparison experiment, the button-type hardware has an average speed of 32 minutes and 39 seconds, and the FSR sensor-type hardware has an average speed of 27 minutes and 8 seconds. In the case of a button, the speed is slow due to the click, but the input accuracy is high, and the advantage is that the user can recognize the input by the sound generated when the button is pressed. On the other hand, the FSR sensor has the advantage of high speed, but there is no feeling when pressing the sensor during input, and the input accuracy is low because it is difficult to estimate the pressure strength. Table 3 shows the time required for each type of hardware input speed experiment.

Sub. Button(min) FSR Sensor(min) One 32.47 27.36 2 33.38 26.57 3 32.43 27.50 4 31.55 26.42 5 32.12 26.37 Avg. 32.39 27.08

Therefore, research related to the existing software and Braille keyboard has limitations such as memorizing the keypad position and input speed, and placing it on an object when using it. In the present invention, a braille keyboard with six input keys and two function keys attached to a smartphone case is implemented, and it can be used while holding a smartphone and provides convenience of input through the braille method. was supplemented. The implemented keyboard uses two types of buttons and FSR sensors, and outputs the characters input through the braille classification algorithm to the application, and it is possible to confirm the written characters by using the TTS function. In order to compare the accuracy and input speed of the implemented system, an experiment was conducted to measure the speed when inputting random phonemes 100 times, 'braille keyboard' 50 times consecutively, and 'braille keyboard' input. As a result, the button-type hardware confirmed the detection performance of 98.6% in the phoneme input experiment and 95.6% in the word input experiment, and was measured at an average speed of 32 minutes and 39 seconds in the word input experiment. In the case of FSR sensor type hardware, the accuracy is 88.2% and 58%, respectively, and the input speed is 27 minutes and 8 seconds. When comparing the experimental results, the high-accuracy button-type hardware is suitable as a braille keyboard, but in the future, by improving the FSR sensor-type hardware through real-time voice output and smartphone vibration functions, a braille keyboard with high accuracy and fast input speed will be developed. can be implemented

Claims (3)

Braille keyboard installed on the back of the smartphone;
an input unit transmitting an input signal of the input key to a control unit;
text-to-speech conversion unit;
audio output unit;
storage to store data
And a smart phone case-based braille keyboard system, characterized in that it comprises a control unit According to claim 1, wherein the input keys of the braille keyboard is composed of eight input keys in the form of an array, the six input keys are arranged in the form of Braille, and function to input characters, and the remaining two input keys are spaced and deleted. , a smart phone case-based braille keyboard system, characterized in that it is a function key responsible for the transmission and line change functions and the role of converting the input text into speech The braille keyboard system according to claim 2, wherein the keyboard is in the form of a button or a form using an FSR sensor.

Images