CN104306102A - Head wearing type vision auxiliary system for patients with vision disorder - Google Patents

Abstract

The invention provides a head-mounted visual aid system for visually impaired patients, including: a head-mounted glasses bracket for installing and connecting various components of the system and a user-friendly head-mounted glasses bracket; respectively used for displaying left and right eye images The left and right eye display panels; the two-way image acquisition device that acquires images from the perspective of human eyes; the sensors used to detect the external environment and check the state of the glasses; used for the overall control of the system, image acquisition, processing and projection Display, acceptance and processing of sensor data, and a data processing center for the implementation of related image processing algorithms; supporting image processing programs designed for visually impaired patients to enhance their vision, as well as training and rehabilitation designed to treat different visual diseases program. The system of the present invention can collect real scenes in real time, process the collected images, and display them in a manner more acceptable to visually impaired patients, so as to achieve the purpose of assisting vision.

Description

Head-mounted visual aid system for visually impaired patients

technical field

The invention relates to the technical field of visual aids, in particular to a head-mounted visual aid system for visually impaired patients.

Background technique

Visual impairment, also known as "visual disability", "visual defect", and "visual impairment", refers to partial or complete impairment of the structure or function of the visual organs (including eyeballs, ophthalmic nerves) and the optic nerve center of the brain due to various reasons. It is manifested by different degrees of visual impairment or narrowing of vision in both eyes, making it difficult to perform work, study and other activities that ordinary people can engage in, thus limiting or hindering normal functions that can be performed according to their age, gender, social and cultural conditions.

According to the estimates of the World Health Organization, there are 40 million to 45 million blind people in the world, and the low-vision population is three times that of the blind people, about 140 million people. Among the more than 83 million disabled people in my country, there are 12.33 million visually disabled people, about 5 million blind people, and about 7.1 million low-vision people. China is one of the countries with the largest population of blind people and low-vision people in the world. How to solve their normal study, work and life problems has become the focus of social attention.

Visual impairment is the partial or total loss of function of the visual system due to congenital or acquired reasons, so that it is impossible to accurately identify external objects. According to the relevant regulations, those whose vision does not reach 0.3, or whose field of vision is within 20°, are considered to be visually impaired. According to the different degrees of impairment, visual impairment can be divided into two categories: amblyopia if the measured value of visual acuity in both eyes is above 0.03 but less than 0.3 or its visual field is within 20°; total blindness is defined as the measured value of visual acuity in both eyes is less than 0.03. People with amblyopia can usually use their vision to carry out daily life, but they have certain obstacles in seeing things, and generally need to use optical aids or special magnifying devices to observe things. But amblyopia itself is very different: there are those who lack stereoscopic vision, those who are sensitive to light, those who cannot distinguish colors, and those whose field of vision is too narrow. Therefore, the optical auxiliary equipment or special magnification equipment required is also very different. People who are completely blind cannot rely on vision for daily life at all, and usually can only use alternative methods to carry out daily life. For example, replacing vision with tactile aids, auditory aids, and other types of aids. It can be said that assistive technology and equipment are the most needed by the visually impaired. Most information accessibility technologies and devices can be used by people with visual impairments. Almost all visually disabled people can improve their quality of life to varying degrees and participate in social life through the compensation function of assistive devices. However, the second sample survey shows that only 7.97% of the visually disabled in my country receive assistive device services; 14.49% of them are urban and 5.82% are rural.

Assistive devices for visual impairment can be divided into two categories: visual and non-visual devices:

Visual aids refer to equipment or devices that can help visually impaired people improve their visual abilities through visual means, and are divided into three categories: optical, non-optical, and electronic vision aids. Optical vision aids are relatively common, including handheld and standing magnifying glasses, glasses, monoculars, and binoculars. Optical vision aids are easy to use, easy to carry, and low in price. They are also the main vision aids used by low-vision patients; but they also have some limitations in use, such as low magnification, small reading range, and long-term use. Causes poor posture, will be affected by the light source, etc. Non-optical vision aids refer to improving visual effects by improving the surrounding environment. Such as: enlarged printed matter to change the size of the reading material, high-intensity illuminators to increase contrast, high-contrast objects to facilitate the distinction of local items, etc. The electronic vision aid is an electronic video device, which can transmit the read documents, pictures, observed objects, etc. through the camera lens to the screen for users to browse. There are many types of electronic vision aids, including: far-seeing, near-seeing (magnifying), dual-purpose near-far, pocket-type, portable, etc. It is characterized by easy adjustment of magnification, brightness, contrast, etc., and can avoid glare; there are various display modes such as reverse white, half-color, etc.; it is ergonomically designed and can provide a large working space, etc.

Non-visual aids refer to assistive devices that make full use of functions other than vision, such as hearing and tactile functions, to make up for the defects of visual functions and improve the quality of life of the visually impaired, involving the life, work, study, entertainment, etc. of the visually impaired. all aspects. For example: mobile phones, telephones, and clocks for the blind used in daily life; blind sticks and roads for the blind; braille, dot displays, audiobooks, and various screen reading software for learning and communicating. This type of assistive device is mostly used by blind patients.

To sum up, different people with visual impairments need different assistive technologies and equipment. The existing miscellaneous visual impairment aids have the characteristics of single function and poor scalability, and do not meet the principles of ease of use, applicability, simplicity, portability, and scalability.

Contents of the invention

Aiming at the defects in the prior art, the object of the present invention is to provide a head-mounted visual aid system for visually impaired patients, which can collect real-world scenes in real time, process the collected images and use visually impaired patients to improve It can be displayed in an acceptable way to improve the ability of visually impaired patients to obtain visual information and achieve the purpose of assisting vision.

In order to achieve the above objectives, the present invention provides a head-mounted visual aid system for visually impaired patients, including:

The head-mounted glasses bracket is used to install and connect various components of the system and is convenient for users to wear;

A display panel, embedded in the frame of the head-mounted glasses holder, for displaying images;

The image acquisition device is arranged on the head-mounted glasses support;

Multiple sensors are used to detect external information and the state of the glasses and transmit the signals to the data processing center;

Data processing center, the data processing center is used for the overall control of the system, including the image acquisition and processing of the image acquisition equipment and the display of the display panel, the acceptance and processing of sensor data; the image processing is realized by the image processing module, the The image processing module collects images from the image acquisition equipment in real time, and processes the collected images. For different types of patients with visual impairment, it uses image processing and machine vision algorithms to process the images, and converts the processed images into visually impaired patients. Real-time display in a form more acceptable to patients, so as to achieve the purpose of visual assistance.

Preferably, the display of the system is a binocular display or a single-eye display; the image acquisition device is a dual-channel acquisition or a single-channel acquisition; if it is a single-eye or a single-channel, only one channel is required for the display panel and the image acquisition device.

Further, if it is binocular and dual-channel, the display panel and the image acquisition device are left and right, that is, the display panel includes:

A display panel for the left eye, embedded in the left eyeglass frame of the head-mounted glasses bracket, for displaying images for the left eye;

The right-eye display panel is embedded on the right eyeglass frame of the head-mounted glasses holder, and is used for displaying images for the right eye;

The two-way image acquisition device is composed of two miniature cameras, which are distributed on the left and right sides of the head-mounted glasses bracket.

Preferably, the display mode of the system is projection type or non-projection type; if projection type display is adopted, the system also includes a projection device, which is projected on the display panel to form a virtual image at a certain distance in front of the human eye; The projection device is one channel or two channels on the left and right; if a non-projection display mode is adopted, the image is directly displayed on the screen of the display panel.

More preferably, the display panel in the non-projection display mode can be a high-definition liquid crystal display or an LED dot matrix display, and the selection of the display can be adjusted according to the actual condition of the patient.

More preferably, if the display panel adopts the transmissive display mode, the image projected on the display panel is superimposed with the real scene to achieve the effect of visual improvement.

Preferably, the display of the system can be binocular or monocular; the image acquisition device can be dual-channel acquisition or single-channel acquisition.

Preferably, the two-way image acquisition device acquires the real scenes seen by the left and right eyes from the perspective of human eyes, and projects them on the left and right display panels in real time after being processed by the data processing center.

Preferably, the image processing algorithm of the data processing center has different settings according to different required functions; different image processing algorithms correspond to different patients.

Compared with the prior art, the present invention has the following beneficial effects:

The invention is a wearable device, which has the advantages of being simple, portable, and easy to expand, and can be used by visually impaired patients in daily life; at the same time, an intelligent controller is adopted, which can have other functions of the intelligent device in addition to the visual assistance function In addition, the hardware disclosed in the present invention can be used as a platform for aids for visual impairments. For different types and degrees of visual impairments, matching image processing programs can be designed in a targeted manner to achieve the purpose of assisting and enhancing vision. At the same time, for different types and degrees of visual impairments, matching programs can be designed specifically for the treatment and rehabilitation training of corresponding visual impairments.

Description of drawings

Other characteristics, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

Fig. 1 is the structural representation of a preferred embodiment of the present invention;

Fig. 2 is a working principle block diagram of a preferred embodiment of the present invention;

Fig. 3 is an effect diagram of an embodiment of the present invention taking amblyopia as an example; wherein: figure (a) is a real scene picture collected by a camera, and figure (b) is a projected and displayed picture after image processing (image enlargement).

In the figure: 1 is the head-mounted glasses holder, 2 is the left eye glasses lens, 3 is the left eye display panel, 4 is the left eye projection device; 5 is the right eye glasses lens, 6 is the right eye display panel, 7 is the right eye Projection device, 8 is a dual-channel image acquisition device, 9 is a data processing center, 10 is a picture of a real scene collected by the dual-channel image acquisition device, 11 is a processed projected and displayed picture, and 12 and 13 are sensors.

Detailed ways

The present invention will be described in detail below in conjunction with specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

As shown in Figure 1, it is a schematic structural diagram of a head-mounted visual aid system for visually impaired patients according to a preferred embodiment of the present invention. In the figure: a head-mounted glasses bracket 1, a left-eye glasses lens 2, and a left-eye display panel 3 , left eye projection device 4, right eye glasses lens 5, right eye display panel 6, right eye projection device 7, two-way image acquisition equipment 8, data processing center 9, sensors 12, 13, wherein:

The head-mounted glasses bracket 1 is used for installing, connecting and fixing various components of the system, and is convenient for users to wear;

The left-eye spectacle lens 2 and the right-eye spectacle lens 5 are used to respectively fix the left-eye display panel 3 and the right-eye display panel 6, and the left-eye spectacle lens 2 and the right-eye spectacle lens 5 are permeable , the user can see the real scene through the lens in addition to seeing the projected display screen;

For the left-eye display panel 3 and the right-eye display panel 6, if the projection display method is adopted, the projection device is embedded in the transparent spectacle lens at a certain angle and distance, and the display panel is also permeable. The virtual image displayed by the projection and the display scene seen through the lens are superimposed; if the non-projection display method is used, the displayed image is a real image;

The left-eye projection device 4 and the right-eye projection device 7 cooperate with the left-eye display panel 3 and the right-eye display panel 6 respectively, and project images to the left-eye display panel 3 and the right-eye display panel 6 respectively. A virtual image is formed in front of the human eye;

The two-way image acquisition device 8 is two cameras, and the two cameras are installed on the left and right sides of the head-mounted glasses bracket 1 respectively, so that the real scenes seen by the left and right eyes can be collected in real time from the perspective of the human eye and displayed. transmitted to the data processing center 9, after being processed by the data processing center 9, they are respectively projected onto the left-eye display panel 3 and the right-eye display panel 6 in real time to form a virtual scene; Two-way image acquisition is performed from the perspective of the human eye, and the collected left and right eye images are processed and projected to the left eye display panel 3 and the right eye display panel 6 respectively for display, so the virtual scene seen by the human eye is a 3D scene;

The data processing center 9 is used for the overall control of the system, image collection, processing and projection display; the data processing center 9 is used for the overall control of the system and the driving of each module.

In this embodiment, the data processing center 9 may be existing data processing equipment on the market, such as smart phones, portable computers, etc., or a self-developed data processing terminal, and the image processing algorithm therein may be based on the required There are different settings for different functions. For example, in addition to the image correction and calibration tasks for color-blind patients, the data processing center 9 can also have other functions possessed by smart devices.

In this embodiment, the data processing center 9 may be connected to the glasses through a data cable, or may be directly connected to the glasses in a wireless manner.

In this embodiment, the sensors 12 and 13 are used to detect external information and the state of the glasses and transmit the signal to the data processing center; including infrared sensors, accelerometers, gyroscopes and other sensors, which will collect The data is transmitted to the data processing center 9, and the image processing and machine vision algorithms will use the data for analysis. Among them, the infrared sensor is used to collect infrared signals for possible night vision detection or obstacle distance detection; the accelerometer and gyroscope respectively collect the acceleration and angle of the glasses in various directions. By analyzing the data collected by these sensors, the data processing center can make corresponding processing on the images to be displayed.

The main function of the system described in this embodiment is to provide a visual aid system for visually impaired patients. This system can not only have different visual enhancement effects for different visual diseases, but also allow patients to rely on the aid of the instrument to improve their vision in life. Moreover, different correction and treatment programs can be designed symptomatically in the data processing center, so that patients can improve their own visual ability during the practice.

In addition to the physical components described above, the system described in this embodiment also includes a matching image processing module designed for visually impaired patients. This module collects images in real time from the dual-channel image acquisition device 8, transmits them to the data processing center 9, and uses the image processing module to process the collected images. For different visually impaired patients, the images are processed using corresponding image processing and machine vision algorithms. After processing, the images that are more intuitive and acceptable to visually impaired patients are projected and displayed. Visually impaired patients can watch the processed images through this system, so as to achieve the purpose of assisting vision. specific:

For patients with night blindness, increase the night vision of patients by improving contrast sensitivity;

For patients with age-related macular degeneration, central serous retinopathy, cataract and other metamorphopsia, patients can produce normal images through image deformation. The lines are all straight lines (in fact, normal people see curved lines after adjustment), and then through image processing, the images captured by the camera will produce the same deformation;

For patients with red-green color blindness, let the computer change red and green into colors that patients can distinguish, such as yellow and blue;

For low-vision patients such as high myopia, the computer can enlarge the image for easy reading;

For moving objects, increase the contrast and sound prompts to make the patient alert, etc.;

For visual impairment caused by other diseases, use the corresponding algorithm to improve or treat the patient's vision.

There are other patients and the corresponding image processing methods are not listed here, and changing the image processing algorithm does not affect the essence of the system. The image processing algorithm can adopt the existing technology, and also can adopt the innovative algorithm to adapt to the visual aid system.

As shown in Figure 2, it is a block diagram of the working principle of the system, and the workflow includes the following steps:

1. Image acquisition

This part is completed by the two cameras of the dual-channel image acquisition device 8. The left and right cameras collect images of real scenes from the perspective of human eyes respectively, and the real scene pictures 10 collected by the cameras are transmitted to the data processing in the form of video streams. Center 9;

2. Image processing

This part is completed by the data processing center 9 of the system, and the data processing center 9 utilizes relevant image processing algorithms to carry out specific processing on the images collected from the two-way image acquisition device 8, such as: enlargement, enhancement, color correction, etc.; this step It is necessary to use image processing and machine vision related algorithms; this system has high requirements on the performance of related image processing algorithms, and it is required to meet real-time requirements. Therefore, real-time issues must be considered when designing image processing algorithms, and cannot be used Algorithms that are too complex to meet real-time requirements; if there are some more complex algorithms, corresponding simplifications or innovations should be made for the system to make the algorithms better match the system. The image processing algorithm can adopt the existing technology, and also can adopt the innovative algorithm to adapt to the visual aid system. The images collected by the dual-channel image acquisition device 8 are processed in real time, and then transmitted to the left-eye display panel 3 and the right-eye display panel 6;

3. Projection display (for transmissive display method) or display by display screen (for non-projection display method),

The display mode of the system of the present invention can be projection type or non-projection type. If a projection type display is adopted, wherein the left and right eye projection devices are respectively projected on the left and right display panels 3, 6 to form a virtual image at a certain distance in front of the human eyes; A convex lens is added between the glasses and the left and right display panels 3 and 6 to make the imaging clearer. The display panel of the non-projection display can be a high-definition liquid crystal display or an LED dot matrix display, and the choice of the display can be adjusted according to the actual situation of the patient. This does not affect the essence of the system. specific:

1) Projection display (for transmissive display method)

This part is completed by the left and right eye projection devices 4, 7 and the left and right eye display panels 3, 6. The left and right eye projection devices 4, 7 transmit the color-blindness image correction and calibration of the left and right images transmitted from the data processing center 9. , the right-eye images are projected to the left and right eye display panels 3 and 6 respectively, so that the images seen by the user from the left and right eye display panels 3 and 6 are images processed by image processing and machine vision-related algorithms;

2) Use the display screen to display (for non-projection display methods)

This part is completed by the left and right display panels 3 and 6 display pictures, and the left and right eye display panels 3 and 6 respectively display the left and right eye pictures after color-blind image correction and calibration transmitted by the data processing center 9, so that the user The pictures seen from the left and right display panels 3 and 6 are images processed by image processing and machine vision related algorithms.

As shown in Figure 3, it is an effect diagram taking amblyopia as an example, wherein: Figure (a) is a real scene picture 10 collected by a dual-channel image acquisition device, and Figure (b) is a projection after related image processing (image enlargement) Picture 11 shown.

Compared with traditional assistive devices for visual impairment, the present invention is a wearable device, which has the advantages of simplicity, portability, and easy expansion, and can be used by visually impaired patients in daily life; in addition, the hardware disclosed in the present invention can be used as a visual aid For different types and different degrees of visual impairment, the platform of obstacle aids can design matching image processing programs and treatment programs in a targeted manner to achieve the purpose of assisting vision and treating diseases.

Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims (10)

1. A head-mounted visual aid system for visually impaired patients, comprising: The head-mounted glasses bracket is used to install and connect various components of the system and is convenient for users to wear; A display panel, embedded in the frame of the head-mounted glasses holder, for displaying images; The image acquisition device is arranged on the head-mounted glasses support; Multiple sensors are used to detect external information and the state of the glasses and transmit the signals to the data processing center; Data processing center, the data processing center is used for the overall control of the system, including the image acquisition and processing of the image acquisition equipment and the display of the display panel, the acceptance and processing of sensor data; the image processing is realized by the image processing module, the The image processing module collects images from the image acquisition equipment in real time, and processes the collected images. For different types of patients with visual impairment, it uses image processing and machine vision algorithms to process the images, and converts the processed images into visually impaired patients. Real-time display in a form more acceptable to patients, so as to achieve the purpose of visual assistance. 2. A head-mounted visual aid system for visually impaired patients according to claim 1, wherein the display of the system is a binocular display or a monocular display; the image acquisition device is a dual-channel acquisition or a single Channel acquisition; if it is a single eye and a single channel, only one channel is required for the display panel and the image acquisition device. 3. A head-mounted visual aid system for visually impaired patients according to claim 2, characterized in that, if it is binocular and dual-channel, the display panel and the image acquisition device are left and right, namely: The display panel includes: A display panel for the left eye, embedded in the left eyeglass frame of the head-mounted glasses bracket, for displaying images for the left eye; The right-eye display panel is embedded on the right eyeglass frame of the head-mounted glasses holder, and is used for displaying images for the right eye; The two-way image acquisition device is composed of two miniature cameras, which are distributed on the left and right sides of the head-mounted glasses bracket. 4. A head-mounted visual aid system for visually impaired patients according to claim 1, wherein the display mode of the system is projection type or non-projection type; If a projection display is adopted, the system also includes a projection device, which is projected on the display panel to form a virtual image at a certain distance in front of the human eye; the projection device is one or two left and right; If a non-projection display method is adopted, the image is directly displayed on the screen of the display panel. 5. A head-mounted visual aid system for visually impaired patients according to claim 4, wherein if a non-projection display method is adopted, a convex lens is further added between the eyes and the display panel to make the imaging clearer. 6 . The head-mounted visual aid system for visually impaired patients according to claim 5 , wherein the display panel in the non-projection display mode is a high-definition liquid crystal display or an LED dot matrix display. 7. A head-mounted visual aid system for visually impaired patients according to claim 4, wherein the display panel is penetrable, and the image formed on the display panel is superimposed with the real scene To achieve the effect of visual improvement. 8. A head-mounted visual aid system for visually impaired patients according to claim 2, wherein the two-way image acquisition device collects the real scene seen by the left and right eyes respectively from the perspective of human eyes , which are processed by the data processing center and then projected onto the left and right display panels in real time. 9. A head-mounted visual aid system for visually impaired patients according to any one of claims 1-8, wherein the image processing algorithm of the data processing center varies according to the required functions. There are different image processing algorithms corresponding to different patients. 10. A head-mounted visual aid system for visually impaired patients according to claim 9, characterized in that, for patients with night blindness, by improving contrast sensitivity, the patient's night vision is increased; For age-related macular degeneration, central serous retinopathy, cataract and other patients with metamorphopsia, through image deformation, patients can produce normal images; For patients with red-green color blindness, let the computer change red and green into colors that patients can distinguish; For patients with low vision, the computer can enlarge the image for easy reading; For moving objects, increase the contrast and sound prompts to alert the patient; For visual impairment caused by other diseases, use the corresponding algorithm to improve or treat the patient's vision.