CN111571619A - Life assisting system and method based on SSVEP brain-controlled mechanical arm grabbing - Google Patents

Abstract

The invention discloses a life assistance system and method based on SSVEP brain-controlled mechanical arm grasping. The system includes an EEG cap, a computer, an intention expression interface, an SSVEP decoding unit and a mechanical arm. The method of the invention is as follows: firstly, a professional helps the user to put on the EEG cap, and secondly, the intention expression interface on the computer terminal is opened, and when the intention expression interface flashes stimuli, the user pays attention to the corresponding area on the intention expression interface according to his own needs. block, the EEG cap collects the EEG signal of the user at this time, transmits it to the SSVEP decoding unit on the computer side, decodes it through preprocessing and FBCCA algorithm, and converts the decoding result into a control command and sends it to the robot arm, and the robot arm executes the control command. Plan the path to grab the corresponding items, so as to achieve the purpose of assisting the user's daily life, especially for the elderly and the disabled to provide more convenience and safety in their lives.

Description

A living assistance system and method based on SSVEP brain-controlled robotic arm grasping

technical field

The present invention designs a life assistance system and method based on SSVEP (Steady-State Visual Evoked Potentials, Steady-State Visual Evoked Potentials) brain-controlled robotic arm grasping for the elderly and the disabled. The system includes an EEG cap, a computer, an intention The expression interface, the SSVEP decoding unit and the robotic arm are used to assist the elderly and the disabled to grasp the required items independently, assist the daily life, solve the problem of inconvenience in life, and improve the quality of life.

Background technique

According to a 2019 report, there are 85 million disabled people in China, and 44 million disabled and semi-disabled elderly people. Many of them need assistive devices. Assistive devices refer to products that compensate and replace human body functions, and are means to help the disabled and the elderly improve their quality of life and enhance their ability to participate in society. There are a large number of disabled people in China, and the obstacles caused by disability can lead to various difficulties in the lives of disabled people, including poorer health status, lower educational acceptance, poorer economic participation, higher poverty rates and higher dependencies, etc. Participating in normal life has become the biggest challenge, especially for special groups with language and communication barriers and loss of physical activity. Therefore, it is very necessary to develop a brain-controlled robotic arm system that can assist the elderly or disabled people in their daily lives, especially for people with physical disabilities, to improve their quality of life and active participation, and to solve the problem of inconvenience in life. Technical problems to be solved urgently.

SUMMARY OF THE INVENTION

In order to solve the problems of the prior art, the purpose of the present invention is to overcome the deficiencies of the prior art, and to provide a life assistance system and method based on the grasping of the SSVEP brain-controlled robotic arm. Blocks represent specific items, and these blocks flash at different frequencies. According to their own needs, that is, the item they want to grab, the user looks at the corresponding block representing the item on the intent expression interface, and the EEG cap collects this item. When the user's EEG signal is transmitted to the SSVEP decoding unit on the computer side by wired or wireless means, filtering and FBCCA filter bank canonical correlation analysis are performed. Decode the frequency corresponding to the user's brain potential induced by visual stimuli, find the block corresponding to the intention expression interface through this frequency, and then know the item the user wants to grab, and then the computer sends instructions to control the robotic arm to grab the corresponding item, so that That is, the operation of assisting the elderly and the handicapped to grab the required items is realized.

In order to achieve the above-mentioned purpose of invention and creation, the present invention adopts the following technical solutions:

A life assistance system based on SSVEP brain-controlled robotic arm grasping, comprising an EEG cap, a computer, an intent expression interface, a SSVEP decoding unit and a robotic arm, the EEG cap is connected to a computer by wired or wireless means, and the computer contains an intent Expression interface and SSVEP decoding unit, the robotic arm is connected to the computer through a wired connection; a professional helps the user to wear the EEG cap, open the intention expression interface on the computer side, and when the intention expression interface flashes stimuli, the user can follow his own needs. , focus on the corresponding block on the intent expression interface, the EEG cap collects the user's EEG signal at this time, and transmits it to the SSVEP decoding unit on the computer side, which is decoded by the FBCCA algorithm through preprocessing and filter bank canonical correlation analysis, and the decoding result is converted. The completed control command is sent to the robotic arm, and the robotic arm executes the control command and grabs the corresponding item according to the planned path.

As a preferred technical solution of the present invention, the EEG cap adopts the 24-lead dry electrode cap of Boruikang, and the user can collect EEG signals without applying EEG cream on the scalp. Connect with the computer through wired serial communication, or connect with the computer through wireless bluetooth; the computer includes an intent expression interface and an SSVEP decoding unit; the robotic arm adopts the Kinova 6-axis bionic robotic arm, and the robotic arm is wired with the computer.

As a preferred technical solution of the present invention, the intention expression interface includes 12 blocks, and the 12 blocks represent 12 instructions: apple 1, banana 1, green tea, orange, cola, apple 2, banana 2, mineral water, orange , Qixi, Cancel and Confirm; the user first looks at the desired item, and then looks at the cancel or confirm block. After the confirmation block is activated, the robotic arm executes the action of grabbing the item.

As a preferred technical solution of the present invention, the SSVEP decoding unit first preprocesses the user's EEG signal, that is, filtering, and then uses the FBCCA filter bank canonical correlation analysis, and finally decodes and identifies the command that the user wants to send, Thus, the user's brain controls the robotic arm to grab the corresponding item.

As a preferred technical solution of the present invention, the manipulator performs path planning through the principle of forward and inverse kinematics, and adopts continuous trajectory planning to avoid the stuck phenomenon.

A life assistance method based on the grasping of the SSVEP brain-controlled robotic arm, using the living assistance system of the present invention based on the grasping of the SSVEP brain-controlled robotic arm to operate, and the operation steps are as follows:

1) Professionals help users wear EEG caps;

2) Open the intent expression interface;

3) When a stimulus flickers occurs on the intention expression interface, the user looks at the corresponding block according to his own needs; at the same time, the EEG cap collects the EEG signal and transmits it to the SSVEP decoding unit on the computer side by wired or wireless means;

4) The SSVEP decoding unit is decoded by the FBCCA algorithm;

5) After the SSVEP decoding unit is decoded by the FBCCA algorithm, it sends a control command to the robotic arm;

6) The robotic arm grabs the corresponding item according to the planned path.

As a preferred technical solution of the present invention, in the step 2), the designed intention expression interface includes 12 blocks, 12 blocks represent 12 instructions, apple 1, banana 1, green tea, orange, cola, apple 2 , Banana 2, Mineral Water, Orange, Seven Up, Cancel and Confirm.

As a preferred technical solution of the present invention, in the step 3), the user first looks at the block corresponding to the desired item in the intention expression interface, and then looks at the cancel or confirmation block. During fixation, the EEG signal is transmitted to the SSVEP decoding unit on the computer side for processing by wired or wireless means.

As a preferred technical solution of the present invention, in the step 4), the SSVEP decoding unit preprocesses or filters the EEG signal of the user, performs a typical correlation analysis of the FBCCA filter bank, and decodes and identifies the command that the user wants to send. , so as to realize the operation of the user's brain to control the robotic arm to grab the corresponding item.

As the preferred technical solution of the present invention, in the described step 5), the FBCCA algorithm realizes the EEG signal decoding steps as follows:

5-1) carry out filter bank analysis, decompose the SSVEP EEG signal by a plurality of different passbands of the filter, obtain the subband signal after each subband of the filter;

5-2) Correlation analysis is carried out with each subband component obtained by filtering and the standard sine and cosine reference signal;

5-3) The frequency corresponding to the maximum correlation is the identification result; according to the algorithm identification result, after the confirmation block is activated, the control command is sent to the robotic arm, and the robotic arm executes the action of grabbing the item.

Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant advantages:

1. The present invention adopts the brain-computer interface technology to establish a channel between the user's brain and a computer or other electronic equipment, and does not rely on peripheral nerves and muscle tissue to realize brain control of external equipment; SSVEP steady-state visual evoked potential is a brain-computer A kind of interface technology, which induces the brain to generate different frequency potentials through visual stimulation of different frequencies, uses the EEG cap to collect and process these EEG signals, decodes the corresponding visual stimulation frequency, decodes the user's intention, and then outputs the control The command controls the movement of the robotic arm, that is, the user controls the operation of the robotic arm through the brain;

2. The present invention is easy to use and does not require tedious scalp massage; a more user-friendly interactive interface enables users to express their intentions clearly; expands the robotic arm for grabbing items, assisting the elderly and the disabled when unmanned. Grab the items you need without help;

3. The system of the present invention is easy to expand, realizes more functions, simple operation method and low cost.

Description of drawings

FIG. 1 is a block diagram of the system structure of the present invention.

Fig. 2 is the overall experiment flow chart of the present invention.

FIG. 3 is a flow chart of a computer terminal program of the present invention.

Figure 4 is an intent expression interface stimulus screen of the present invention.

FIG. 5 is an intent expression interface feedback screen of the present invention.

Detailed ways

The above scheme will be further described below in conjunction with specific embodiments, and preferred embodiments of the present invention are described in detail as follows:

Example 1:

In this embodiment, as shown in FIG. 1 , a life assistance system based on SSVEP brain-controlled robotic arm grasping includes an EEG cap 1 , a computer 2 , an intention expression interface 3 , an SSVEP decoding unit 4 and a robotic arm 5 , The EEG cap 1 is connected to the computer 2 through wired or wireless means, the computer 2 includes an intention expression interface 3 and an SSVEP decoding unit 4, and the robotic arm 5 is connected to the computer 2 through a wired connection; a professional helps the user to wear the EEG. Cap 1, open the intent expression interface 3 on the computer 2 side, when the intent expression interface 3 flickers, the user can focus on the corresponding block on the intent expression interface 3 according to his own needs, and the EEG cap 1 collects the user’s current The EEG signal is transmitted to the SSVEP decoding unit 4 of the computer 2, and is decoded by the FBCCA algorithm through preprocessing and filter bank canonical correlation analysis. The decoding result is converted into a control command and sent to the robotic arm 5. The robotic arm 5 executes the control command, according to the plan. The path grabs the corresponding item.

Embodiment 2:

This embodiment is basically the same as the first embodiment, and the special features are:

In this embodiment, the intention expression interface 3 includes 12 blocks, and the 12 blocks represent 12 instructions: apple 1, banana 1, green tea, orange, cola, apple 2, banana 2, mineral water, orange, Seven-Up, Cancel and Confirm; the user first looks at the desired item, and then looks at the cancel or confirm block. After the confirmation block is activated, the robotic arm executes the action of grabbing the item.

The SSVEP decoding unit 4 first preprocesses the user's EEG signal, that is, filtering, and then uses the FBCCA filter bank canonical correlation analysis, and finally decodes and identifies the command that the user wants to send, thereby realizing the user's brain to control the machine. The arm 5 grabs the corresponding item for operation.

The robotic arm 5 performs path planning based on the principle of forward and inverse kinematics, and adopts continuous trajectory planning to avoid the stuck phenomenon.

Embodiment 3

This embodiment is basically the same as the previous embodiment, and the special features are:

In this embodiment, as shown in FIG. 2 , a life assistance method based on SSVEP brain-controlled robotic arm grasping uses the living assistance system based on SSVEP brain-controlled robotic arm grasping described in the previous embodiment to operate. The operation steps as follows:

1) Cap 1 collects the EEG signal, and transmits it to the SSVEP decoding unit 4 of the computer 2 through wired or wireless means;

4) SSVEP decoding unit 4 decodes by FBCCA algorithm;

5) After the SSVEP decoding unit 4 is decoded by the FBCCA algorithm, it sends a control command to the robotic arm 5;

6) The robotic arm 5 grabs the corresponding item according to the planned path.

Embodiment 4

This embodiment is basically the same as the previous embodiment, and the special features are:

In this embodiment, as shown in FIG. 1 , a life assistance system and method based on SSVEP brain-controlled robotic arm grasping includes an EEG cap, a computer, an intent expression interface, an SSVEP decoding unit, and a robotic arm. The EEG cap adopts the 24-lead dry electrode cap of Boruikang, and the user can collect EEG signals without applying EEG cream on the scalp. The dry electrode cap amplifies the collected microvolt-level EEG signals, and converts the EEG analog signals into digital signals through A/D analog-to-digital conversion, with a sampling rate of 250Hz. It can be connected to a computer through wired serial communication, or can be connected to a computer through wireless Bluetooth; the computer includes an intention expression interface and an SSVEP decoding unit; the intention expression interface is used for user interaction; the SSVEP decoding unit is used for user interaction. The EEG signal is preprocessed, that is, filtered, and the FBCCA filter bank can be used for typical correlation analysis to decode and identify the instructions that the user wants to send; the robotic arm adopts Kinova 6-axis bionic robotic arm, and the robotic arm is connected to the computer by wire; The above operation steps are as follows: firstly, a professional helps the user to wear the EEG cap, and secondly, open the intention expression interface on the computer side. When the intention expression interface flashes stimuli, the user pays attention to the corresponding area on the intention expression interface according to his own needs. block, the EEG cap collects the user's EEG signal at this time, and transmits it to the SSVEP decoding unit on the computer for decoding through the FBCCA algorithm. The decoding result is converted into a control command and sent to the robotic arm. The robotic arm executes the control command to grab the corresponding item.

As shown in Figure 2, the experimental process of a living assistance system and method based on SSVEP brain-controlled robotic arm grasping: first, a professional helps the user to wear the 24-lead dry electrode cap of Boruikang, and secondly, the intention to open the computer terminal In the expression interface, when the stimulus flickers appears on the intention expression interface, the user can focus on the corresponding block on the intention expression interface according to their own needs. The SSVEP decoding unit on the computer performs decoding. After the decoding is successful, it sends control commands to the Kinova 6-axis bionic robotic arm, and the robotic arm executes the control commands to grab the corresponding items.

As shown in Figure 3, a computer-side program flow of a living assistance system and method based on SSVEP brain-controlled robotic arm grasping, the computer-side program is divided into two parts: the SSVEP decoding unit and the intention expression interface. Here, the SSVEP decoding unit is called the processor, and the intent expression interface is called the stimulator. The EEG signal is collected by the EEG cap and sent to the processor (SSVEP decoding unit) on the computer side. The processor and the stimulator communicate through TCP/IP to exchange data.

Processor (SSVEP decoding unit) program flow:

1) Initialize the EEG parameters, EEG data acquisition frequency, the required data channel extraction, and the time required for data processing.

2) Initialize the communication interface of the manipulator, and establish communication with the manipulator, which is convenient for sending control information later.

3) Establish TCP/IP communication to provide a transmission data channel for the stimulator.

4) Receive the EEG signal of the EEG cap, perform filtering and FBCCA filter group canonical correlation analysis, and decode the frequency of the block that the user is watching.

5) Send the processed data to the stimulator.

6) Send the processed data to the robotic arm, and the robotic arm grabs the item.

Stimulator (intent expression interface) program flow:

1) Connect the TCP/IP generated by the processor program to receive the data processed by the processor

2) Initialize stimulation paradigm interface parameters, interface size, interface frequency parameters

3) Draw the stimulus paradigm interface, that is, draw the interface that requires the robotic arm to grasp the object

4) The stimulation paradigm interface generates stimulation signals

5) Receive the data processed by the processor

6) The stimulus interface displays the processed results

The processor and the stimulator coordinate and cooperate, that is, the stimulator generates an intention expression interface, and the user generates the corresponding EEG signal through the stimulation paradigm of the intention expression interface. The EEG signal is collected and transmitted to the processor through the EEG cap. The middle SSVEP decoding unit decodes and analyzes the EEG signal, and finally sends the EEG signal to the stimulator. The intent expression interface displays the processing results, and sends control signals to the robotic arm, which finally grabs the desired item.

As shown in Figure 4, a life assistance system and method based on SSVEP brain-controlled robotic arm grasping intends to express the stimulator interface. The interface includes 12 blocks, 12 blocks represent 12 instructions, apple 1, banana 1, Green Tea, Orange, Cola, Apple 2, Banana 2, Mineral Water, Tangerine, Seven Up, Cancel and Confirm. 12 blocks correspond to 12 flicker frequencies, 9, 9.25, 9.5, 9.75, 10.25, 10.5, 10.75, 11, 11.25, 11.5, 11.75 and 12Hz. The frequencies of different blocks are different. According to the principle of steady-state visual induction of SSVEP, the user looks at different blocks, that is, at different frequencies, and the brain generates different EEG signals. By decoding the EEG signals, the user's intention can be known. .

EEG decoding adopts FBCCA (Filter bank canonical correlation analysis, filter bank canonical correlation analysis) algorithm, this method is an improvement of CCA (canonical correlation analysis, canonical correlation analysis) algorithm, on the basis of CCA, a filter bank is added, The harmonic components of the EEG signal that are not fully utilized in the traditional CCA algorithm are used to improve the accuracy of the algorithm. Algorithm decoding steps:

1) Construct N bandpass filters, all bandpass filters cover all passband frequencies as fully as possible;

2) It is assumed that the solution of the grade is the maximum correlation coefficient between the signal of frequency f _k and the EEG signal X. Pass the EEG data into N different band-pass filters respectively to obtain N groups of data after passing through the band-pass filters;

3) using the CCA algorithm to solve the maximum correlation coefficient with the reference signal formed by each group of data and the standard sine and cosine to obtain N correlation coefficients;

4) Calculate the harmonic weight according to the formula w(n)=ae ^-bn +c, and substitute the obtained result into the formula

Solve the sum of the weights as the maximum correlation coefficient ρ _k of the signal of frequency f _k and the EEG signal X;

5) Substitute the reference signals of 12 frequencies into steps 2) to 4) respectively, and find the largest correlation coefficient therein, and the corresponding frequency is the frequency of the SSVEP signal at this time, that is, the identification result is obtained;

As shown in Figure 5, a life assistance system and method based on the grasping of the SSVEP brain-controlled robotic arm intends to express the interface feedback screen. If the user wants to grab the Apple 2, firstly, when the user produces a stimulus flickering on the intention expression interface, he gazes at the sixth block (Apple 2) on the interface, and when there is feedback on the interface, the Apple 2 will be displayed on the top of the interface. When the stimuli flicker again on the intent expression interface, look at the last block (confirm) on the interface. When feedback appears on the interface, grab will be displayed on the top of the interface, thus completing a grab command output. After the interface appears to grab, the computer sends control commands to the robotic arm, and the robotic arm grabs the corresponding item.

In the process of grasping, in order to achieve a better grasping effect, the robotic arm does not touch the target object during the grasping process, and a pre-grabbing pose is set. Set it to 10cm in the negative direction of the Z axis of the grasping pose. Because the pre-grab pose is set, there are two trajectories in the grabbing process.

1) From the starting position to the pre-grab pose

2) From pre-grabbing pose to grasping pose

Since each planned path is from the initial velocity 0 to the final velocity 0, if the two trajectories are directly merged, there will be frequent pauses during the movement, so the problem of trajectory re-planning is involved.

First, plan the two movements separately. After merging the two planned trajectory points, the interface of the IPTP time optimization algorithm is called, and the speed, acceleration, time and other information of the new trajectory are re-planned. After the planning is completed, the continuous motion can be called.

To sum up, the above-mentioned embodiments of the present invention are based on the living assistance system and method for grasping by the SSVEP brain-controlled robotic arm. The method of the present invention is as follows: firstly, a professional helps the user to wear the EEG cap 1, and secondly, the intention expression interface 3 on the computer 2 end is opened, and when the intention expression interface 3 flashes with stimulation, the user pays attention to the intention expression interface according to his own needs. In the corresponding block on 3, the EEG cap 1 collects the EEG signal of the user at this time, and transmits it to the SSVEP decoding unit 4 on the computer 2 side. After preprocessing and FBCCA (Filter bank canonical correlation analysis, filter bank canonical correlation analysis) The algorithm is decoded, and the decoded result is converted into a control command and sent to the robotic arm 5. The robotic arm 5 executes the control command and grabs the corresponding items according to the planned path, so as to achieve the purpose of assisting the user's daily life, especially for the elderly and the disabled. More convenience and safety in life.

The embodiments of the present invention have been described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned embodiments, and various changes can also be made according to the purpose of the invention and creation of the present invention. Changes, modifications, substitutions, combinations or simplifications should be equivalent substitution methods, as long as they meet the purpose of the invention, as long as they do not deviate from the invention of a living assistance system and method based on SSVEP brain-controlled robotic arm grasping. The technical principle and the inventive concept all belong to the protection scope of the present invention.

Claims (6)

1. A life assistance system based on SSVEP brain-controlled robotic arm grasping, comprising an EEG cap (1), a computer (2), an intention expression interface (3), an SSVEP decoding unit (4) and a robotic arm (5), It is characterized in that: the EEG cap (1) is connected to the computer (2) by wired or wireless means, the computer (2) includes an intention expression interface (3) and an SSVEP decoding unit (4), and the robotic arm (5) is wired through the computer (2). The connection is connected to the computer (2); a professional helps the user to put on the EEG cap (1), and opens the intention expression interface (3) on the computer (2) end. When the intention expression interface (3) flashes a stimulus, the user According to your own needs, look at the corresponding block on the intention expression interface (3), the EEG cap (1) collects the EEG signal of the user at this time, and transmits it to the SSVEP decoding unit (4) on the computer (2) side. Processing and filter bank canonical correlation analysis FBCCA algorithm for decoding, the decoding result is converted into a control command and sent to the robot arm (5), the robot arm (5) executes the control command, and grabs the corresponding item according to the planned path. 2. a kind of life assistance system based on SSVEP brain-controlled robotic arm grasping according to claim 1, is characterized in that: described intention expression interface (3) comprises 12 blocks, and 12 blocks represent 12 instructions: Apple 1, Banana 1, Green Tea, Orange, Cola, Apple 2, Banana 2, Mineral Water, Orange, Seven-Up, Cancel and Confirm; the user first looks at the desired item, then looks at the cancel or confirm block, and after the confirmation block is activated , the robotic arm performs the action of grabbing the item. 3. a kind of life assistance system based on SSVEP brain-controlled mechanical arm grasping according to claim 1, is characterized in that: described SSVEP decoding unit (4) first preprocesses the user's EEG signal, namely filtering, Then, the canonical correlation analysis of the FBCCA filter bank is used to finally decode and identify the instruction that the user wants to send, so that the user's brain controls the robotic arm (5) to grab the corresponding item. 4. the life assistance system based on the SSVEP brain-controlled robotic arm grasping according to claim 1, it is characterized in that: the robotic arm (5) carries out path planning by forward and inverse kinematics principle, and adopts continuous trajectory planning to avoid stuck phenomenon . 5. a kind of living assistance method based on SSVEP brain-controlled robotic arm grasping, adopts the living assistance system grasped based on SSVEP brain-controlled robotic arm according to claim 1 to operate, it is characterized in that, operating steps are as follows: 1) Professionals help users to wear EEG caps (1); 2) Open the intent expression interface (3); 3) When the intention expression interface (3) flashes a stimulus, the user will look at the corresponding block according to their own needs; at the same time, the EEG cap (1) collects the EEG signal and transmits it to the SSVEP on the computer (2) side by wire or wirelessly. decoding unit (4); 4) SSVEP decoding unit (4) decodes by FBCCA algorithm; 5) After the SSVEP decoding unit (4) is decoded by the FBCCA algorithm, it sends a control command to the robotic arm (5); 6) The robotic arm (5) grabs the corresponding item according to the planned path. 6. according to the described living assistance method of SSVEP brain-controlled mechanical arm grasping according to claim 5, it is characterized in that: the step of FBCCA algorithm in described step 4) is as follows: 4-1) carry out filter bank analysis, decompose the SSVEP EEG signal by a plurality of different passbands of the filter, obtain the subband signal after each subband of the filter; 4-2) Correlation analysis is carried out with each subband component obtained by filtering and the standard sine and cosine reference signal; 4-3) The frequency corresponding to the maximum correlation is the identification result.

Images