CN114356074B - Animal brain-computer interface implementation method and system based on in-vivo fluorescence signals - Google Patents

Abstract

The invention provides an animal brain-computer interface realization method and system based on an in-vivo fluorescence signal, comprising the following steps: injecting a calcium ion indicator or indicator protein into the cortex or deep nucleus of the brain to express the indicator or indicator protein in cells by using in vivo viruses; after the indication medium enters the cells, implanting a brain interface in a target brain area of the experimental animal, recording, and transmitting a recording result to a signal processing host; after the data result is transmitted, the data result enters a decoding algorithm to identify a pattern rule, and an output interface outputs the pattern rule to a control end; after the control terminal receiving device receives the signals, recoding the signals finally realizes coding control on external equipment, and brain-computer interaction is formed. The invention can solve the problems that the signal source of the traditional brain-computer interface of the brain-computer signal is greatly influenced by external electrical noise, and the problems of single coded information and low coding efficiency.

Description

Animal brain-computer interface implementation method and system based on in vivo fluorescence signal

Technical Field

The present invention relates to the technical field of electroencephalogram (EEG) signal acquisition, and in particular to a method and system for realizing an animal brain-computer interface based on in vivo fluorescence signals.

Background technique

A brain-computer interface refers to a direct connection created between the human or animal brain and an external device to achieve information exchange between the brain and the device. Currently, the commonly used brain-computer interface uses electrical signals of neural activity to decode information. The external electrical interference encountered in life scenarios is very large, and its use in free activity scenarios is relatively limited. In recent years, in animal experiments, through in vivo fluorescence microscopy technology, neurochemical probes can be used to record changes in neural activity in detail. This method can accurately analyze the activity patterns of different neurons, and at the same time provide a good signal source for signal recognition in brain-computer interfaces, for information decoding and re-encoding.

The invention patent with publication number CN106293088A discloses a brain-computer interface processing system and its implementation method. The brain-computer interface processing system includes a brain-computer interface processing device and a software application platform. The brain-computer interface processing device includes dry electrodes, EEG signal acquisition and analysis modules, microprocessors, wireless communication modules and processing and display terminals connected in sequence. The software application platform includes an EEG signal storage module, an EEG signal online feedback module and an open API interface. The microprocessor interacts with the wireless communication module, the EEG signal storage module, the EEG signal online feedback module and the open API interface respectively; the wireless communication module interacts with the processing and display terminal.

The existing technology has the following defects: the EEG signals in commonly used brain-computer interfaces are greatly affected by external electrical noise, and have high requirements for the implementation environment; the spatial resolution of EEG signals is low, which limits the encoding ability of the brain-computer interface; the recorded neuronal signals cannot correspond for a long time, and interference is very easy to occur between signals.

Summary of the invention

In view of the defects in the prior art, the present invention provides a method and system for realizing an animal brain-computer interface based on in vivo fluorescence signals.

According to the present invention, a method and system for realizing an animal brain-computer interface based on in vivo fluorescence signals are provided, and the scheme is as follows:

In a first aspect, a method for implementing an animal brain-computer interface based on an in vivo fluorescent signal is provided, the method comprising:

Step S1: injecting a calcium ion indicator or an indicator protein in vivo virus into the cerebral cortex or deep nuclei to express the indicator or indicator protein in cells;

Step S2: After the indicator medium enters the cell, a brain interface is implanted in the target brain region of the experimental animal, and the recording is performed through an in vivo miniature fluorescence microscope. The recording results are transmitted to the signal processing host through the signal acquisition host;

Step S3: After the data result is input, it enters the decoding algorithm to identify the pattern and is output to the control end through the command output interface;

Step S4: After receiving the signal, the control end receiving device re-encodes the signal to finally realize the coded control of the external device to form brain-computer interaction.

Preferably, after the fluorescent indication signal in the indicator is expressed, the image data is transmitted to the signal processing end through the acquisition end for signal processing, specifically including: the original fluorescent signal of the experimental animal's brain activity is collected by the acquisition device and then transmitted to the signal processing host for online processing; the original data needs to be processed by an algorithm for neuronal activity extraction and identification, mainly including noise reduction correction and cell labeling.

Preferably, the noise reduction correction is to reduce the signal noise interference caused by movement and to perform real-time correction on the positions of the same neuron at different time points, which is achieved by applying anisotropic diffusion denoising operation on the original image frame;

For each frame image, I∈R ^HxW , for a given diffusion time τ , the evolution follows the equation:

Among them, div is the divergence operator, and Δ are the gradient and Laplacian operators, respectively, and C is the diffusion coefficient matrix that depends on pixels and τ;

I represents the current frame; R ^HxW represents a frame set; H represents the frame height; W represents the frame width;

Subsequently, a KTL tracker is used between different frames to estimate the displacement of potential angular features between two adjacent frames. After alignment based on the KTL tracker, diffeomorphic Log-Demons is used for further unit alignment.

The image data finally collected will minimize motion noise for subsequent analysis and processing.

Preferably, the cell marking includes: after image smoothing, it is necessary to mark the potential cell outline image:

First, randomly select a portion of the frames:

Where a is the frame number of the random contour set; s represents the contour projection; T represents the maximum frame number;

Then calculate the maximum projection map of the selected frame:

And detect all local maximum points on the graph as S _α , repeat the above process multiple times, any two of them do not contain the same frame;

Finally, the integration of S _α of all maximum projection images and the maximum projection image S calculated with the complete contour data set is collected as the final neuron contour seed set.

Preferably, after obtaining the neuron profile, the system will perform fluorescence activity monitoring, pattern learning and classification on the corresponding neurons, specifically:

First, the fluorescence brightness of the image data needs to be normalized. The average brightness of the target cell outline _Ft minus the average background fluorescence brightness _F0 is divided by the background fluorescence brightness value _F0 . The calculation formula is as follows, where _ΔFt / _F0 is abbreviated as ΔF/F:

Secondly, it is necessary to use the exponentially weighted moving average method to de-noise and smooth the recorded calcium signals;

Thirdly, after obtaining the smoothed fluorescence intensity change value, the fluorescence activity events are detected online;

Finally, the experimental animal behavior is collected synchronously, and the animal behavior and real-time fluorescence imaging results are paired for pattern learning. Based on the PCA and SVM algorithms, feature extraction and pattern classification are performed to classify and learn the existing fluorescence activity patterns.

After pattern learning and classification, the output instructions are encoded accordingly according to the classification pattern. When executing the work, the fluorescent activity pattern is judged online, and the corresponding instructions are output to control the external equipment.

Preferably, after obtaining the neuron outline, an overall fluorescence observation method is used to indicate the activity of the nucleus. After obtaining the neuron outline, the average grayscale value of the entire frame image is calculated. The steps are to count all the pixels of the entire image, accumulate the sum of the grayscale values of all the pixels, and the average grayscale value is the quotient of the sum of the grayscale values and the number of pixels.

Preferably, the specific steps for calculating the average gray value are:

1) Count the grayscale values of the pixels of each frame of the data source image and add them up;

2) Calculate the total number of pixels in the image: n = view width * view height;

3) Find the average gray value of the image: avg_value = sum/n;

After obtaining the average value, the change of average fluorescence intensity is tracked in real time, the fluorescence intensity curve is drawn, and nuclear activity is monitored; and the activity threshold is set according to research needs. In the actual working process, over-threshold monitoring is performed in real time. Once the threshold is exceeded, the command output is triggered to achieve control of external equipment or closed-loop regulation.

In a second aspect, an animal brain-computer interface system based on in vivo fluorescence signals is provided, the system comprising:

Module M1: Inject calcium ion indicator or indicator protein in vivo virus into the cerebral cortex or deep nuclei to express the indicator or indicator protein in cells;

Module M2: After the indicator medium enters the cell, a brain interface is implanted in the target brain area of the experimental animal, and the recording is performed through an in vivo miniature fluorescence microscope. The recording results are transmitted to the signal processing host through the signal acquisition host;

Module M3: After the data result is input, it enters the decoding algorithm for pattern recognition and is output to the control end through the command output interface;

Module M4: After receiving the signal, the control end receiving device re-encodes the signal to finally realize the coded control of the external device to form brain-computer interaction.

Preferably, after the fluorescent indication signal in the indicator is expressed, the image data is transmitted to the signal processing end through the acquisition end for signal processing, specifically including: the original fluorescent signal of the experimental animal's brain activity is collected by the acquisition device and then transmitted to the signal processing host for online processing; the original data needs to be processed by an algorithm for neuronal activity extraction and identification, mainly including noise reduction correction and cell labeling.

Preferably, the noise reduction correction is to reduce the signal noise interference caused by motion and to perform real-time correction on the positions of the same neuron at different time points, which is achieved by applying anisotropic diffusion denoising operation on the original image frame;

For each frame image, I∈R ^HxW , for a given diffusion time τ , the evolution follows the equation:

Among them, div is the divergence operator, and Δ are the gradient and Laplacian operators, respectively, and C is the diffusion coefficient matrix that depends on pixels and τ;

I represents the current frame; R ^HxW represents a frame set; H represents the frame height; W represents the frame width;

Subsequently, a KTL tracker is used between different frames to estimate the displacement of potential angular features between two adjacent frames. After alignment based on the KTL tracker, diffeomorphic Log-Demons is used for further unit alignment.

The image data finally collected will minimize motion noise for subsequent analysis and processing.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention collects the fluorescence of neuron activity probes as the source of neural activity signals, thereby solving the problem that the signal source of traditional EEG brain-computer interface is greatly affected by external electrical noise;

2. The present invention performs large-scale data decoding and encoding through large-scale neuron activity recognition, thereby solving the problems of single coding information and low coding efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

FIG1 is a diagram showing the system configuration of the present invention;

FIG. 2 is a schematic diagram of the specific working principle of the present invention.

Reference numerals:

Collection front end 1 Experimental animals 2

Brain interface 3 Signal acquisition host 4

Signal processing host 5 First signal mode 6

Second signal pattern 7 Pattern recognition 8

Command output interface 9 Control end receiving device 10

External Devices 11 Fluorescence Raw Signals 101

Cell Identification Module 102 Cell Identification Results 103

Pattern Learning and Classification 104 Pattern Library 105

Mode judgment instruction output module 106 Mean value calculation module 107

Activity Monitoring Module 108 Threshold Monitoring Module 109

Command output module 110

Detailed ways

The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

In order to expand the brain signal source of the brain-computer interface, reduce the impact of external electrical noise on the function of the brain-computer interface, enrich the coding capability of the brain-computer interface, and avoid mutual interference between neural electrical signals, an embodiment of the present invention provides an animal brain-computer interface system based on in vivo fluorescence signals, which can monitor the activities of thousands of neurons at the same time, is not affected by external electrical noise, and can perform complex encoding; the system can decode complex local brain activities, interact with external devices 11, and control the devices.

An animal brain-computer interface system based on in vivo fluorescence signals can indirectly reflect the activity pattern and coding state of neurons and nervous systems by marking active substances in neurons with fluorescent probes. Decoding neuronal signals using this as a signal source can provide a richer signal source in addition to electrical signals for decoding and recoding brain-computer interface signals. Referring to Figures 1 and 2, the system mainly includes: a chemical indicator medium, an in vivo fluorescence detection and acquisition device, a signal processing device, and an output control terminal.

The chemical indicator medium is mainly realized by calcium ion indicators or other types of chemical fluorescent probes, including all ion and compound indicator probes whose concentration changes can indicate the functional state of nerve cells;

The in vivo fluorescence detection equipment is mainly composed of a miniature in vivo fluorescence imaging microscope;

The signal processing equipment and output control terminal are composed of computer-based decoding and encoding algorithms and software and hardware.

The system includes the following modules: Module M1: injecting calcium ion indicator or indicator protein in vivo virus into the cerebral cortex or deep nuclei to express the indicator or indicator protein in cells;

Module M2: After the indicator medium enters the cell, a brain interface is implanted in the target brain area of the experimental animal, and the recording is performed through an in vivo miniature fluorescence microscope. The recording results are transmitted to the signal processing host through the signal acquisition host;

Module M3: After the data result is input, it enters the decoding algorithm for pattern recognition and is output to the control end through the command output interface;

Module M4: After receiving the signal, the control end receiving device re-encodes the signal to finally realize the coded control of the external device to form brain-computer interaction.

After the fluorescent indication signal in the indicator is expressed, the image data is transmitted to the signal processing end through the acquisition end for signal processing, which specifically includes: the original fluorescent signal of the experimental animal's brain activity is collected by the acquisition device and then transmitted to the signal processing host for online processing; the original data needs to be processed by the algorithm for neuronal activity extraction and identification, mainly including noise reduction correction and cell labeling.

Denoising correction is to reduce the signal noise interference caused by motion and to correct the position of the same neuron at different time points in real time, which is achieved by applying anisotropic diffusion denoising operation on the original image frame;

For each frame image, I∈R ^HxW , for a given diffusion time τ , the evolution follows the equation:

Among them, div is the divergence operator, and Δ are the gradient and Laplacian operators, respectively, and C is the diffusion coefficient matrix that depends on pixels and τ;

I represents the current frame; R ^HxW represents a frame set; H represents the frame height; W represents the frame width;

Subsequently, a KTL tracker is used between different frames to estimate the displacement of potential angular features between two adjacent frames. After KTL tracker-based registration, diffeomorphic Log-Demons is used for further unit registration;

The image data finally collected will minimize motion noise for subsequent analysis and processing.

The present invention also provides a method for realizing an animal brain-computer interface based on an in vivo fluorescent signal, the method comprising:

Step S1: injecting a calcium ion indicator or an indicator protein in vivo virus into the cerebral cortex or deep nuclei to express the indicator or indicator protein in cells;

Step S2: After the indicator medium enters the cell, a brain interface 3 is implanted in the target brain region of the experimental animal 2, and the recording is performed through an in vivo miniature fluorescence microscope. The recording result is transmitted to the signal processing host 5 through the signal acquisition host 4;

Step S3: After the data result is input, it enters the decoding algorithm for pattern recognition 8 and is output to the control end through the instruction output interface 9;

Step S4: After receiving the signal, the control end receiving device 10 re-encodes the signal to finally implement coding control of the external device 11 to form brain-computer interaction.

Specifically, after the fluorescent indication signal in the indicator is expressed, the image data is transmitted to the signal processing end through the acquisition end for signal processing, which specifically includes: the original fluorescent signal 101 of the brain activity of the experimental animal 2 is collected by the acquisition device and transmitted to the signal processing host 5 for online processing; the original data needs to be extracted and identified by the algorithm of the cell recognition module 102 for neuronal activity extraction and identification, which mainly includes noise reduction correction and cell labeling.

Among them, noise reduction correction is to reduce the signal noise interference caused by movement and perform real-time correction on the position of the same neuron at different time points, which is achieved by applying anisotropic diffusion denoising operation on the original image frame;

For each frame image, I∈R ^HxW , for a given diffusion time τ , the evolution follows the equation:

Among them, div is the divergence operator, and Δ are the gradient and Laplacian operators, respectively, and C is the diffusion coefficient matrix that depends on pixels and τ;

I represents the current frame; R ^HxW represents a frame set; H represents the frame height; W represents the frame width;

Subsequently, a KTL tracker is used between different frames to estimate the displacement of potential angular features between two adjacent frames. After KTL tracker registration, diffeomorphic Log-Demons is used for further unit registration;

The image data finally collected will minimize motion noise for subsequent analysis and processing.

Cell marking includes: After image smoothing, the potential cell outline image needs to be marked:

First, randomly select a portion of the frames:

Where a is the frame number of the random contour set; s represents the contour projection; T represents the maximum frame number;

Then calculate the maximum projection map of the selected frame:

And detect all local maximum points on the graph as S _α , repeat the above process multiple times, any two of them do not contain the same frame;

Finally, the integration of S _α of all maximum projection images and the maximum projection image S calculated with the complete contour data set is collected as the final neuron contour seed set.

After obtaining the neuron outline cell recognition result 103, the system will perform fluorescence activity monitoring, pattern learning and classification 104 on the corresponding neurons, specifically:

First, the fluorescence brightness of the image data needs to be normalized. The average brightness of the target cell outline _Ft minus the average background fluorescence brightness _F0 is divided by the background fluorescence brightness value _F0 . The calculation formula is as follows, where _ΔFt / _F0 is abbreviated as ΔF/F:

Secondly, it is necessary to use the exponentially weighted moving average method to de-noise and smooth the recorded calcium signals;

Thirdly, after obtaining the smoothed fluorescence intensity change value, the fluorescence activity events are detected online;

Finally, the behavior of experimental animal 2 is collected synchronously, and the animal behavior and real-time fluorescence imaging results are paired to perform pattern learning, and feature extraction and pattern classification are performed based on PCA and SVM algorithms to classify the existing fluorescence activity patterns;

After pattern learning and classification, a pattern library 105 is established, and the output instructions are encoded accordingly according to the classification pattern through the pattern judgment instruction output module 106. When executing the work, the fluorescent activity pattern is judged online and the corresponding instructions are output to control the external device 11.

At the same time, after obtaining the neuron contour cell identification result 103, the overall fluorescence observation method can also be used in parallel to indicate the activity of the nucleus. After obtaining the neuron contour cell identification result 103, the average grayscale value of the entire frame image is calculated in the mean calculation module 107. The steps are to count all the pixel points of the entire image, accumulate and obtain the sum of the grayscale values of all the pixel points, and the average grayscale value is the quotient of the sum of the grayscale values and the number of pixels.

The specific steps are:

1) Count the grayscale values of the pixels of each frame of the data source image and add them up;

2) Calculate the total number of pixels in the image: n = view width * view height;

3) Find the average gray value of the image: avg_value = sum/n;

After obtaining the average value, the change of the average fluorescence intensity is tracked in real time, the fluorescence intensity curve is drawn, and the nuclear activity is monitored in the activity monitoring module 108; and the activity threshold is set according to the research requirements. In the actual working process, the threshold monitoring module 109 performs real-time over-threshold monitoring. Once the threshold is exceeded, the command output is triggered in the command output module 110 to realize the control of the external device 11 or closed-loop regulation.

Next, the present invention will be described in more detail.

The implementation process of the present invention is as follows: by injecting calcium ion indicator or indicator protein in vivo virus into the cerebral cortex or deep nuclei, the indicator or indicator protein is expressed intracellularly. After the indicator medium enters the cell, a brain interface 3 is implanted in the target brain area of the experimental animal 2, and the recording is performed through an in vivo miniature fluorescence microscope. The recording results are transmitted to the signal processing host 5 through the signal acquisition host 4. The data contains two forms. The fluorescence intensity change value can obtain the first signal mode 6 and the image data can obtain the second signal mode 7. After the data results are transmitted, they enter the decoding algorithm for pattern regularity recognition 8, and are output to the control end through the instruction output interface 9. After the control end receiving device 10 receives the signal, it re-encodes the signal to finally realize the coding control of the external device 11 to form brain-computer interaction.

Among them, after the fluorescence indication signal is expressed, the image data is transmitted to the signal processing end through the acquisition end for signal processing. The working principle is:

After the raw fluorescent signal 101 of the brain activity of the experimental animal 2 is collected by the acquisition device, it is transmitted to the signal processing host 5 for online processing. The raw data needs to be processed by an algorithm for neuronal activity extraction and identification, which mainly includes noise reduction correction and cell labeling.

The main function of the noise reduction correction module is to reduce the signal noise interference caused by motion and to correct the position of the same neuron at different time points in real time. It is mainly achieved by applying anisotropic diffusion denoising operation on the original image frame. For each frame image, I∈R ^HxW , for a given diffusion time τ, the evolution follows the equation

Among them, div is the divergence operator, and Δ are the gradient and Laplacian operators, respectively, and C is the diffusion coefficient matrix that depends on pixels and τ;

I represents the current frame; R ^HxW represents a frame set; H represents the frame height; W represents the frame width;

By choosing the specific form of C and τ, we can control the level of smoothing along or perpendicular to the boundary between the neuron and the background. C selects the classic Perona-Malik filter Prioritize smoothing of low-contrast areas, where exp(.) represents an exponential function with base e; ||.|| represents the value of the determinant of the matrix; and k represents boundary sensitivity.

Subsequently, the Kanade-Lucas-Tomasi (KTL) tracker is used between different frames to estimate the displacement of potential corner features between two adjacent frames. After the KTL tracker-based registration, diffeomorphic Log-Demons is used for further unit registration. Diffeomorphic Log-Demons is a non-parametric registration method that can find the displacement of all pixels by minimizing the global energy in the logarithmic domain:

It includes similarity, correspondence and regularization terms, where Sim(.) represents the similarity term; dist(.) represents the distance term; Reg(.) represents the annotation term; s represents the contour projection; F and M are the fixed image and the moving image respectively, u is the displacement/update field, σ _i ,σ _x and σ _T represent the noise level, the corresponding spatial uncertainty and the regularization level. After the above steps, the collected image data will minimize the motion noise and can be processed for subsequent analysis.

After image smoothing, the potential cell contour images need to be labeled. The initial generated contour set of all potential actual cell contour ROI centers, but the contours contain a large number of false positive cell contours. The process first randomly selects a part of the frame

Where a is the frame number of the random contour set; s represents the contour projection; T represents the maximum frame number;

Then calculate the maximum projection map of the selected frame:

And detect all local maximum points on the graph as S _α , repeat the above process multiple times, any two of which do not contain the same frame. Finally, collect the integration of S _α of all maximum projection graphs and the maximum projection graph S calculated with the complete contour data set as the final neuron contour seed set.

After obtaining the neuron contour cell recognition result 103, the system will monitor the fluorescence activity of the corresponding neuron and perform pattern learning and classification. First, the fluorescence brightness of the image data needs to be normalized. The average brightness of the target cell contour _Ft minus the average background fluorescence brightness _F0 is divided by the background fluorescence brightness value _F0 . The calculation formula is as follows, where _ΔFt / _F0 is abbreviated as ΔF/F:

Secondly, the recorded calcium signals need to be denoised and smoothed using the exponentially weighted moving average (EWMA) method;

Next, after obtaining the smoothed fluorescence intensity change value, the fluorescence activity event is detected online. The steps mainly include defining the baseline length, setting the detection window length, and defining the rising rate threshold. After exceeding the threshold, it is defined as a fluorescence activity event, and the fluorescence value of the peak point is collected in real time.

Finally, the behavior of experimental animal 2 is collected synchronously, and the animal behavior and real-time fluorescence imaging results are paired for pattern learning. Feature extraction and pattern classification are performed based on PCA and SVM algorithms to classify the existing fluorescence activity patterns.

After pattern learning and classification, the output instruction is encoded accordingly according to the classification pattern, and the fluorescence activity pattern is judged online during execution, and the corresponding instruction is output to control the external device 11.

At the same time, after obtaining the neuron outline cell recognition result 103, the overall fluorescence observation method can also be used in parallel to indicate the activity of the nucleus. First, after obtaining the neuron outline cell recognition result 103, the average gray value of the entire frame image is calculated in the mean calculation module 107. The steps are to count all the pixels of the entire image and accumulate the sum of the gray values of all the pixels. The average gray value is the quotient of the sum of the gray values and the number of pixels. The steps are as follows:

1) Count the grayscale values of the pixels of each frame of the data source image and add them up;

2) Calculate the total number of pixels in the image: n = view width * view height;

3) Find the average grayscale value of the image: avg_value = sum/n.

After obtaining the average value, the change of the average fluorescence intensity is tracked in real time, and the fluorescence intensity curve is drawn. The activity monitoring module 108 monitors the nuclear group activity. The activity threshold is set according to the research requirements. In the actual working process, the threshold monitoring module 109 performs real-time over-threshold monitoring. Once the threshold is exceeded, the command output is triggered in the command output module 110 to realize the control of the external device 11 or closed-loop regulation.

Workflow of software and hardware cooperating with each other: The invented system includes a chemical indicator medium, an in vivo fluorescence detection and acquisition device, a signal processing device, and an output control terminal. By monitoring the activity of chemical substances in the brain, indicating neuronal activity, identifying the neuronal activity patterns caused by different behavioral paradigms, realizing pattern recognition and output, and then controlling external devices 11. Using fluorescent indicator signals as signal sources can avoid interference from external electrical noise, and achieve more intuitive and effective neuronal activity decoding capabilities, thereby improving the coding effect of brain-computer interfaces. This design is different from existing brain-computer interfaces based on EEG signals. It uses real-time activity of chemical substance fluorescence signals to realize the recognition of neuronal activity patterns, interact with external devices, enrich the signal sources of brain-computer interfaces, and can achieve in-depth research on neural activity coding.

Brain-computer control experiment based on calcium ion signals:

a) Experimental Animal 2: Healthy adult C57bl/6 mice, clean grade, weighing 20-25 g. Experimental Animal 2 was housed in an independent environment with a 12-hour-12-hour day and night cycle, with room temperature maintained at 24±2°C, free access to water and food, and the experiment was conducted after acclimatization for 1 week.

b) Experimental equipment: calcium signal indicator transgenic mice (GCaMP6 transgenic mice), specific calcium signal indicator protein vector virus (GCaMP6 vector virus driven by a specific promoter), virus injection instrument, resin lens, fixed surgical equipment, brain-computer system.

C) Experimental steps: As shown in Figures 1 and 2, taking the motor cortex as an example, transgenic mice are selected or specific neurons in the target brain area are infected by specific calcium signal indicator protein viruses to achieve indicator marks for calcium signal activities of specific neuronal groups in the target brain area. The brain interface 3 is implanted into the target brain area by stereotaxic means, and fixed for a long time using a fixing material. After the mouse recovers, the acquisition front end 1 and the signal acquisition host 4 are used to collect and analyze the calcium ion activities of neurons in the target brain area in real time. The calcium ion activities of the first signal mode 6 and the second signal mode 7 are identified by the signal processing host 5. After the data results are transmitted, they enter the decoding algorithm for pattern rule recognition 8, including fluorescence intensity and neuronal activation order and pattern rules, and are output to the control end through the command output interface 9. After the control end receiving device 10 receives the signal, it re-encodes the signal and finally realizes the encoding control of the external device 11, forming brain-computer interaction, controlling the external device 11 or closed-loop regulation of the corresponding functions of the body.

The embodiments of the present invention provide an animal brain-computer interface implementation method and system based on in vivo fluorescence signals, which utilizes the fluorescence signals of neuron activity probes as the brain-computer interface signal source to solve the problem that the traditional EEG signal brain-computer interface signal source is greatly affected by external electrical noise; the brain activity characteristics during different behavioral processes are collected and acquired by in vivo fluorescence detection means; the neural activity is decoded and re-encoded through the dynamic changes of chemical substances inside local neurons in the animal brain, solving the problem of single encoding information and low encoding efficiency.

Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other at will.

Claims (10)

1. An animal brain-computer interface implementation method based on an in-vivo fluorescence signal is characterized by comprising the following steps:

Step S1: injecting a calcium ion indicator or indicator protein into the cortex or deep nucleus of the brain to express the indicator or indicator protein in cells by using in vivo viruses;

Step S2: after the indication medium enters cells, a brain interface is implanted in a target brain area of an experimental animal, the brain interface is recorded by an in-vivo micro fluorescent microscope, and a recording result is transmitted to a signal processing host through a signal acquisition host;

Step S3: after the data result is transmitted, the data result enters a decoding algorithm to identify a pattern rule and is output to a control end through an instruction output interface;

step S4: after the control terminal receiving device receives the signals, recoding the signals finally realizes coding control on external equipment, and brain-computer interaction is formed.

2. The method for realizing an animal brain-computer interface based on an in-vivo fluorescence signal according to claim 1, wherein after the fluorescence indication signal in the indicator is expressed, the image data is transmitted into a signal processing end for signal processing through an acquisition end, and the method specifically comprises the following steps: after being collected by the collecting equipment, the brain activity fluorescence original signals of the experimental animals are transmitted to the signal processing host for on-line processing; the original data is subjected to neuron activity extraction and identification through an algorithm, and mainly comprises noise reduction correction and cell marking.

3. The method for realizing the animal brain-computer interface based on the in-vivo fluorescence signal according to claim 2, wherein the noise reduction correction is to reduce signal noise interference caused by motion and correct the positions of the same neuron at different time points in real time by applying an anisotropic diffusion noise removal operation on an original image frame;

For each frame of image, I ε R ^HxW, the evolution follows the equation for a given diffusion time τ:

Where div is the divergence operator, And delta is the gradient and laplace operator, respectively, C is the diffusion coefficient matrix dependent on pixel and τ;

I represents the current frame; r ^HxW represents a set of frames; h represents a frame height; w represents the frame width;

Subsequently, a KTL tracker is used between different frames to estimate the displacement of potential angular features between two adjacent frames; after KTL tracker based registration, further unit registration is performed using diffeomorphic Log-Demons;

finally, the acquired image data is minimized in motion noise, and further subsequent analysis and processing are performed.

4. The method of claim 3, wherein the cell labeling comprises: after the image smoothing process, the potential cell contour image needs to be subjected to cell marking:

First, a portion of the frames is randomly selected:

Where a is the frame number of the random profile set; s represents contour projection; t represents the maximum frame number;

The maximum projection map for the selected frame is then calculated:

Detecting that all local maximum points on the graph are S _α, repeating the process for a plurality of times, wherein any two of the local maximum points do not contain the same frame;

and finally, collecting S _α of all the maximum projection graphs and calculating the integration of the maximum projection graphs S with the complete contour data set as a final neuron contour seed set.

5. The method according to claim 4, wherein after obtaining the neuron profile, the system will perform fluorescence activity monitoring and pattern learning and classification of the corresponding neurons, in particular:

First, the fluorescence brightness of the image data needs to be normalized, and the average brightness F _t of the target cell contour is subtracted by the average background fluorescence brightness F ₀ of the tie and divided by the background fluorescence brightness value F ₀, and the calculation formula is as follows, wherein Δf _t/F₀ is abbreviated as Δf/F:

Secondly, denoising and smoothing the recorded calcium signals by using an exponential weighted moving average method;

thirdly, after the smoothed fluorescence intensity change value is obtained, detecting a fluorescence activity event on line;

Finally, synchronously collecting the behaviors of the experimental animal, pairing the behaviors of the animal with real-time fluorescence imaging results, performing pattern learning, extracting features and classifying patterns based on PCA and SVM algorithms, and performing classification learning on existing fluorescence activity patterns;

After pattern learning and classification, corresponding coding is carried out on the output instruction according to the classification pattern, the fluorescence activity pattern is judged online when the operation is executed, and the corresponding instruction is output to control the external equipment.

6. The method for realizing the animal brain-computer interface based on the in-vivo fluorescence signal according to claim 4, wherein after obtaining the neuron outline, a whole fluorescence observation method is selected to indicate the activity of the nucleus, after obtaining the neuron outline, the average gray value of the whole frame image is calculated, wherein the method comprises the steps of counting all pixel points of the whole image, accumulating and calculating the gray value sum of all the pixel points, and the average gray value is the quotient of the gray value sum and the number of the pixel points.

7. The method for realizing the animal brain-computer interface based on the in-vivo fluorescence signal according to claim 6, wherein the specific steps of calculating the average gray value are as follows:

1) Counting the gray value of the pixel of each frame of data source image to carry out accumulation summation;

2) Calculating the total number of pixels of the image: n= VIEW WIDTH × VIEW HEIGHT;

3) Calculating the average gray value of the image: avg_value=sum/n;

After the average value is obtained, the change of the average fluorescence intensity is tracked in real time, a fluorescence intensity curve is drawn, and nuclear mass activity monitoring is carried out; and the activity threshold is set according to the research requirement, the threshold exceeding monitoring is carried out in real time in the actual working process, and once the threshold is exceeded, the command output is triggered, so that the external equipment is controlled or closed-loop regulation and control are carried out.

8. An animal brain-computer interface system based on in vivo fluorescence signals, comprising:

Module M1: injecting a calcium ion indicator or indicator protein into the cortex or deep nucleus of the brain to express the indicator or indicator protein in cells by using in vivo viruses;

Module M2: after the indication medium enters cells, a brain interface is implanted in a target brain area of an experimental animal, the brain interface is recorded by an in-vivo micro fluorescent microscope, and a recording result is transmitted to a signal processing host through a signal acquisition host;

Module M3: after the data result is transmitted, the data result enters a decoding algorithm to identify a pattern rule and is output to a control end through an instruction output interface;

Module M4: after the control terminal receiving device receives the signals, recoding the signals finally realizes coding control on external equipment, and brain-computer interaction is formed.

9. The animal brain-computer interface system based on in-vivo fluorescence signals according to claim 8, wherein after the fluorescence indication signals in the indicator are expressed, the image data are transmitted into the signal processing end through the acquisition end for signal processing, and specifically comprising: after being collected by the collecting equipment, the brain activity fluorescence original signals of the experimental animals are transmitted to the signal processing host for on-line processing; the original data is subjected to neuron activity extraction and identification through an algorithm, and mainly comprises noise reduction correction and cell marking.

10. The animal brain-computer interface system based on an in-vivo fluorescence signal according to claim 9, wherein said noise reduction correction is to reduce motion-induced signal noise interference and correct the positions of the same neuron at different time points in real time by applying a hetero-diffusion denoising operation on the original image frame;

For each frame of image, I ε R ^HxW, the evolution follows the equation for a given diffusion time τ:

Where div is the divergence operator, And delta is the gradient and laplace operator, respectively, C is the diffusion coefficient matrix dependent on pixel and τ;

I represents the current frame; r ^HxW represents a set of frames; h represents a frame height; w represents the frame width;

Subsequently, a KTL tracker is used between different frames to estimate the displacement of potential angular features between two adjacent frames; after KTL tracker based registration, further unit registration is performed using diffeomorphic Log-Demons;

finally, the acquired image data is minimized in motion noise, and further subsequent analysis and processing are performed.