CN111144379B - Automatic identification method for visual dynamic response of mice based on image technology - Google Patents

Abstract

The invention discloses an automatic identification method of a mouse optokinetic reaction based on an image technology, and relates to an image identification method. The invention aims to solve the problems that the conventional mouse optokinetic reaction identification method consumes a lot of time and labor cost, and is low in identification accuracy and efficiency. The automatic identification method of the visual movement reaction of the mouse based on the image technology comprises the following specific processes: 1. extracting contours of the body, ears and tails of the mice; 2. identifying the orientation of the head of the mouse based on the contours of the body, ears and tail of the mouse; 21. preliminarily positioning the nose tip of the mouse based on the contours of the body, ears and tails of the mouse; 22. performing position correction on the initial positioning result of the nasal tip of the mouse obtained in the second step to obtain the nasal tip of the corrected mouse; 23. based on the nasal tips of the mice obtained by the two methods, the head orientations of the mice are identified. 3. The mouse optokinetic response is identified based on the contour of the body, ear, tail and head orientation of the mouse obtained. The invention is used in the field of biology.

Description

Automatic recognition method of mouse optokinetic response based on image technology

Technical Field

The invention relates to an image recognition method.

Background Art

Mice (commonly known as experimental mice, not necessarily white in actual applications) are a model animal often used in biological experiments. In life science and biological research, it is often necessary to evaluate the stress response of animals. However, since experimental animals cannot express their subjective feelings independently, it is necessary to study accurate and effective indirect methods to measure their response levels. There are large differences in behavioral patterns between different animal individuals. For example, mice have a visual-motor response to moving objects, that is, there is an autonomous following, resetting, and following process. A commonly used mouse visual-motor experiment method is to place the mouse in a circular grating and let it watch the rotating grating. Different experimental individuals will have different degrees of visual-motor response to the rotating grating, which is specifically manifested in the frequency of the head following the grating movement and the change in the single duration. Figure 1 is a typical simple visual-motor experiment device. During the experiment, a camera is generally used to record the movement state of the mouse, and the video is manually analyzed and counted after the experiment. For visual-motor experiments, manual statistics have the following problems:

1. The mice's response to the grating is not continuous, but intermittent, usually lasting for a few seconds before turning into irregular movements. Therefore, the experimenters need to watch the entire video (usually several minutes in length). For some movements that are difficult to identify, they need to watch them repeatedly to make a judgment.

2. Visual-kinematic experiments generally require a large number of experimental samples, and a single sample needs to be recorded for a certain length of video, so the workload of manual viewing and statistics is very huge.

3. Due to individual differences among mice, there will be subjective interference in manual statistics, which will lead to changes in the evaluation criteria. Especially in long-term experimental statistics, fluctuations in people's potential psychological factors will affect the reliability and objectivity of the entire experimental results.

Therefore, establishing an automatic recognition method for mouse visual-motor response based on computer image technology can effectively reduce labor costs and improve experimental efficiency and accuracy.

Summary of the invention

The purpose of the present invention is to solve the problems that the existing mouse visual-motor response recognition method consumes a lot of time and manpower costs and has low recognition accuracy and efficiency, and to propose an automatic mouse visual-motor response recognition method based on image technology.

The specific process of the automatic recognition method of mouse visual motion response based on image technology is as follows:

Step 1: Extract the outlines of the mouse's body, ears, and tail;

Step 2: Identify the position of the mouse head based on the outlines of the mouse body, ears, and tail;

Step 3: Based on the outlines of the mouse body, ears, and tail obtained in step 1 and the position of the mouse head obtained in step 2, identify the mouse's visual motion response.

The beneficial effects of the present invention are:

1. Fully automatic real-time recognition. You only need to set the experimental parameters and put the mice into the experimental device. After reaching the specified experimental time, you can immediately get the visual-motor experimental data without manual intervention and monitoring, which greatly saves time and labor costs. Assuming that each mouse needs to undergo a 2-minute experiment, and each group of experiments has 20 mice as an example, the manual method usually requires 2×20=40 minutes of experiment and about 3 times the video monitoring time, for a total of 160 minutes. However, using the present invention for automatic recognition, it only takes 40 minutes to complete.

2. Objective evaluation criteria. The recognition and evaluation of visual-motor responses by computers can completely avoid the influence of factors such as fatigue and lack of experience of experimenters on the experimental results, and obtain objective and accurate experimental results, which is of great significance for medical research.

3. Ability to realize high-throughput experiments. The present invention can achieve the effect of doubling the experimental efficiency by setting up multiple devices in parallel. For example, if 5 devices work at the same time, the experiment of 100 mice can get the results in only 40 minutes. For manual experiments, such a large number of samples requires huge manpower costs to improve the efficiency and accuracy of the experiment.

4. It helps to ensure the consistency of experimental conditions. Since the behavior of mice changes over time, it is necessary to ensure that all mice complete the test at the same time to ensure the consistency of experimental conditions. The application of the present invention to high-throughput experiments can achieve this requirement with a small amount of manpower investment.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of artificially observing the visual motion response of mice according to the present invention;

FIG2 is a diagram of the device used in the present invention, 1 is a grating display module, which is composed of 4 liquid crystal display screens, and the screen can display rolling light and dark stripes, and its width, brightness, speed, color and other parameters can be preset and modified by a computer; 2 is a mouse stand, which is used to place mice; 3 and 4 are the upper cover and the bottom plate, respectively, which are made of mirror material and are used to extend the longitudinal depth of the grating; 5 is a camera, which is used to collect video during the experiment;

FIG3 is the extraction result of the mouse outline of the present invention, FIG6 is the outline of the mouse body, FIG7 is the outline of the mouse ear, and FIG8 is the outline of the mouse tail;

FIG4 is a result of preliminary positioning of the nose tip of a mouse according to the present invention, 9 is the calculated center of gravity of the mouse body contour, 10 is the selected area around the mouse ear to avoid the area around the mouse auricle during the recognition process, 11 is the circumscribed rectangle of the mouse tail contour, and 12 is the obtained preselected nose tip point;

FIG5 is the correction result of the nose tip of the mouse of the present invention, 13 is the center of gravity of the front half of the mouse, 14 is the area near the head of the mouse used to select the nose tip, and 15 is the final nose tip P _n ;

6 is a result of identifying the direction of the mouse head according to the present invention, 16 and 17 are circles with _Pn as the center and radii _r1 and _r2 , 18 and 19 are the centroids _Gr1 and _Gr2 , and 20 is the orientation of the mouse head obtained later;

FIG7a is a schematic diagram of the local integration of the first-order difference used in the present invention to find the preselected area of the mouse optokinetic response;

FIG7 b is a schematic diagram of the local integration of the second-order difference used in the present invention to find the preselected area of the mouse optokinetic response;

Figure 8 is a schematic diagram of the visual-motor response state recognition of mice using the method described in the present invention. In the figure, the dotted line is a random curve of head movement, the solid line marked part is the visual-motor response state of the mouse, and except for the third solid line, the other solid line marked parts are the identified visual-motor response states of the mouse.

DETAILED DESCRIPTION

Specific implementation method 1: The specific process of the automatic recognition method of mouse visual motion response based on image technology in this implementation method is as follows:

The core content of the present invention is a method for automatically identifying the visual-motor response of mice based on computer vision technology. The method can be applied to an automatic experimental device for the visual-motor response of mice based on the principle of a rotating grating. The video of the mice inside the grating is captured by a camera, and the video is analyzed in real time by a computer to identify and mark the reaction status, number of times, duration and other information of the mice to the grating.

The structure of the experimental device used in the present invention is shown in Figure 2, wherein 1 is a grating display module, which is composed of 4 liquid crystal screens. The screen can display scrolling light and dark stripes, and its parameters such as width, brightness, speed, color, etc. can be preset and modified by a computer; 2 is a mouse platform for placing mice; 3 and 4 are the upper cover and bottom plate, respectively, which are made of mirror material and are used to expand the longitudinal depth of the grating; 5 is a camera, which is used for video acquisition during the experiment.

During the experiment, the mouse was placed on the mouse platform, and the screen displayed continuously scrolling grating stripes. The camera captured video during the experiment and sent the image to the computer, where the recognition algorithm analyzed and judged the visual motion response of the mouse. The detection process of the algorithm is as follows:

Step 1: Extract the outlines of the mouse's body, ears, and tail;

Step 2: Identify the position of the mouse head based on the outlines of the mouse body, ears, and tail;

Step 3: Based on the outlines of the mouse body, ears, and tail obtained in step 1 and the orientation of the mouse head obtained in step 2, identify the mouse's visual motion response; based on the previous two steps, the mouse's head orientation and movement changes can be continuously tracked in the video. The visual motion response data of the mouse can be obtained by analyzing the tracking data.

Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that the outlines of the mouse body, ears, and tail are extracted in step 1; the specific process is:

Binarize the collected grayscale image and extract the edge contour of the mouse;

The formula for binarization is:

Among them, s ₁ is the minimum pixel value of the region of interest, s ₂ is the maximum pixel value of the region of interest; i _k,l is the pixel value of the element in the kth row and lth column of the acquired image, and bi _k,l is the pixel value of the element in the kth row and lth column of the image after binarization; when identifying the body, ears and tail of a mouse, since their colors are not necessarily the same, different s ₁ and s ₂ should also be set when extracting the contour.

On this basis, the contours of the mouse body, ears, and tail can be extracted according to their different sizes and shapes. To improve the stability of contour extraction, the three channels of the color image RGB (red, green, and blue) can be used for binarization to increase the stability of contour extraction.

The schematic diagram of the extracted contour is shown in FIG3 , where 6 is the contour of the mouse body, 7 is the contour of the mouse ear, and 8 is the contour of the mouse tail.

The other steps and parameters are the same as those in the first embodiment.

Specific implementation method three: This implementation method is different from the specific implementation method one in that the outlines of the mouse body, ears, and tail are extracted in step one; the specific process is:

When processing images captured by ordinary cameras, the extraction of the mouse outline is sometimes incomplete, which will have a certain impact on the subsequent positioning of the mouse head and the recognition of the head direction. In order to better extract the mouse outline, a depth camera can be used to collect data on the mouse.

The depth camera contains two infrared cameras, an infrared dot projector and an RGB camera. The three work together to obtain the depth information of objects perceived from the surroundings. In the visual motion reaction detection device, the distance between the mouse and the camera is fixed, and the set of points composed of the depth information of the pixels in the image within this distance range is the body part of the mouse. The outer edges of the mouse body parts form the outline of the mouse body. The mouse body outline separated in this way is closer to the real outline of the mouse than the outline extracted by the algorithm of the ordinary camera.

Use a depth camera to collect data on mice. The distance between the mouse and the depth camera meets the following requirements:

f _min ＜f ＜f _max (2)

Among them, f is the distance between the mouse and the depth camera. The distance between different parts of the mouse and the camera is different; f _min is the distance between the upper end of the mouse (the highest point of the mouse posture) and the depth camera. Since mice are of different sizes and there are basically no objects on the upper end of the mouse, the value of f _min can be appropriately smaller; f _max is the distance between the bottom end of the mouse (the lowest point of the mouse posture) and the depth camera;

According to the depth information of the pixel points in the image, a set of points within the distance range of formula (2) is selected to form the body part of the mouse; then the outer edge of the mouse body part is extracted to obtain the outline of the mouse body; the extraction of the mouse body outline can be separated from the image in this way to obtain better results.

Binarize the outline of the mouse body and extract the edge outline of the mouse;

The formula for binarization is:

Wherein, _s1 is the minimum pixel value of the region of interest, _s2 is the maximum pixel value of the region of interest; i _k,l is the pixel value of the element in the kth row and lth column of the acquired image, and bi _k,l is the pixel value of the element in the kth row and lth column of the binarized image;

The outlines of the mouse body, ears and tail are extracted respectively according to their different sizes and shapes.

The other steps and parameters are the same as those in the first embodiment.

Specific implementation method 4: This implementation method is different from specific implementation methods 2 or 3 in that, in step 2, the position of the mouse head is identified based on the contours of the mouse body, ears, and tail; the specific process is as follows:

Step 21: Preliminarily locate the mouse nose tip based on the contours of the mouse body, ears, and tail;

Step 22: performing position correction on the preliminary positioning result of the mouse nose tip obtained in step 21 to obtain the corrected mouse nose tip P _n ;

Step 23: Based on the nose tip P _n of the mouse obtained in step 22, identify the orientation of the mouse head.

The other steps and parameters are the same as those in the second or third embodiment.

Specific implementation method 5: This implementation method is different from specific implementation methods 1 to 4 in that in step 21, the nose tip of the mouse is initially located based on the contours of the mouse body, ears, and tail; the specific process is:

After obtaining the outline of the mouse body, the center of gravity G _c of the mouse body outline is calculated;

The calculation formula is as follows:

Among them, C _t is the set of coordinates of each point in the mouse body contour;

Generally, the center of gravity of the mouse's body outline is at the back of the mouse, and the point on the outline farthest from the center of gravity is the tip of the mouse's nose. However, in some cases, the center of gravity of the mouse appears at the front of the mouse, which can easily cause the tip of the mouse's nose to be mistakenly identified as being near the base of the tail. In addition, when identifying the mouse's body outline, the area around the mouse's auricle will also be identified because it has the same fur color as the rest of the body, and it is particularly easy for a small tip to form around the auricle, which can easily cause the tip of the mouse's nose to be mistakenly identified as being around the auricle.

Due to the existence of these two cases of misidentification, this method uses the contours of the mouse tail and ear to correct the preliminary positioning of the mouse nose tip when using the center of gravity to determine the mouse nose tip. Select an area around the mouse ear (such as a circle with the center of gravity of the mouse ear contour as the center and a diameter slightly larger than the ear size) so that the area around the mouse external auricle is avoided during the recognition process. For the case of misidentification near the tail root, select the point on the mouse body contour outside the area around the mouse ear that is farthest from _Gc and the point on the mouse body contour outside the area around the mouse ear that is farthest from this point as the pre-selected points of the mouse nose tip preliminary positioning point. Calculate the distance between these two points and the mouse tail and ear respectively (the distance can be calculated using the center of gravity of the mouse and ear contours). Select the point far from the mouse tail and close to the mouse ear among these two points as the pre-selected point _Ppn of the mouse nose tip, and _Ppn is the result of the preliminary positioning of the mouse nose tip.

After identifying the mouse tail and ear outlines, first calculate the area around the mouse ear:

Take the center of gravity of the mouse ear contour as the center of the circle, and the farthest distance between the mouse ear contour and the center of gravity of the mouse ear contour is l _ec . Draw two circles with a distance slightly larger than l _ec (1.1l _ec can be used in the experiment) as the radius, which are the areas around the mouse ear.

In the case of avoiding the area around the mouse ear, select the point on the mouse body contour outside the area around the mouse ear that is farthest from the center of gravity _Gc of the mouse body contour as one of the candidate points _Ppn1 for the mouse nose tip pre-selected point, and calculate the point on the mouse body contour outside the area around the mouse ear that is farthest from _Ppn1 as the second candidate point _Ppn2 for the mouse nose tip pre-selected point;

The head of the mouse is far away from the tail and close to the ear; the pre-selected point of the mouse's nose is determined based on the distance of the candidate point from the tail and the ear. The experiment found that the mouse's ears are more accurately recognized than its tail. The judgment is to use the tail for soft judgment first, and then use the ear for hard judgment. This ensures that even if the tail is misidentified, the correct result can be obtained through the ear judgment. Since mice cannot recognize ears in some cases, the process of using the tail for judgment cannot be omitted.

The distances between the points _Ppn1 and _Ppn2 and the mouse's tail and ear are calculated respectively (the distance can be calculated using the centroid of the mouse and ear contours), and the point _Ppn1 and _Ppn2 that is far from the mouse's tail and close to the mouse's ear is selected as the pre-selected point _Ppn of the mouse's nose tip, and _Ppn is the result of the preliminary positioning of the mouse's nose tip. This ensures that _Ppn will not be mistakenly identified near the tail root or the outer ear.

The result of preliminary positioning of the mouse nose tip is shown in FIG4 , where 9 is the calculated center of gravity of the mouse body contour, 10 is the selected area around the mouse ear to avoid the area around the mouse auricle during the recognition process, 11 is the circumscribed rectangle of the mouse tail contour, and 12 is the obtained mouse nose tip preselected point P _pn .

The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

Specific implementation example 6: This implementation example is different from specific implementation examples 1 to 5 in that in step 22, the preliminary positioning result of the mouse nose tip obtained in step 21 is corrected to obtain the corrected mouse nose tip P _n ; the specific process is as follows:

After obtaining the preliminary positioning result of the mouse nose tip and the center of gravity _Gc of the mouse body contour, calculate the distance _lpnc between the preliminary positioning result of the mouse nose tip and the center of gravity _Gc of the mouse body contour, select the part of the mouse body contour within a circle with the preliminary positioning mouse nose tip as the center and a radius slightly larger than _lpnc (such as _1.2lpnc ) as the front half of the mouse contour, and calculate the center of gravity of the front half of the mouse contour;

The area around the mouse ears is avoided in the mouse body contour. The part inside the circle with the mouse nose tip as the center and a radius slightly less than half of _lpnc (such as _0.4lpnc ) is selected as the head vicinity of the mouse contour. The point farthest from the center of gravity of the front half of the mouse contour in the head vicinity is selected as the nose tip _Pn of the mouse after correction.

The preselected point _Ppn of the mouse nose tip can be selected as the nose tip in general, but when the difference between the mouse head direction and the body direction is large, the position of the preselected point will deviate from the true position of the nose tip. The appropriate correction method is to select the point farthest from the center of gravity of the front half of the mouse in the area near the mouse head while avoiding the mouse ears as the new mouse nose tip. After the center of gravity of the mouse contour is changed to the center of gravity of the front half of the mouse, the degree of shortening of the center of gravity from _Ppn is greater than the degree of shortening from the true nose tip, making the nose tip the point farthest from the center of gravity of the front half. The recognition result is shown in Figure 5, 13 is the center of gravity of the front half of the mouse, 14 is the area near the mouse head used to screen the nose tip, and 15 is the final nose tip _Pn .

The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

Specific implementation example 7: This implementation example is different from specific implementation examples 1 to 6 in that, in step 23, the mouse head position is identified based on the mouse nose tip P _n obtained in step 22; the specific process is as follows:

According to the symmetry of the mouse head, we can choose the mouse nose tip _Pn as the center and draw circles with radii of _r1 and _r2 , _r1 < _r2 . The center of gravity of the part of the mouse outline inside these two circles is _Gr1 and _Gr2 respectively. The direction of the mouse head can be calculated according to the following formula:

表示小鼠头部方位的向量是从G_r2指向G_r1；in,

The vector representing the orientation of the mouse's head is from _Gr2 to _Gr1 ;

The result of mouse head direction recognition is shown in FIG6 , where 16 and 17 are circles with P _n as the center and radii r ₁ and r ₂ , 18 and 19 are the centroids _Gr1 and _Gr2 , and 20 is the mouse head orientation obtained later.

The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

Specific implementation eight: This implementation is different from any one of specific implementations one to seven in that the values of _r1 and _r2 should change with the size of the mouse. The value of _r1 is roughly initially selected as the length of the mouse's head, and the value of _r2 is roughly initially selected as twice the length from the tip of the mouse's nose to the mouse's neck. _r1 and _r2 can be continuously adjusted so that the center of gravity _Gr1 is at the center of the mouse's head, and _Gr2 is at the neck connecting the mouse's head and body.

The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

Specific implementation method 9: This implementation method is different from any one of specific implementation methods 1 to 8 in that, in step 3, based on the outlines of the mouse body, ears, and tail obtained in step 1 and the orientation of the mouse head obtained in step 2, the visual motion response of the mouse is identified; based on the previous two steps, the head orientation and movement changes of the mouse can be continuously tracked in the video. The visual motion response data of the mouse can be obtained by analyzing the tracking data. The specific process is:

Step 31: Establish the mouse head movement curve;

Step 32: Extraction of visual-motor response; the specific process is as follows:

Step 321: Preliminary extraction of visual motion response based on difference;

Step 322: Determine the visual-motor response using the least squares method based on step 321.

The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

Specific implementation method 10: This implementation method is different from specific implementation methods 1 to 9 in that the mouse head movement curve is established in step 31; the specific process is:

Based on step 2, each frame of the image will get a mouse head orientation. A coordinate system is established to convert the direction into an angle. With _Gr2 as the origin of the coordinate system, the mouse head coordinates are connected to establish the angle value of the mouse head in each frame of the image, which can be expressed as:

Wherein, θ represents the angle calculated according to the head orientation, _Gr1·x and _Gr1·y are the horizontal and vertical coordinates of _Gr1 in the pixel space coordinate system, _Gr2·x and _Gr2·y are the horizontal and vertical coordinates of _Gr2 in the pixel space coordinate system;

It is worth noting that when calculating the angle by the direction of the line connecting _Gr1 and _Gr2 , the angle can only change within the range of -180° to 180°. When the data changes near -180° and 180°, the data cannot maintain the continuity of small differential changes, which affects the subsequent judgment of the mouse's head shaking. In this regard, when the data changes from near -180° to near 180° or from near 180° to near -180°, the current value is adjusted down or up by 360° so that the current value and the value of the previous data maintain the continuity of small differential changes.

Connecting the corresponding angles of continuous videos will form the mouse's head movement curve.

The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

Specific implementation eleven: This implementation is different from specific implementations one to ten in that the step three-two-one is based on the initial extraction of visual motion response based on the difference; the specific process is:

The visual motion reaction of mice following the grating is gentle and does not change dramatically, that is, its speed change is relatively small. When measuring the change of mouse speed, it can be judged by the first-order derivative and second-order derivative curves of the change of the mouse head orientation angle. For discrete signals, it is judged by calculating the first-order difference and the second-order difference.

The formula for first-order difference is as follows:

θ′ _i = θ _i - _{θ i-1} (6)

Wherein, θ _i and θ _i-1 are the angle values of the i-th and i-1-th head orientations in the head motion curve of the mouse, and θ′ _i is the i-th first-order difference in the head motion curve of the mouse;

The formula for the second-order difference is as follows:

θ″ _i =θ′ _i -θ _i-1 (7)

Among them, θ′ _i and θ′ _i-1 are the i-th and i-1-th first-order differences in the mouse's head motion curve, and θ″ _i is the i-th second-order difference in the mouse's head motion curve;

When the mouse has a visual reaction to follow the grating, its first and second order difference values are relatively small, while in other cases, the first and second order differences are relatively large. Therefore, the first and second order differences can be used to distinguish whether the change in the orientation of the mouse's head is drastic.

In order to make the difference more clearly distinguish the severity of the change in the mouse head position, the difference can be locally integrated so that the result detected in the place with larger difference is larger, making it easier to detect the severity of the mouse head position. The formula for the local integration of the first-order difference is as follows:

Wherein, θ′ _isum is the local integral of the z ₁ consecutive first-order differences before θ′ _i (including θ′ _i ), and z ₁ is the length of the integral interval; θ′ _j is the jth first-order difference in the head movement curve of the mouse, and the value of j ranges from i+1-z ₁ to i;

When the _z1 value is increased, the difference result is more significant and the detection stability is higher. However, when the _z1 value is too large, the result obtained by integration will not be representative of the change in the current mouse head angle, which will lead to errors in the detection results. According to experience, _z1 can generally be taken as 5.

Similarly, the formula for the second-order difference local integral is as follows:

Wherein, θ″ _j is the jth second-order difference in the head motion curve of the mouse, θ″ _isum is the local integral of z ₂ consecutive second-order differences before θ″ _i (including θ″ _i ), and according to experience, it is generally taken as 2;

When the local integral of the first-order difference is greater than the first-order integral threshold or the local integral of the second-order difference is greater than the second-order integral threshold, it is recorded as a non-smooth motion;

When the number of non-smooth movements is greater than the number threshold, the interval (the interval corresponding to the non-smooth movement) is recorded as the non-smooth movement interval, and the non-smooth interval part of the entire mouse head movement curve is eliminated to obtain the gentle interval. The gentle interval contains the part of the mouse's visual motion response, completing the preliminary extraction of the visual motion response area;

For example, the first-order integration threshold is 14°, the second-order integration threshold is 7°, and the number threshold is 3.

The first-order difference eliminates the steeper part of the curve, the second-order difference eliminates the curve with particularly drastic fluctuations, and the remaining part of the curve is the part that contains the mouse's visual motion response.

Figures 7a and 7b show the first and second order difference diagrams of the mouse head angle curve. When the mouse head moves violently, its differential data will have obvious peaks. By segmenting the peaks, the interval containing the visual motion response can be extracted.

The other steps and parameters are the same as those in Specific Implementation Examples 1 to 10.

Specific implementation 12: This implementation is different from specific implementations 1 to 11 in that the least square method is used in step 322 to determine the visual motion response based on step 321; the specific process is:

The speed of the mouse's visual reaction to the grating should be within a certain range. By analyzing the speed in the area where the mouse's head position changes more slowly, we can determine whether the mouse has a visual reaction. The speed of the mouse's head position change can be expressed by the slope.

The least squares method is used to fit the curve of the visual motion response area extracted in step 321. The slope and correlation coefficient of the fitted straight line can be used to evaluate the speed of the mouse head position change. The specific process is as follows:

For n observations (x ₁ , y ₁ ), (x 2 , y 2 ), ..., (x n , _yn ) among the points on the curve of the visual response area extracted in step 321 (each group is a point on the curve of the visual response area extracted in step ₃₂₁ ), the steps of calculating the slope _by the least squares method are as follows _:

The mean value of the independent variable is:

The mean value of the dependent variable is:

The calculated slope is:

Where x _k is the abscissa of the k-th observation, and y _k is the ordinate of the k-th observation;

The calculated correlation coefficient is:

Where x _k is the abscissa of the k-th observation, and y _k is the ordinate of the k-th observation;

The obtained flat interval is divided into a series of small intervals of fixed length (such as 30 points), and there is some overlap between the small intervals (such as 20 points overlap, such as

)；

);

The slope between the cells is calculated by formula (10), (11), and (12), and the correlation coefficient is calculated by formula (13). The slope and the correlation coefficient can be used to determine whether the speed of the change of the mouse head orientation angle within the cell is within the range of the visual motion response.

If the slope and the correlation coefficient are within the range of the visual-motor response, it is judged that the visual-motor response has occurred in the small interval;

If the slope and the correlation coefficient are not within the range of the optokinetic response, it is judged that no optokinetic response occurs in the small interval;

The small intervals that are continuously judged as visual motion reactions are combined into large intervals, and each large interval is judged as an visual motion reaction. In this way, the number of visual motion reactions of mice and the duration of visual motion reactions can be determined;

The range of the optokinetic response is that the slope range is greater than or equal to -0.65 and less than or equal to -0.1, and the correlation coefficient range is greater than -1 and less than or equal to -0.8 (or the slope range is greater than or equal to 0.1 and less than or equal to 0.65, and the correlation coefficient range is greater than or equal to 0.8 and less than 1).

The number of times the optokinetic responses were judged in the video and the duration of the optokinetic responses were recorded.

The curve in Figure 8 is the identified visual-motor response state of the mouse, the number of visual responses is 7, and the total duration of these 7 times is 13.87 seconds. The number of visual-motor responses and the duration of the visual-motor responses in the video are recorded.

The automatic recognition method for visual-motor response of mice established by the present invention can provide recognition data in real time during the experiment, including the number of visual-motor responses of mice, the duration, the synchronization rate with the stripe speed and other data.

The other steps and parameters are the same as those in Specific Implementations 1 to 11.

The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims (4)

1. the mouse optokinetic response automatic recognition method based on image technology, is characterized in that: described method concrete process is: Step 1, extracting the contours of the mouse body, ears, and tail; Step 2, based on the contours of the mouse body, ears, and tail, identify the head position of the mouse; Step 3, based on the contours of the mouse body, ears, and tail obtained in step 1 and step 2 to obtain the head position of the mouse, and identify the optokinetic response of the mouse; In the described step 2, based on the contours of the mouse body, ears and tail, the head position of the mouse is recognized; the specific process is: Step 21, based on the contours of the mouse body, ears, and tail, initially locate the nose tip of the mouse; Step 22, correcting the position of the mouse nose tip preliminary positioning result obtained in step 21, and obtaining the corrected nose tip P _n of the mouse; Step two and three, based on the nose tip P _n of the mouse obtained in step two and two, identify the head position of the mouse; In the step 21, the nose tip of the mouse is preliminarily located based on the contours of the mouse body, ears, and tail; the specific process is: After obtaining the outline of the mouse body, calculate the center of gravity G _c of the mouse body outline; The calculation formula is as follows:

Wherein, C _t is the set of coordinates of each point in the mouse body outline; After identifying the mouse tail and ear contours, first calculate the area around the mouse ears: Taking the center of gravity of the outline of the mouse ear as the center of the circle, the farthest distance between the outline of the mouse ear and the center of gravity of the outline of the mouse ear is l _ec . Draw a circle with a radius slightly greater than l _ec to form two circles, which are the area around the mouse ear ; In the case of avoiding the area around the mouse ear, select the point farthest from the center of gravity _Gc of the mouse body outline on the mouse body outline outside the area around the mouse ear as one of the candidate points P for the pre-selected point of the mouse nose tip _pn1 , calculating the point on the body outline of the mouse outside the area around the mouse ear that is farthest from P _pn1 as the second candidate point P _pn2 of the pre-selected point of the mouse nose tip; Calculate the distances between points P _pn1 and P _pn2 and the tail and ear of the mouse respectively, and select points P _pn1 and P _pn2 that are far away from the tail of the mouse and close to the ear of the mouse as the pre-selected points P _pn and P _pn of the mouse nose It is the result of the preliminary positioning of the nose tip of the mouse; In the step 22, the position correction is performed on the result of the preliminary positioning of the nose tip of the mouse obtained in the step 21, and the corrected nose tip P _n of the mouse is obtained; the specific process is as follows: After obtaining the result of the preliminary positioning of the mouse nose tip and the center of gravity G _c of the mouse body contour, calculate the distance l _pnc between the result of the preliminary positioning of the mouse nose tip and the center of gravity G _c of the mouse body contour, and select the mouse In the body contour, the nose tip of the mouse obtained by preliminary positioning is the center, and the part in the circle with a radius greater than 1 _pnc length is used as the first half of the mouse contour, and the center of gravity of the first half of the mouse contour is calculated; In the body contour of the selected mouse, avoid the area around the ear of the mouse, and take the nose tip of the mouse as the center, and the inner part of the circle with a radius less than half the length of l _pnc as the area near the head of the mouse outline, and select the area near the head The farthest point from the center of gravity of the front half of the mouse profile is taken as the nose tip P _n of the mouse after correction; In the step two or three, based on the nose tip P _n of the mouse obtained in step two or two, the head position of the mouse is identified; the specific process is: Select the point P _n of the nose tip of the mouse as the center of the circle, draw circles with radii r ₁ and r ₂ respectively, r ₁ < r ₂ , and the centers of gravity of the mouse contours inside these two circles are G _r1 and G _r2 respectively, The direction of the mouse head is calculated according to the following formula:

in,

The vector representing the head orientation of the mouse is from G _r2 to G _r1 ; In the step 3, the outline of the mouse body, ears and tail obtained in the step 1 and the head position of the mouse are obtained in the step 2, and the optokinetic response of the mouse is identified; the specific process is: Step 31, establishing the mouse head movement curve; Step 32, the extraction of optokinetic response; the specific process is: Step 321. Preliminary extraction of optokinetic response based on difference; Step 322, based on step 321, adopting the least squares method to determine the optokinetic response; Establish mouse head motion curve in described step 31; Concrete process is: Based on step 2, each frame of image will get a mouse head orientation, establish a coordinate system, convert the direction into an angle, take G _r2 as the coordinate origin, connect the coordinates of the mouse head, and establish the mouse head position in each frame of image The angle value of the head, the formula is expressed as:

Among them, θ represents the angle calculated according to the head orientation, G _r1 .x and G _r1 .y are the horizontal and vertical coordinates of G _r1 in the pixel space coordinate system respectively, G _r2 .x and G _r2 .y are the horizontal and vertical coordinates of G _r2 in the pixel space coordinate system, respectively Horizontal and vertical coordinates of the pixel space coordinate system; Connect the angles corresponding to the continuous videos to form the mouse's head movement curve; In the step 321, the optokinetic response based on the difference is initially extracted; the specific process is: The formula for the first difference is as follows: θ′ _i ＝θ _i -θ _i-1 (6) Among them, θ _i and θ _i-1 are the angle values of the i-th and i-1 head orientations in the head movement curve of the mouse, and θ′ _i is the i-th one in the head movement curve of the mouse. order difference; The formula for the second order difference is as follows: θ″ _i = θ′ _i -θ′ _i-1 (7) Among them, θ′ _i and θ′ _i-1 are the i-th and i-1 first-order differences in the head motion curve of the mouse, and θ″ _i is the i-th second-order difference in the head motion curve of the mouse difference; The formula for the local integral of the first difference is as follows:

Among them, θ′ _isum is the local integral of z ₁ consecutive first-order differences before θ′ _i , and z ₁ is the interval length of the integral; θ′ _j is the jth first-order difference in the head movement curve of the mouse; Likewise, the formula for the partial integral of the second difference is as follows:

Wherein, θ″ _j is the jth second-order difference in the head motion curve of the mouse, and θ″ _isum is the local integral of the continuous z ₂ second-order differences before θ″ _i ; When the local integral of the first-order difference is greater than the first-order integral threshold or the local integral of the second-order difference is greater than the second-order integral threshold, it is recorded as a non-smooth motion; When the number of non-smooth movements is greater than the number threshold, this interval is recorded as the non-smooth movement interval, and the portion of the non-smooth interval in the entire mouse head movement curve is removed to obtain a gentle interval, which includes the optokinetic response of the mouse In the part, complete the preliminary extraction of optokinetic response area; In the step 322, the optokinetic response is determined based on the step 321 using the least squares method; the specific process is: Use the least squares method to fit the curve of the optokinetic response area extracted in step 321, and the slope of the fitted line obtained is used to evaluate the speed of the mouse head orientation change; the specific process is: For the n observed values in the points on the curve of the optokinetic response region extracted in step 321, the steps for calculating the slope by the least squares method are as follows: The mean of the independent variable is:

The mean of the dependent variable is:

The calculated slope is:

Among them, x _k is the abscissa of the k-th observation value, and y _k is the ordinate of the k-th observation value; Calculate the slope of the gentle interval by formula (10), (11), (12), and determine whether the speed of the mouse head orientation angle change is within the scope of the optokinetic response by the size of the slope; If the magnitude of the slope is within the scope of optokinetic response, it is judged that optokinetic response has occurred; If the magnitude of the slope is not within the scope of optokinetic response, it is judged that optokinetic response has not occurred; From this, the number of optokinetic responses and the duration of optokinetic responses were determined. 2. according to the described mouse optokinetic response automatic identification method based on image technology of claim 1, it is characterized in that: extract the outline of mouse body, ear, tail in the described step 1; Concrete process is: Binarize the collected grayscale image to extract the edge contour of the mouse; The formula for binarization is:

Among them, s ₁ is the minimum value of the pixel value of the region of interest, s ₂ is the maximum value of the pixel value of the region of interest; i _k,l is the pixel value of the element in the kth row and lth column in the collected image, bi _{k, l} is the pixel value of the kth row and lth column element in the image after binarization; According to the different size and shape structures of the mouse body, ears, and tail, the contours of the mouse body, ears, and tail are respectively extracted. 3. according to the described mouse optokinetic response automatic identification method based on image technology of claim 1, it is characterized in that: extract the outline of mouse body, ear, tail in the described step 1; Concrete process is: Use the depth camera to collect data from the mouse, and the distance range between the mouse and the depth camera satisfies: f _min < f < f _max (2) Among them, f is the distance from the mouse to the depth camera, f _min is the distance from the upper end of the mouse to the depth camera; f _max is the distance from the bottom end of the mouse to the depth camera; Select the set of points formed between the distance ranges of formula (2) to form the body part of the mouse; then extract the outer edge of the body part of the mouse to obtain the outline of the mouse body; Binarize the outline of the mouse body to extract the edge outline of the mouse; The formula for binarization is:

Among them, s ₁ is the minimum value of the pixel value of the region of interest, s ₂ is the maximum value of the pixel value of the region of interest; i _k,l is the pixel value of the element in the kth row and lth column in the collected image, bi _{k, l} is the pixel value of the kth row and lth column element in the image after binarization; According to the different size and shape structures of the mouse body, ears, and tail, the contours of the mouse body, ears, and tail are respectively extracted. 4. according to claim 2 or 3 described based on the mouse optokinetic response automatic identification method of image technology, it is characterized in that: the value of described r ₁ is initially selected as the length of the mouse head, and the value of r ₂ is initially selected as Twice the length from the nose tip of the mouse to the neck of the mouse, constantly adjust r ₁ and r ₂ so that the obtained center of gravity G _r1 is at the center of the head of the mouse, and G _r2 is at the neck connecting the head and body of the mouse.

Images