CN115797451A - Acupuncture point identification method, device and equipment and readable storage medium - Google Patents

Abstract

The application relates to an acupoint identification method, in particular to the technical field of automatic identification. The method comprises the following steps: acquiring three-dimensional reconstruction data of the surface of a human body; classifying and identifying the three-dimensional reconstruction data of the human body surface to obtain classification data of the human body region; carrying out pose registration on the classification data of the human body region and the data of the virtual human body acupuncture point data, and mapping acupuncture point positions in the three-dimensional reconstruction data of the human body surface; the virtual human body acupuncture point data is pre-established according to a human body acupuncture point model; and displaying the image corresponding to the three-dimensional reconstruction data of the human body surface and the acupuncture point position in display equipment. Based on the scheme, the acupuncture point identification difficulty is simplified.

Description

A method, device, equipment and readable storage medium for identifying acupuncture points

technical field

The present application relates to the technical field of image recognition, and in particular to an acupuncture point recognition method, device, equipment and readable storage medium.

Background technique

The traditional method is to find acupuncture points based on body surface anatomy and joints, or to find acupuncture points through the finger measurements of one's own fingers.

For non-professionals, acupoints are not easy to find, so automatic acupoint recognition is particularly important. In the prior art, technicians have developed a virtual simulation system software based on Web-based acupuncture facial acupoint positioning, marking acupoints on static face pictures, which reduces the difficulty of acupoint identification to a certain extent.

However, this method cannot dynamically display acupuncture points following the movement of the face.

Contents of the invention

The present application provides an acupoint identification method, device, equipment and readable storage medium, and the technical solution is as follows.

On the one hand, provide a kind of acupuncture point recognition method, described method comprises:

Obtain 3D reconstruction data of human body surface;

classify and identify the three-dimensional reconstruction data of the human body surface, and obtain classified data of human body regions;

performing pose registration on the human body area classification data and the virtual human acupoint data, and mapping the acupoint positions in the three-dimensional reconstruction data of the human body surface; the virtual human acupoint data is pre-established according to the human acupoint model;

The image corresponding to the three-dimensional reconstruction data of the human body surface and the positions of the acupoints are displayed on a display device.

In yet another aspect, an acupoint identification device is provided, the device comprising:

The data acquisition module is used to obtain the three-dimensional reconstruction data of the human body surface;

A classification identification module, configured to classify and identify the three-dimensional reconstruction data of the human body surface, and obtain classification data of human body regions;

The acupuncture point mapping module is used to perform pose registration on the human body area classification data and the virtual human acupuncture point data, and map the acupuncture point positions in the three-dimensional reconstruction data of the human body surface; the virtual human body acupuncture point data is pre-established according to the human body acupuncture point model;

The display module is configured to display the image corresponding to the three-dimensional reconstruction data of the human body surface and the positions of the acupuncture points on a display device.

In a possible implementation manner, the three-dimensional reconstruction data of the human body surface is a first point set obtained by photographing the human body surface and performing three-dimensional reconstruction;

The step of classifying and identifying the three-dimensional reconstruction data of the human body surface to obtain classified data of human body regions includes:

Obtain the initial relative position of the camera and the human body; the initial relative position is captured by the camera and detected by face feature points;

Combining the initial relative position of the camera and the human body and the movement trajectory of the camera collected by the gyroscope and the accelerometer, the first relative position of the current camera and the human body is obtained in real time;

According to the initial relative position and the first relative position, the current body region is acquired.

In a possible implementation manner, the acquiring the current body region according to the initial relative position and the first relative position includes:

Extracting the contour texture of the image corresponding to the three-dimensional reconstruction data of the human body surface;

According to the first point set and the outline texture, combined with the initial relative position and the first relative position, match with the human body area information in the virtual human acupuncture point data, and the part with the highest matching degree is the current human body area.

In a possible implementation manner, the pose registration of the human body area classification data and the virtual human body acupoint data, and mapping the position of the acupoints in the three-dimensional reconstruction data of the human body surface includes:

roughly registering the first point set and the second point set of the virtual human acupoint data according to the human body area classification data;

Through rigid registration, fine registration is performed on the first point set after rough registration and the second point set of virtual human acupoint data after rough registration.

In a possible implementation manner, the rough registration of the first point set and the second point set of virtual human acupoint data according to human body region classification data includes:

Extracting a subset of the first point set and a subset of the second point set for each human body area according to the human body region classification data;

Extracting the convex hulls of each subset of the first point set, the convex hull of each subset of the second point set, and the human body region patches corresponding to each convex hull;

Coarse registration is performed based on the convex hull and the human body area patch.

In a possible implementation manner, the fine registration of the first point set after rough registration and the second point set of virtual human acupoint data after rough registration through rigid registration includes:

Set a spherical target area with the acupuncture point position in the subset of the second point set as the center and the first threshold as the radius;

extracting the three-dimensional curvature distribution and contour distribution of the acupoint positions in the subset of the second point set, and weighting the dot product to obtain the first spatial feature map;

extracting a second spatial feature map corresponding to a subset of the first point set according to the spherical target area;

Set the sampling window based on the second spatial characteristic map, and traverse the first spatial characteristic map through the sampling window, and obtain the target sampling window with the smallest measure between the first spatial characteristic map, as the acupuncture point position in the subset of the first point set .

In a possible implementation manner, the surface distance between the positions of the acupuncture points in the subset of the first point set is acquired by a projection method;

Perform contour recognition on the image corresponding to the three-dimensional reconstruction data of the human body surface, obtain each closed contour, and calculate the distance between the midline of each closed contour and the position of the acupuncture point in the closed contour;

Combining the surface distance and the distance between the midline of the closed contour and the position of the acupuncture point in the closed contour, compare the position of the acupuncture point with the recorded position corresponding to the knowledge map, and if the comparison result exceeds the second threshold, then display The device displays that the position of the acupoint is marked incorrectly; the knowledge graph is pre-built based on the existing acupoint knowledge.

In another aspect, a computer device is provided, the computer device includes a processor and a memory, at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to realize the above-mentioned acupoint identification method.

In yet another aspect, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned acupoint identification method.

In yet another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned acupoint identification method.

The technical solution provided by this application may include the following beneficial effects:

This application first obtains the three-dimensional reconstruction data of the human body surface; classifies and identifies the three-dimensional reconstruction data of the human body surface, and obtains the classification data of the human body area; performs pose registration on the classification data of the human body area and the virtual human acupuncture point data, and puts them in the three-dimensional reconstruction data of the human body surface The location of the acupuncture points is mapped; the virtual human body acupuncture point data is pre-established according to the human body acupuncture point model; the image corresponding to the three-dimensional reconstruction data of the human body surface and the acupuncture point positions are displayed on the display device. The above scheme simplifies the difficulty of acupoint recognition by performing pose registration on the classification data of the human body area and the virtual human acupoint data, and mapping the acupoint positions in the three-dimensional reconstruction data of the human body surface.

Description of drawings

In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings that need to be used in the description of the specific embodiments or prior art. Obviously, the accompanying drawings in the following description The drawings are some implementations of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic structural diagram of an acupoint recognition system according to an exemplary embodiment.

Fig. 2 is a flow chart of a method for identifying acupuncture points according to an exemplary embodiment.

Fig. 3 is a flow chart of a method for identifying acupuncture points according to an exemplary embodiment.

Fig. 4 shows an example diagram of connection between a tablet computer and a camera according to the embodiment of the present application.

Fig. 5 is a structural block diagram of an acupuncture point recognition device according to an exemplary embodiment.

Fig. 6 is a structural block diagram of a computer device according to an exemplary embodiment.

Detailed ways

The technical solutions of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Apparently, the described embodiments are some of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

In the description of the embodiments of the present application, the term "corresponding" may indicate that there is a direct or indirect correspondence between the two, or that there is an association between the two, or that it indicates and is indicated, configuration and is configuration etc.

In the embodiment of this application, "predefinition" can be realized by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in devices (for example, including terminal devices and network devices). The implementation method is not limited.

Fig. 1 is a schematic structural diagram of an acupoint recognition system according to an exemplary embodiment. The acupoint recognition system includes a data processing device 110 , an image acquisition device 120 and a display device 130 .

Optionally, the image acquisition device 120 includes a data storage device. When the image acquisition device collects the target image and obtains target image data, the image data can be stored in the data storage device. For example, the image acquisition device may be a SLR camera, a digital camera, a mobile phone or a depth camera.

Optionally, the data processing device 110 may be a computer device with relatively high computing power, and the data processing device is used to analyze the collected target image data, so as to obtain the characteristics of the target image data.

Optionally, the data processing device 110 may be a terminal device installed with image analysis software. When the terminal device receives an instruction to analyze the target image data, the terminal device may read the data from the data memory in the image acquisition device 120. The corresponding target image data is obtained, and the target image data is analyzed, so as to obtain the characteristics of the target image data.

Optionally, the data processing device 110 can also be a server equipped with image analysis software, the image acquisition device can be a terminal device, and after the terminal device collects the target image data, it can transmit the target image data to the server In order to complete the characteristic analysis of the target image data.

Optionally, after the characteristic analysis of the target image data is completed, the display device 130 may display the characteristic analysis result.

Optionally, the data processing device 110, the image acquisition device 120, and the display device 130 may be connected through a wired or wireless network.

Optionally, the above server may be a server cluster or a distributed system composed of multiple physical servers, and may also provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware Cloud server for computing services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

Optionally, the system may also include a management device, which is used to manage the system (such as managing the connection status between each module and the server, etc.), and the management device and the server are connected through a communication network. Optionally, the communication network is a wired network or a wireless network.

Optionally, the aforementioned wireless network or wired network uses standard communication technologies and/or protocols. The network is typically the Internet, but can be any other network including, but not limited to, any combination of local area networks, metropolitan area networks, wide area networks, mobile, wired or wireless networks, private networks, or virtual private networks. In some embodiments, data exchanged over a network is represented using techniques and/or formats including Hypertext Markup Language, Extensible Markup Language, and the like. In addition, all or some links may be encrypted using conventional encryption techniques such as Secure Sockets Layer, Transport Layer Security, Virtual Private Network, Internet Protocol Security, etc. In some other embodiments, customized and/or dedicated data communication technologies may also be used to replace or supplement the above data communication technologies.

Fig. 2 is a flow chart of a method for identifying acupuncture points according to an exemplary embodiment. The method is performed by a computer device, which may be a data processing device 110 as shown in FIG. 1 . As shown in Figure 2, the acupuncture point identification method may include the following steps:

Step 201, acquiring 3D reconstruction data of the human body surface.

Three-dimensional reconstruction is to use a camera to shoot real-world objects and scenes, and process them through computer vision technology to obtain a three-dimensional model of the object.

Optionally, the human body is photographed by a camera to obtain a two-dimensional image of the human body, and depth information is restored according to information such as texture distribution of the two-dimensional image of the human body, and then three-dimensional reconstruction data of the human body surface is obtained.

Optionally, the 3D information of the human body is directly acquired through the depth camera.

Optionally, the three-dimensional reconstruction data of the human body surface can also be obtained by structured light method and laser scanning method.

In step 202, classify and identify the three-dimensional reconstruction data of the human body surface, and obtain classified data of human body regions.

Since the 3D reconstruction data of the human body surface is a whole 3D data, the 3D reconstruction data of the human body surface can be classified and identified firstly, and the 3D reconstruction data of the human body surface can be divided into different human body regions according to each area of the human body surface, so that the The 3D reconstruction data of the human body surface are registered with the virtual human acupuncture point data.

After obtaining the 3D reconstruction data of the human body surface, divide the human body surface into different areas, and classify and identify the 3D reconstruction data of the human body surface according to the divided areas, so as to divide the 3D reconstruction data of the human body surface into corresponding different areas, It is convenient for the next pose registration. The different divided areas can be adjusted according to actual needs, for example, divided into head, torso, arms, and legs, or divided into upper body and lower body.

In step 203, pose registration is performed between the classification data of the human body region and the data of virtual human acupoints, and the positions of the acupoints are mapped in the three-dimensional reconstruction data of the human body surface; the virtual human acupoint data is pre-established according to the human acupoint model.

The virtual human acupoint data is pre-established according to the existing human acupoint model, which is a human body model with human acupoints marked on its surface. The virtual human body data may also include descriptive text of each acupuncture point, such as name, location and number.

Therefore, after the pose registration of the human body region classification data and the virtual human acupoint data, the corresponding acupoint positions in the human body region classification data can be mapped out by an algorithm according to the virtual human acupoint data.

In step 204, the image corresponding to the three-dimensional reconstruction data of the human body surface and the position of the acupoint are displayed on a display device.

Optionally, in the display device, the image of the human body captured by the camera is displayed, and the position of the acupoint, the name of the acupoint, and the information of the acupoint are superimposed on the image, so that the user can view it.

To sum up, this application first obtains the 3D reconstruction data of the human body surface; classifies and recognizes the 3D reconstruction data of the human body surface, and obtains the classification data of the human body area; performs pose registration on the classification data of the human body area and the data of virtual human acupuncture points, and The position of the acupuncture point is mapped in the surface three-dimensional reconstruction data; the virtual human body acupuncture point data is pre-established according to the human body acupuncture point model; the image corresponding to the three-dimensional reconstruction data of the human body surface and the position of the acupuncture point are displayed on the display device. The above scheme simplifies the difficulty of acupoint recognition by performing pose registration on the classification data of the human body area and the virtual human acupoint data, and mapping the acupoint positions in the three-dimensional reconstruction data of the human body surface.

Fig. 3 is a flow chart of a method for identifying acupuncture points according to an exemplary embodiment. The method is executed by a computer device, which may be a data processing device in the acupoint recognition system as shown in FIG. 1 . As shown in Figure 3, the acupuncture point identification method may include the following steps:

In step 301, the three-dimensional reconstruction data of the human body surface is obtained.

Optionally, the 3D reconstruction data of the human body surface is the first point set obtained by photographing the human body surface and performing 3D reconstruction.

For example, the surface of the human body is photographed by a depth camera to obtain point cloud data of the surface of the human body, and then 3D reconstruction is performed based on the point cloud data of the surface of the human body to obtain 3D reconstruction data of the human body surface.

In step 302, classify and identify the three-dimensional reconstruction data of the human body surface, and obtain classified data of human body regions.

Since the 3D reconstruction data of the human body surface is a whole 3D data, the 3D reconstruction data of the human body surface can be classified and identified firstly, and the 3D reconstruction data of the human body surface can be divided into different human body regions according to each area of the human body surface, so that the The 3D reconstruction data of the human body surface are registered with the virtual human acupuncture point data. For example, each area on the surface of the human body may be the left side of the head, the right side of the head, the left side of the chest, the right side of the chest, etc., and the human body is distinguished according to front, back, left, and right.

The virtual human acupoint data is pre-established according to the existing human acupoint model, which is a human body model with human acupoints marked on its surface. The virtual human body data may also include descriptive text of each acupuncture point, such as name, location and number.

Optionally, a camera is used to shoot the acupoint model of the human body to obtain an image of the acupoint model of the human body, and perform three-dimensional reconstruction based on the image of the acupoint model of the human body to obtain data of the virtual acupoint point of the human body.

After classifying and identifying the 3D reconstruction data of the human body surface, in the actual application scenario, when the acupuncture point information of a certain area of the human body is needed, it is necessary to move the camera to point at the area of the human body to shoot and identify the acupuncture points, so it is necessary to know the current What exactly is the body area.

Optionally, the tablet is rigidly connected to the camera, and the camera faces the back of the tablet. The tablet is equipped with a gyroscope and an accelerometer, which means that the current position and location of the camera can be obtained through the gyroscope and accelerometer in the tablet. The distance and relative bearing of the camera's initial position. Fig. 4 shows an example diagram of connection between a tablet computer and a camera according to the embodiment of the present application. As shown in Figure 4, the cameras include depth cameras and visible light cameras.

Furthermore, a certain area of the human body can be used as a reference, and the initial relative position between the camera and the human body can be obtained first, and then the current human body area can be obtained through calculation.

Optionally, the initial relative position between the camera and the human body is obtained; the initial relative position is obtained by taking pictures with the camera and detecting facial feature points. Since the human face is an easily identifiable area in the human body, the human face can be selected as a benchmark. Adjust the position of the camera, shoot the human body and recognize it through face feature point detection until the face is recognized, and take the relative position between the camera and the human body at this time as the initial relative position between the camera and the human body.

For example, the camera is a depth camera, and the top of the head is captured by the depth camera to obtain surface point cloud data on the top of the head. Click the collected area on the screen, and call the surface classification model built based on PointNet's deep neural network. This region is identified as the head.

Further, the current first relative position between the camera and the human body is acquired in real time in combination with the initial relative position of the camera and the human body and the movement track of the camera collected by the gyroscope and the accelerometer.

Further, the current body region is acquired according to the initial relative position and the first relative position. First extract the contour texture of the image corresponding to the 3D reconstruction data of the human body surface, and then combine the initial relative position and the first relative position with the human body area information in the virtual human body acupoint data according to the first point set and the contour texture. Matching, the area with the highest matching degree is the current human body area.

For example, the first relative position of the current camera and the human body is roughly estimated in combination with the initial relative position of the camera and the human body and the movement track of the camera collected by the gyroscope and the accelerometer. Further, the image corresponding to the 3D reconstruction data of the human body surface can be a visible light image, the contour texture of the image is extracted, and the extracted contour texture is matched with the 3D reconstruction data of the human body surface after classification and recognition, and the matching degree The highest area is identified as the current human body area captured by the current camera.

Step 303 , performing pose registration on the human body area classification data and the virtual human acupoint data, and mapping the acupoint positions in the three-dimensional reconstruction data of the human body surface.

Through step 302, the classification data of the human body region and the human body region currently captured by the camera can be obtained. Next, how to obtain the acupuncture points in the human body region currently captured by the camera will be described. Optionally, since the virtual human acupoint data contains the names and positions of each acupoint on the human body, the body area classification data and the virtual human acupoint data are subjected to pose registration, and then the corresponding human body can be obtained in the three-dimensional reconstruction data of the human body surface. The name and location of each acupuncture point.

Optionally, rough registration is performed on the first point set and the second point set of the virtual human acupoint data according to the classification data of human body regions.

Further, according to the human body region classification data, for each human body region, a subset of the first point set and a subset of the second point set are respectively extracted. The three-dimensional reconstruction data of the human body surface (that is, the first point set) is divided into multiple subsets according to the classification data of human body regions, and then the second point set is also correspondingly divided into multiple subsets according to the classification data of human body regions.

Further, the convex hulls of each subset of the first point set, the convex hulls of each subset of the second point set, and the human body region patches corresponding to each convex hull are extracted. The convex hull is the smallest convex polygon including all points in a point set. In the embodiment of the present application, the convex hull is the smallest convex set including all points in the target point cloud. By extracting the convex hulls of each subset of the first point set and each subset of the second point set, the surface of the first point set and the second point set can be approximated as a convex polyhedron, and each face of the convex polyhedron is the human body Region patches.

Further, rough registration is performed based on the convex hull and the human body area patch. Coarse registration is performed based on the convex hull of the convex polyhedron and the human body area patch corresponding to the first point set and the convex hull of the convex polyhedron and the human body area patch corresponding to the second point set.

For example, the convex hull of the first point set A and the convex hull of the second point set B are constructed, and the vertex data a of the convex hull of the first point set and the vertex data b of the convex hull of the second point set are respectively obtained. Calculate the centers, long and short diameters, and surface curvatures of a and b, and then adjust the poses of a and b according to the standards of center coincidence, long and short diameter coincidence, and the same curvature distribution, to obtain the pose transformation matrix M of a and b, and apply M to to A and B. Finally, AA-ICP or 4PCS or algorithms based on t-distribution mixed model registration are used to further register A and B to complete the rough registration.

Further, through rigid registration, fine registration is performed on the first point set after rough registration and the second point set of virtual human acupoint data after rough registration. Rigid registration refers to the registration in which the distance between any two points of the registration object remains unchanged before and after transformation.

Optionally, a spherical target area is set with the acupuncture point position in the subset of the second point set as the center and the first threshold as the radius;

Further, extract the three-dimensional curvature distribution and contour distribution of the acupoint positions in the subset of the second point set, and perform weighted point multiplication to obtain the first spatial feature map;

Further, a second spatial feature map corresponding to a subset of the first point set is extracted according to the spherical target area;

Further, the sampling window is set based on the second spatial feature map, and the first spatial feature map is traversed through the sampling window, and the target sampling window with the smallest measure between the first spatial feature map is obtained as a subset of the first point set Acupoint location.

Optionally, mutual information or Euclidean distance is used as a measure between the sampling window of the second spatial feature map and the sampling window of the first spatial feature map. Optionally, the center position of the target sampling window is used as the acupuncture point position in the subset of the first point set.

For example, taking the acupoint position X on the second point set B as an example, a spherical area with a radius of 10 mm is constructed in space with X as the center, and the corresponding point set y contained in the spherical area on A is obtained. Extract the three-dimensional curvature distribution and contour distribution of 20mm around X, and perform weighted point multiplication on the two data, in which the curvature distribution weight is 0.6, and the contour distribution weight is 0.4 to form the first spatial feature map P, and extract the corresponding second spatial feature map Q of y . Based on Q, set the sampling window according to the same scale, traverse P, and calculate the distance measure (such as mutual information, Euclidean distance, etc.) between the sampling window and P at each position, and find the window with the smallest measure, then At this time, the center position corresponding to the window is the maximum possible position Y of the corresponding acupuncture point in A. For each acupoint on B, repeatedly find the maximum possible position of the corresponding acupoint in A, and obtain the estimated position {Y} of each acupoint on A.

Step 304, displaying the image corresponding to the three-dimensional reconstruction data of the human body surface and the position of the acupoint on the display device.

For example, the image of the human body captured by the camera is a visible light camera video image, and the estimated acupoint point, the identified contour, the distance between the acupoint and the contour, and the distance between the identified acupoint and the acupoint are superimposed on the image , the distance between the recognized acupoint and the midline, and the midline and the position description information corresponding to the acupoint involved in the current image.

Step 305, comparing the location of the acupoint with the recorded location corresponding to the knowledge graph, and if the comparison result exceeds the second threshold, displaying that the location of the acupoint is incorrectly marked on the display device.

It should be noted that the knowledge map is pre-constructed based on the existing acupoint knowledge. For example, use entity extraction or keyword recognition to extract fields related to location, edge, distance, and acupoints in the acupoint description information, as well as related specific scale information, and construct a knowledge map between each acupoint, measurement area, measurement value, and acupoint location .

Optionally, the surface distance between the positions of various acupoints in the subset of the first point set is acquired through a projection method.

Further, contour recognition is performed on the image corresponding to the three-dimensional reconstruction data of the human body surface, each closed contour is obtained, and the distance between the midline of each closed contour and the position of the acupoint in the closed contour is calculated.

Further, in combination with the surface distance and the distance between the midline of the closed contour and the acupoint position in the closed contour, the acupoint position is compared with the recorded position corresponding to the knowledge map through an algorithm, and if the comparison result exceeds the second threshold, the display device It shows that the position of the acupoint is marked incorrectly, and further, the position of the acupoint with the wrong mark can be readjusted.

For example, for {Y}, for the data that already has predicted acupoints, the surface distance between multiple acupoints on the same plane is calculated based on the projection method, that is, the average shortest distance between the connecting line of two acupoint points and the point set is used as the projection plane , to calculate the length of the intersection line between the projected plane and the surface side of the point set. Identify the outline in the video image of the visible light camera, get the outline of the top of the head, calculate the long and short axes, and get the midline similar to the long axis, and calculate the distance between each acupoint in {Y} and the midline. Extract the corresponding position description information from the knowledge map according to the number of acupuncture points, and automatically compare it with the distance and contour of the automatic recognition. When there is a discrepancy of more than 20% in the value deviation rate, it will output the identification of the positioning error. The order of comparison is to first compare the description of a single acupoint, and then compare the description of the correlation of multiple acupoints. For the acupoints identified by output positioning errors, adjust the position according to the called description, but when the position deviation exceeds 20mm, keep the original position unchanged. After the adjustment of all acupoints is completed, the real acupoint position {Y'} is obtained. Coordinate transformation is performed on {Y’} according to the internal and external parameters of the camera and the parameters of the screen coordinate system to obtain the coordinate position on the screen, which is superimposed and displayed on the human body surface on the screen.

Optionally, the three-dimensional reconstruction data of the human body surface is acquired in real time, and the above steps are repeated, so as to update the acupoint positions in real time.

To sum up, this application first obtains the 3D reconstruction data of the human body surface; classifies and recognizes the 3D reconstruction data of the human body surface, and obtains the classification data of the human body area; performs pose registration on the classification data of the human body area and the data of virtual human acupuncture points, and The position of the acupuncture point is mapped in the surface three-dimensional reconstruction data; the virtual human body acupuncture point data is pre-established according to the human body acupuncture point model; the image corresponding to the three-dimensional reconstruction data of the human body surface and the position of the acupuncture point are displayed on the display device. The above scheme simplifies the difficulty of acupoint recognition by performing pose registration on the classification data of the human body area and the virtual human acupoint data, and mapping the acupoint positions in the three-dimensional reconstruction data of the human body surface.

Fig. 5 is a structural block diagram of an acupuncture point recognition device according to an exemplary embodiment. The acupoint recognition device includes:

A data acquisition module 501, configured to acquire three-dimensional reconstruction data of the human body surface;

Classification and recognition module 502, configured to classify and recognize the three-dimensional reconstruction data of the human body surface, and obtain classification data of human body regions;

The acupuncture point mapping module 503 is used to perform pose registration on the human body area classification data and the virtual human body acupuncture point data, and map the acupuncture point positions in the three-dimensional reconstruction data of the human body surface; the virtual human body acupuncture point data is pre-established according to the human body acupuncture point model;

The display module 504 is configured to display the image corresponding to the three-dimensional reconstruction data of the human body surface and the position of the acupoint on a display device.

In a possible implementation manner, the 3D reconstruction data of the human body surface is a first point set obtained by photographing the human body surface and performing 3D reconstruction;

The three-dimensional reconstruction data of the human body surface should be classified and identified to obtain the classification data of the human body area, including:

Obtain the initial relative position of the camera and the human body; the initial relative position is obtained by shooting with the camera and detecting facial feature points;

Combining the initial relative position of the camera and the human body and the movement trajectory of the camera collected by the gyroscope and the accelerometer, the first relative position of the current camera and the human body is obtained in real time;

According to the initial relative position and the first relative position, the current body region is obtained.

In a possible implementation manner, acquiring the current human body region according to the initial relative position and the first relative position includes:

Extracting the contour texture of the image corresponding to the three-dimensional reconstruction data of the human body surface;

According to the first point set and the contour texture, combined with the initial relative position and the first relative position, it is matched with the human body area information in the virtual human acupuncture point data, and the part with the highest matching degree is the current human body area.

In a possible implementation, the pose registration is performed on the human body area classification data and the virtual human acupoint data, and the acupoint positions are mapped in the three-dimensional reconstruction data of the human body surface, including:

Roughly registering the first point set and the second point set of the virtual human acupoint data according to the human body area classification data;

Through rigid registration, fine registration is performed on the first point set after rough registration and the second point set of virtual human acupoint data after rough registration.

In a possible implementation manner, the rough registration of the first point set and the second point set of the virtual human acupoint data according to the human body region classification data includes:

Extracting a subset of the first point set and a subset of the second point set for each human body area according to the human body region classification data;

Extracting the convex hulls of each subset of the first point set, the convex hull of each subset of the second point set, and the human body region patches corresponding to each convex hull;

Coarse registration is performed based on the convex hull and the human body area patch.

In a possible implementation, the rigid registration is used to fine-register the first point set after rough registration and the second point set of virtual human acupoint data after rough registration, including:

Set a spherical target area with the acupuncture point position in the subset of the second point set as the center and the first threshold as the radius;

extracting the three-dimensional curvature distribution and contour distribution of the acupoint positions in the subset of the second point set, and weighting the dot product to obtain the first spatial feature map;

extracting a second spatial feature map corresponding to a subset of the first point set according to the spherical target area;

A sampling window is set based on the second spatial characteristic atlas, and the first spatial characteristic atlas is traversed through the sampling window to obtain a target sampling window with the smallest measure between the first spatial characteristic atlas as the acupoint position in the subset of the first point set.

In a possible implementation manner, the surface distance between the positions of the acupuncture points in the subset of the first point set is acquired by a projection method;

Perform contour recognition on the image corresponding to the three-dimensional reconstruction data of the human body surface, obtain each closed contour, and calculate the distance between the midline of each closed contour and the position of the acupuncture point in the closed contour;

Combining the surface distance and the distance between the midline of the closed contour and the position of the acupuncture point in the closed contour, the position of the acupuncture point is compared with the recorded position corresponding to the knowledge map, and if the comparison result exceeds the second threshold, it is displayed on the display device The acupoint location is marked incorrectly; the knowledge map is pre-built based on the existing acupoint knowledge.

To sum up, this application first obtains the 3D reconstruction data of the human body surface; classifies and recognizes the 3D reconstruction data of the human body surface, and obtains the classification data of the human body area; performs pose registration on the classification data of the human body area and the data of virtual human acupuncture points, and The position of the acupuncture point is mapped in the surface three-dimensional reconstruction data; the virtual human body acupuncture point data is pre-established according to the human body acupuncture point model; the image corresponding to the three-dimensional reconstruction data of the human body surface and the position of the acupuncture point are displayed on the display device. The above scheme simplifies the difficulty of acupoint recognition by performing pose registration on the classification data of the human body area and the virtual human acupoint data, and mapping the acupoint positions in the three-dimensional reconstruction data of the human body surface.

Fig. 6 shows a structural block diagram of a computer device 600 shown in an exemplary embodiment of the present application. The computer device can be implemented as the server in the above solutions of the present application. The computer device 600 includes a central processing unit (Central Processing Unit, CPU) 601, a system memory 604 including a random access memory (Random Access Memory, RAM) 602 and a read-only memory (Read-Only Memory, ROM) 603, and A system bus 605 that connects the system memory 604 and the central processing unit 601 . The computer device 600 also includes a mass storage device 606 for storing an operating system 609 , application programs 610 and other program modules 611 .

The mass storage device 606 is connected to the central processing unit 601 through a mass storage controller (not shown) connected to the system bus 605 . The mass storage device 606 and its associated computer-readable media provide non-volatile storage for the computer device 600 . That is to say, the mass storage device 606 may include a computer-readable medium (not shown) such as a hard disk or a Compact Disc Read-Only Memory (CD-ROM) drive.

Without loss of generality, such computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM) flash memory or other solid state Storage technology, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, cassette, tape, disk storage or other magnetic storage devices. Certainly, those skilled in the art know that the computer storage medium is not limited to the above-mentioned ones. The above-mentioned system memory 604 and mass storage device 606 may be collectively referred to as memory.

According to various embodiments of the present disclosure, the computer device 600 may also operate on a remote computer connected to a network through a network such as the Internet. That is, the computer device 600 can be connected to the network 608 through the network interface unit 607 connected to the system bus 605, or in other words, the network interface unit 607 can also be used to connect to other types of networks or remote computer systems (not shown ).

The memory also includes at least one computer program, the at least one computer program is stored in the memory, and the central processing unit 601 implements all or part of the steps in the methods shown in the above embodiments by executing the at least one computer program.

In an exemplary embodiment, there is also provided a computer-readable storage medium for storing at least one computer program, and the at least one computer program is loaded and executed by a processor to implement all or part of the steps in the above method . For example, the computer-readable storage medium may be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a read-only optical disc (Compact Disc Read-Only Memory, CD-ROM), Magnetic tapes, floppy disks, and optical data storage devices, etc.

In an exemplary embodiment, there is also provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes all or part of the steps of the method shown in any one of the embodiments in FIG. 2 or FIG. 3 above.

Other embodiments of the present application will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the application, these modifications, uses or adaptations follow the general principles of the application and include common knowledge or conventional technical means in the technical field not disclosed in the application . The specification and examples are to be considered exemplary only, with a true scope and spirit of the application indicated by the following claims.

It should be understood that the present application is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. An acupoint identification method, comprising:

acquiring three-dimensional reconstruction data of the surface of a human body;

classifying and identifying the three-dimensional reconstruction data of the human body surface to obtain human body region classification data;

carrying out pose registration on the classification data of the human body region and the data of the virtual human body acupuncture point data, and mapping acupuncture point positions in the three-dimensional reconstruction data of the human body surface; the virtual human body acupuncture point data are pre-established according to a human body acupuncture point model;

and displaying the image corresponding to the three-dimensional reconstruction data of the human body surface and the acupuncture point position in display equipment.

2. The method according to claim 1, wherein the three-dimensional reconstruction data of the human body surface is a first set of points obtained by photographing and three-dimensionally reconstructing the human body surface;

the classifying and identifying the three-dimensional reconstruction data of the human body surface to obtain human body region classification data comprises the following steps:

acquiring an initial relative position of a camera and a human body; the initial relative position is obtained by shooting through a camera and detecting the human face characteristic points;

acquiring a first relative position of the current camera and the human body in real time by combining the initial relative position of the camera and the human body and the movement track of the camera acquired by the gyroscope and the accelerometer;

and acquiring the current human body area according to the initial relative position and the first relative position.

3. The method according to claim 2, wherein the obtaining a current body region according to the initial relative position and the first relative position comprises:

extracting contour texture of an image corresponding to the three-dimensional reconstruction data of the human body surface;

and matching the initial relative position and the first relative position with human body region information in the virtual human body acupoint data according to the first point set and the contour texture, wherein the part with the highest matching degree is the current human body region.

4. The method according to claim 2, wherein the pose registration of the human body region classification data and the virtual human body acupoint data and the mapping of the acupoint positions in the human body surface three-dimensional reconstruction data comprises:

carrying out coarse registration on the first point set and a second point set of the virtual human body point data according to the human body region classification data;

and performing fine registration on the first point set subjected to the coarse registration and the second point set of the virtual human body acupoint data subjected to the coarse registration through rigid registration.

5. The method according to claim 4, wherein the coarsely registering the first set of points with a second set of points of virtual human body acupoint data according to human body region classification data comprises:

respectively extracting a subset of the first point set and a subset of the second point set for each human body region according to the human body region classification data;

extracting a convex hull of each subset of the first point set, a convex hull of each subset of the second point set and a human body region patch corresponding to each convex hull;

and performing coarse registration based on the convex hull and the human body region surface patch.

6. The method according to claim 5, wherein the fine registration of the first point set after the coarse registration and the second point set of the virtual human body point data after the coarse registration through rigid registration comprises:

setting a spherical target area by taking the acupuncture point position in the subset of the second point set as a center and taking the first threshold value as a radius;

extracting three-dimensional curvature distribution and contour distribution of the acupuncture point positions in the subset of the second point set, and weighting point multiplication to obtain a first spatial feature map;

extracting a second spatial feature map corresponding to the subset of the first point set according to the spherical target region;

and setting a sampling window based on the second spatial feature map, traversing the first spatial feature map through the sampling window, and acquiring a target sampling window with the minimum measure between the target sampling window and the first spatial feature map as the acupoint position in the subset of the first point set.

7. The method of claim 6, further comprising:

acquiring the surface distance between the acupuncture point positions in the subset of the first point set by a projection method;

carrying out contour recognition on an image corresponding to the three-dimensional reconstruction data of the human body surface to obtain each closed contour, and calculating the distance between the central line of each closed contour and the acupuncture point position in each closed contour;

comparing the acupuncture point positions with recorded positions corresponding to the knowledge graph by combining the surface distance and the distance between the midline of the closed contour and the acupuncture point positions in the closed contour, and displaying the acupuncture point position marking errors in display equipment if the comparison result exceeds a second threshold value; the knowledge map is constructed in advance according to the existing acupoint knowledge.

8. An acupuncture point recognition device, the device comprising:

the data acquisition module is used for acquiring three-dimensional reconstruction data of the surface of a human body;

the classification identification module is used for performing classification identification on the three-dimensional reconstruction data of the human body surface to obtain human body region classification data;

the acupuncture point mapping module is used for carrying out pose registration on the classification data of the human body region and the virtual human body acupuncture point data and mapping acupuncture point positions in the three-dimensional reconstruction data of the human body surface; the virtual human body acupuncture point data is pre-established according to a human body acupuncture point model;

and the display module is used for displaying the image corresponding to the three-dimensional reconstruction data of the human body surface and the acupuncture point position in display equipment.

9. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction, the at least one instruction being loaded and executed by the processor to implement the method of acupoint identification according to any one of claims 1 to 7.

10. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the acupoint identification method according to any one of claims 1-7.

Images