CN110689586B - Tongue image identification method in traditional Chinese medicine intelligent tongue diagnosis and portable correction color card used for same - Google Patents

Abstract

The invention provides an identifiable portable tongue image color correction color card and its identification method used in intelligent tongue diagnosis in traditional Chinese medicine, which is used to solve the defects of the traditional tongue image correction color card being inconvenient to carry around and difficult to be recognized by a computer. Considering convenience, the color card is small in size and easy to carry. But too small size is not conducive to the automatic recognition of the computer. In the present invention, by adding a special geometric pattern in a specific position of the color card, and adding it to the color card area according to a specific size and geometric position, and supporting a specific and improved computer recognition algorithm, the automatic recognition of the color card by the computer is realized. Positioning, color picking, and achieved a high recognition rate. The invention eliminates the cumbersome process of manual calibration of color cards in color correction, and improves the efficiency of color correction processing. The invention also provides a solution to the errors caused by factors such as low identification rate of the intelligent tongue diagnosis of traditional Chinese medicine under low pixels, perspective deformation of the color card, etc., and obtains good results.

Description

Tongue Image Recognition Method in Intelligent Tongue Diagnosis of Traditional Chinese Medicine and Portable Calibration Used for the Method Color Card

technical field

The invention provides an identifiable portable tongue image color correction color card and an identification method thereof.

Background technique

The traditional TCM tongue image diagnosis method requires the doctor to directly observe the condition of the patient's tongue and draw a conclusion. This method requires the patient and the doctor face-to-face, which lacks convenience and promotion. In recent years, with the popularization of network communication, especially smart phones, the use of mobile phones for intelligent collection and diagnosis of tongue images has gradually become a new trend, which can well make up for the shortcomings of traditional methods that are poor in promotion and convenience. However, the use of mobile phones to collect and diagnose tongue images is susceptible to interference from external factors, and has high requirements for the light, angle, and definition of tongue images, especially the color restoration problem in color photography. These technical difficulties have become mobile phone tongue images. Acquisition development is an urgent problem to be solved.

Using the color card for color correction is one of the more commonly used methods at present. This method analyzes and fits the color change of the color card, thereby inferring the color change of the tongue color indirectly, and finally realizes the standardized processing of the tongue color. However, the color cards used in the existing tongue color correction research are usually large in size, and the smallest one is the size of a book, which is not conducive to carrying around. There is still a lack of effective solutions in the intelligent collection of tongue images in traditional Chinese medicine. In addition, the color cards used in the existing color correction lack the function of automatic positioning, and in practical applications, it is often necessary to manually identify and position the color cards. Such a method requires high labor costs, and requires a certain amount of time for operation and processing, which will have a great impact on the efficiency and accuracy of the standardized processing of tongue image acquisition.

In addition, there is no practical and effective solution in the prior art for tongue image color recognition in the intelligent collection of tongue images in traditional Chinese medicine.

Contents of the invention

The invention provides an identifiable portable tongue image color correction color card and an identification method thereof.

Description of drawings

Fig. 1 is a flowchart of a tongue image color correction color card identification method according to an embodiment of the present invention.

Fig. 2 is a tongue image color correction color card according to an embodiment of the present invention.

Fig. 3 is used to illustrate the identification of the color card position identifier according to an embodiment of the present invention.

Figure 4 is used to show the shape change of the rectangle after perspective deformation.

Fig. 5 shows the change graph of sampling points for recognition success and failure.

detailed description

In view of the above-mentioned problems in the prior art, the inventor proposes a portable color correction color card with automatic positioning function and its identification method. The purpose of the present invention includes two, one is to utilize the special color card design and automatic identification method, utilize the machine to complete the positioning work of the color card identification that is usually done manually in the past, thereby improve the working efficiency and standardization of the color correction process, for It is convenient for large-scale tongue image color correction or color correction in other scenes. For this purpose, the present invention provides a color card positioning method based on a position identifier, by setting a pattern of a special shape at the apex of the color card, using an image recognition algorithm to identify and position the symbol, and then inferring all the color blocks through the geometric relationship The location and calculate the corresponding color value. The second is to improve the shape, size, and layout of traditional color cards, so that the color cards can be carried and used in actual use.

As shown in Figure 1, the above-mentioned color card identification and color correction process includes:

Step 1: Set the special position identifier of the color card. Design different special identifiers on the four vertices of the color card. Three of the vertices use one symbol, while the last vertex uses another symbol, and the two symbols have different geometric characteristics. According to a specific embodiment of the present invention, after the location identifier is set, a color block is filled in a rectangular area formed by four symbols.

Step 2: Use the color card for image acquisition. Place the color card and the object in the same scene in the same light condition area to shoot, and get an image that contains both the target object and the color card.

Step 3: Search the image, and use the geometric features of the symbols to search out the positions of the four symbols. Due to problems such as shooting angles, the rectangular color card may be distorted into an arbitrary quadrilateral due to the perspective effect, so it is necessary to use the geometric relationship of the four vertices to correct the coordinates and restore the area to a rectangle.

Step 4: Use the pre-designed set relationship between the color block and the color card to calculate and locate the color block area, and find the center point of the color block. Use the center point to search the image to find the boundary of the color block, so as to determine the position of the entire color block. Finally, the color of the entire color block is calculated. According to a specific embodiment, the region growing method is used to find the boundary of the color block.

Step 5: According to the color selection result, use the color information of the color block in the standard environment to perform linear fitting correction, and automatically judge the recognition result according to the goodness of fit. When the goodness of fit reaches and/or exceeds a certain preset When the threshold is reached, it is judged that the recognition is successful.

Finally, after completing all the above steps, the color correction algorithm can be used to correct the color of the image and finally obtain a color-standardized tongue image.

Embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

Color card design:

Fig. 2 is a portable tongue image color correction color card supporting automatic positioning according to an embodiment of the present invention. The color card includes two areas: the handheld area and the color card area.

The hand-held area is set on one side of the color card to ensure sufficient space when holding it with one hand, ensuring the convenience of use, and also to fully avoid the hand covering the color card and/or producing shadows in the color card area, which will affect the color correction. Effect.

According to a specific embodiment of the present invention, the color card area is a rectangle, including three parts, which are respectively a position identifier area, an additional information identification area, and a color block area of the color card.

According to a specific embodiment, the location identifier area is distributed on four vertices of the color card area, and they have special geometric features. According to a specific embodiment, according to different geometric features, the identifier includes a first identifier and a second identifier.

According to a specific embodiment of the present invention. Type 1 identifiers include three rectangles with a side length ratio of 7:5:3, and there are 3 first-type identifiers in total, and they are respectively located at three of the four vertices of the color card area (such as upper left, upper right, lower left apex). The second identifier includes three rectangles with a side length ratio of 5:3:1, and is set at the fourth vertex among the four vertices of the color card area (such as at the lower right corner of the color card area).

The additional information identification area contains some simple marking points, such as marking points that can be parsed according to binary rules, such as four color blocks used to record the version number of the color card, as parameters for further subsequent analysis of the color card set up. The color block area of the color card includes several squares with the same size, which are evenly arranged in the rectangle formed by the location identifiers. Each color block has a black border, which can help the recognition program better identify the boundary of the color block. The inner area of the color block is filled with different colors, and specific colors can be properly selected and/or replaced to meet different practical purposes.

Identifier identification:

After using the above color card for image acquisition, the image needs to be processed in grayscale and binary. According to an embodiment of the present invention, a dynamic global threshold method is used to perform binary processing on the image, wherein several groups of candidate thresholds are preset, and the thresholds are input one by one for binarization. The binarized image is scanned on the x-axis and y-axis respectively in the manner shown in Figure 3, and is identified according to the geometric features of the identifier.

The color card identifier location algorithm according to an embodiment of the present invention comprises:

(1) Determine a threshold th in the candidate threshold library as the initial binarization threshold, and binarize the grayscale image to obtain a binary image G.

(2) Count the length of continuous black or white dots in each row. If it satisfies black:white:black:white:black=1:1:3:1:1, calculate the coordinates of the midpoint of the target area and add it to the candidate point library.

(3) After obtaining the candidate point library, search within r pixels around it to determine whether there are other candidate points and count the number. If the number of surrounding candidate points is greater than the preset target n, the point is considered to be the target point and added to Target point library.

(4) Swap the directions of X and Y axes and repeat steps (2) and (3) to finally get the X and Y coordinates of all points

(5) Select the next candidate threshold th in the above candidate preset library and repeat steps (2)-(4)

(6) Summarize all the results and perform DBSCAN cluster analysis, select the top three areas with the number of cluster points as the final area, and obtain the coordinates of its center point as the coordinates of the identifier.

Positioning of the center point of the color block:

Due to perspective, the rectangle of the color card may be distorted into any quadrilateral, as shown in Figure 4, where points A, B, and D are Type 1 identifiers, and Point C is Type 2 identifiers. If the geometric calculation is performed directly, the error of the obtained result is relatively large. In one embodiment of the present invention, the method adopted is to restore any quadrilateral to a rectangle, and then perform further processing. The coordinates of the image are transformed using the geometric relationship in the gray space to determine the target coordinates of the pixels, and the gray value retains the original gray value. Since the calculated coordinates are not necessarily integers, the gray values corresponding to specific integer coordinate points in the target image need to be interpolated. In one embodiment of the present invention, bilinear interpolation is used for processing. However, such a method has a large amount of computation. In a preferred embodiment of the present invention, the determination of the coordinates of the center point of the color block is directly carried out by using the perspective principle formula, including:

Let the coordinates of the center point X of a certain color block in the standard rectangle be i, j, the length of the original rectangle is a, and the height is b. After perspective transformation, the four vertices of the rectangle become (a ₁ ,b ₁ ),(a ₂ ,b ₂ ),(a ₃ ,b ₃ ),(a ₄ ,b ₄ ), and the coordinates of point X After perspective transformation, it becomes:

Thus, when the coordinates i, j of the center point of the color block in the standard rectangle are determined, the coordinates i', j' corresponding to the center point in the actually captured image can be deduced.

Color block selection and color value calculation:

After obtaining the coordinates of the center point of the color block, the image is processed by the region growing algorithm in the gray space. Since the color block has obvious boundaries, the color and gray value in the same area are approximately uniform, so a relatively complete image can be obtained by the region growing algorithm. Color block area. After determining the color block area, considering that the boundary may cause a deviation to the value of the average color, it is necessary to cut the boundary of the color block area. In a specific embodiment of the present invention, the specific cut size is the color block area 1/10 of the side length; thus the final color block area is obtained. Finally, the specific color value of the color block can be obtained by calculating the average value of the color of the color block area. In this way, by processing each color block area, the colors of all the color blocks in the entire color card are obtained. In addition, by determining the value of the additional information identification area, additional information such as the excellent card version is read.

Judgment on whether the color card recognition is successful:

After in-depth and specific research, the present invention finds that: 1) the identification of the color card fails; therefore, 2) a means for judging the identification failure is needed.

Specifically, situations where recognition fails include:

Identification failure due to ambiguous and/or unrecognizable several position identifiers of the color chip, or

Due to the serious deformation of the color card image and other reasons, the position of the identified color block is obviously deviated, and the identified color block area may be some irrelevant areas and/or patterns.

In the above cases, the color value of the color block obtained is not the actual color value of the relevant color block of the color card, that is, the recognition fails.

Therefore, according to an embodiment of the present invention, after the color card identification is performed, the recognition effect of the color card is judged, and only the sampled values meeting the requirement of "identification success" are subjected to subsequent processing.

On the basis of a large number of experimental verifications, the inventors found that under normal shooting conditions, the color changes of the color blocks of the color card compared with the colors in the standard environment are roughly in line with the linear law in the RGB channels. This experiment is realized on the basis of the existing 48-color color card, and the design RGB values of each color block of the color card are as follows.

Figure 5 shows the color changes in the R channel of the sampling points for successful and failed recognition. It can be seen that the change of R is more in line with the linear law when the recognition is successful. Accordingly, the inventor proposed a solution for judging the recognition result by using the linear regression method, which specifically includes:

Assume that the color card has n sampling points, which are embodied as n color blocks on the color card, numbered 1~n, Y ₁ , Y ₂ ... Y _N are the colors of n color blocks in the standard environment y ₁ , y ₂ . . . y _n are the color measurement values of the color blocks numbered 1-n on the color card under the general environment (shooting environment). Y _R , y _R are the values of each sampling point on the R channel, then the formula for expressing the linear approximate relationship between Y _R and y _R can be obtained by using the least square method:

Y _R ＝f _R (y _R )＝k _R y _R +b _R

Among them, f _R is the mapping function under the R channel, and k _R and b _R are the fitting coefficients of the primary term and the constant term of the mapping function f _R.

Similarly, the mapping functions f _G and f _B under the G channel and the B channel can be obtained respectively.

已知，要得到i点在在标准环境中的修正值

在根据本发明的一个实施例中，使用下方公式分别计算出

的值：When there is a R, G, B channel measurement value of point i to be calibrated

Known, to get the corrected value of point i in the standard environment

In one embodiment according to the present invention, use the following formula to calculate respectively

value of:

的直线方程进行求解，而实现的。Among them, all the coefficients k and b in the above formula are determined by using the classical least squares principle, for the shape such as

The straight line equation is solved and realized.

After obtaining the coefficients k and b of the equation, the fitting equation and the actual measured value can be statistically analyzed and the linear fitting goodness R ² can be obtained. Where R2 can be calculated according to the following ^formula :

指第i个样本通过线性回归方程计算出的修正值，

为所有测量值y的平均值，其中1≤i≤n。where y _i refers to the measured value of the i-th sample,

Refers to the correction value calculated by the i-th sample through the linear regression equation,

is the average value of all measured values y, where 1≤i≤n.

According to another embodiment of the present invention, the linearity of fit R ² is determined by the following steps:

Determine the linear goodness of fit R ² of the distribution of a first subset of y _i , i=1,2...n:

in:

The summation in the above formula is performed on the first subset,

是第i个样本的修正值，

为从以下项中选出的一种：

is the corrected value of the i-th sample,

is one selected from the following:

mean of all measurements y, where 1≤i≤n,

the average value of the measured values y for the first subset, and

The average value of each measured value y corresponding to a second subset of y _i .

In an exemplary embodiment according to the present invention, when R ² >0.8, it is considered that the recognition is successful.

After judging that the recognition is successful, use the information of the color change of the color block of the color card relative to the color in the standard environment to carry out the next step of color correction processing, that is, to correct the color of the tongue image, so as to obtain the color-corrected tongue image . The color correction processing in the next step can use interpolation method or regression methods such as linear regression and polynomial regression.

In a series of experiments carried out by the inventor, 134 color card image samples taken under different conditions were classified and compared. The above-mentioned evaluation method of R ² >0.8 was used to evaluate the color card recognition results. In the harsh environment with low pixels, the recognition accuracy rate can reach about 94.2%, while in the better environment with high pixels, the recognition rate can reach 98.5%, and the overall recognition rate reaches 96.3%, which is enough to meet the actual use needs. Specific results include:

Claims (10)

1. A tongue image recognition method with the help of a portable tongue image color correction color card, the correction color card includes a plurality of color blocks with different colors respectively, including: A) For the image of the calibration color card taken under the same shooting conditions as the tongue image, determine whether the color recognition of the calibration color card is successful, B) When step A) determines that the color recognition of the calibration color card is successful, the color of the tongue image is corrected by using the information of the color change of the color block of the calibration color card relative to the color in the standard environment, thereby obtaining the color correction rear tongue image, It is characterized in that step A) comprises: A1) Determine the value y _i of each of the n color blocks of the calibration color card on at least one color channel under the shooting conditions, i=1, 2...n, A2) For the n measured values y _i , use the principle of least squares to determine the fitting coefficients k and b of the linear term and the constant term, and then determine the corresponding color correction value

A3) Determine the linear goodness of fit R ² of the distribution of a first subset of y _i , i=1,2...n:

in: The summation in the above formula is performed on the first subset,

is the corrected value of the i-th sample,

is one selected from the following: mean of all measurements y, where 1≤i≤n, the average value of the measured values y for the first subset, and The mean value of each measurement value y corresponding to a second subset of y _i , A4) When R ² is greater than or equal to the predetermined threshold, it is determined that the image recognition is successful, and when R ² is less than the predetermined threshold, it is determined that the image recognition fails. 2. The tongue image recognition method according to claim 1, further comprising: Determine the center point of each color block by using the predetermined set relationship between the color block and the calibration color card, The center point of the color block is used to search the image of the calibration color card, so as to determine the boundary of each color block and determine the color of each color block. 3. The tongue image recognition method according to claim 1, characterized in that: The calibration color card includes a plurality of identifiers respectively set at a plurality of predetermined positions of the calibration color card, said plurality of identifiers includes at least two different symbols having different geometric characteristics, searching the image, using the geometric features, to determine the position of each symbol, The coordinates are corrected using the position of each symbol to correct for possible distortions due to perspective effects of the shot. 4. The tongue image recognition method according to claim 1, characterized in that: The image and tongue image of the calibrated color chart are obtained by placing the color chart and the object in the same scene in the same light condition area and shooting. 5. The tongue image recognition method according to claim 2, characterized in that: Determining the boundaries of each color block includes using a region growing method to determine the boundaries of each color block. 6. The tongue image recognition method according to claim 1, characterized in that: The first subset and the second subset are randomly selected. 7. A computer-readable storage medium, on which a computer program is stored, wherein the computer program enables a processor to execute the tongue image recognition method according to any one of claims 1-6. 8. A portable calibration color card, characterized in that the portable calibration color card is suitable for the tongue image recognition method according to any one of claims 1-6, and includes: a plurality of color blocks each having a different color, A plurality of identifiers respectively set at a plurality of predetermined positions of the calibration color card, in: The plurality of identifiers includes at least two different symbols having different geometrical characteristics such that the location of each symbol can be determined by searching the image and using the geometrical characteristics. 9. The portable calibration color card according to claim 8, characterized in that: The portable calibration color card includes a hand-held area and a color card area, The hand-held area is set on one side of the color card to ensure sufficient space when holding it with one hand, and to fully avoid the hand covering the color card and/or producing shadows in the color card area, The color card area is a rectangle and includes the position identifier area and the color block area. 10. The portable calibration color card according to claim 9, characterized in that: The location identifier area is distributed on the four vertices of the color card area, and they have predetermined geometric characteristics, The color blocks are set in the color block area.

Images