CN118230052A - Cervical panoramic image few-sample classification method based on visual guidance and language prompt - Google Patents

Abstract

The present invention relates to a panoramic image classification technology for cervical pathology, and aims to propose a multimodal integrated diagnosis solution to fill the current application gap of visual language technology in cervical tissue pathology diagnosis and improve the efficiency of cervical tissue pathology diagnosis. Existing visual language models are usually trained on large-scale image-text pairs, and have strong representation, generalization and migration capabilities. However, this approach faces challenges in the field of cervical tissue pathology. Due to the high privacy of data, high annotation costs, and difficulty in replicating expert experience, it is difficult to construct large-scale image-text pairs. In response to these problems, the present invention proposes a small-sample classification method for cervical panoramic images based on visual guidance and language prompts. Experimental results show that the method can effectively identify cervical tissue pathology features and provide pathological diagnosis basis and diagnosis results. The present invention has broad application prospects in the field of cervical tissue pathology diagnosis.

Description

A few-shot classification method for panoramic cervical images based on visual guidance and language prompts

Technical Field

The present invention designs a few-sample classification method for cervical panoramic images based on visual guidance and language prompts

Background technique

In recent years, the high incidence of cervical cancer has become a social problem that threatens women's lives. Among women worldwide, about 527,600 are newly diagnosed with cervical cancer each year, and nearly 265,000 die from the disease. In China, about 75,000 new cases of cervical cancer are discovered each year, accounting for 1/7 of the total number of new cervical cancer cases worldwide, and 35,000 die from the disease. However, not only are the number of cervical histopathologists limited, but their rich clinical experience and professional knowledge are difficult to replicate and popularize, and the contradiction between supply and demand is significant. Cervical histopathology is regarded as the "gold standard" for diagnosing cervical cancer and related lesions. It is crucial for identifying early lesions, determining the nature of lesions, guiding the formulation of individualized treatment plans, and even evaluating prognosis. Therefore, cervical histopathology research has great practical significance and long-term value in improving the accuracy of cervical cancer diagnosis, reducing misdiagnosis and missed diagnosis, and improving overall quality of life.

At present, the study of cervical cancer tissue pathology mainly relies on deep learning image processing technology to achieve automatic detection, accurate segmentation and classification and identification of lesion areas in pathological sections. Although such technology improves diagnostic efficiency to a certain extent, due to the limitations of the feature extraction method of the image itself and the size of the data, it is difficult to approach the understanding and description of complex lesion patterns and subtle pathological changes mastered by pathologists based on clinical experience. In contrast, the multimodal learning method based on language cues combines visual and natural language descriptions, which can more comprehensively capture and understand the complex pathological features in pathological images, helping to make up for the shortcomings of relying solely on image analysis. Although visual language model learning has been initially explored in other medical imaging fields and has shown potential, the application of such methods in the field of cervical tissue pathology is still in its infancy and has not been widely promoted. The reasons are that, on the one hand, the professional knowledge barrier involved in cervical tissue pathology diagnosis is relatively high, and the pathological characteristics between different disease types vary significantly, making it difficult for existing general models to directly adapt to the specific diagnostic needs of cervical cancer; on the other hand, due to its sensitivity and privacy protection requirements, public data sets for cervical tissue pathology data are extremely scarce. In addition, the production styles of pathological sections vary and the labeling costs are high, which makes large-scale data-driven learning methods face serious challenges in cervical pathology.

In view of the above background, the present invention proposes a small sample classification method for panoramic cervical images based on visual guidance and language prompts. This method uses a combination of visual prior information and text description to effectively deal with practical problems such as lack of refined annotation data and differences in production styles, achieve efficient learning and accurate recognition of cervical pathological characteristics, improve the accuracy and efficiency of cervical tissue pathology diagnosis, and provide a more scientific and accurate basis for early detection, accurate staging, personalized treatment decisions and prognosis evaluation of cervical cancer.

Summary of the invention

This invention aims to address the two core challenges in the field of cervical tissue pathology diagnosis: the lack of large-scale finely labeled data and the significant differences in production styles, and aims to fill the current application gap of visual language technology in this field. Its innovative ideas come from a deep understanding of the doctor's diagnostic process and a close fit with the needs of clinical practice. To this end, we propose a few-sample classification method for cervical panoramic images based on visual guidance and language prompts, which aims to effectively cope with diagnostic tasks under conditions of limited labeled data by integrating the advantages of visual information and language description, and ensure accurate identification of cervical pathological features under various production styles, thereby improving the accuracy and efficiency of cervical cancer diagnosis and meeting the high standards of clinical practice.

The above invention objectives are mainly achieved through the following technical solutions:

S1. Generation of visual priori guidance map of panoramic image of cervical tissue:

Preprocessing stage: First, a low-magnification (2.5x) panoramic image of the cervix was obtained and subjected to dye normalization processing, including RGB to Lab color space conversion, channel normalization, and Z-score standardization to reduce image color differences. Then, the tissue boundary was determined by binarization and contour detection.

Boundary distance calculation stage: Since each grade of cervical tissue pathology presents a specific distribution pattern, pathologists pay attention to the evolution of lesions in the cervical epithelium and the depth of invasion during the diagnosis process. Therefore, the area of interest of the cervical tissue pathology grading task should be from the edge to the center of the tissue section. The boundary distance can be calculated based on this as prior knowledge. The specific plan is as follows:

According to the morphological characteristics of cervical slice tissue, the epithelial tissue usually wraps the stroma part, and the epithelial part is mostly strip-shaped. In order to make the epithelial tissue part obtain a larger boundary weight, first consider obtaining the centroid of the slice tissue block, and calculate the area A and the first-order moments _M10 and _M01 enclosed by the outer contour line of the tissue slice obtained in the preprocessing stage;

Then calculate the centroid (cX, cY) based on the contour area and the first-order moment;

Next, according to the tissue contour _Ci , find the four vertices _P1 ( _x1 , _y1 ), _P2 ( _x2 , _y2 ), _P3 ( _x3 , _y3 ), _P4 ( _x4 , _y4 ) of the minimum circumscribed rectangle R of the contour, expressed as: { _P1 , _P2 , _P3 , _P4 } = minAreaRect( _Ci );

Then calculate the equations of the straight lines l ₁ , l ₂ passing through the centroid and parallel to the sides of the circumscribed quadrilateral, and find the distances dist ₁ , dist ₂ from each pixel point in the contour to the straight lines;

Finally, the ratio of the distance value of the pixel inside the contour to the distance of the point on the contour is calculated as the distance representation of the pixel point. In this way, the distance representation of the pixel point closer to the contour is closer to 1, and the distance representation of the pixel point outside the contour is recorded as -1. In this way, the image-level distance representation in two directions is formed, which is recorded as With/>

ROI hard segmentation stage: Since the cervical tissue epithelium and its glandular epithelial cells show a more dense arrangement than the stromal cells, and the chromatin density of such cells is relatively large, the epithelial tissue area often shows a darker color in histopathological staining; in the process of pathological diagnosis, doctors pay special attention to the careful observation of the arrangement of the epithelial basal layer, parabasal layer and glandular epithelial cells. The unique dark staining characteristics of these areas can be used as an important visual prior to improve the diagnostic efficiency. The steps for extracting the visual prior region of interest (ROI) are as follows:

Firstly, the red (R) and blue (B) channels in the cervical tissue image block were binarized using the OTSU global threshold method.

Then, according to the calculated threshold, the pixel value of each channel is compared with its threshold. The pixel value greater than the threshold is set to 1 (considered as chromatin-dense area), and the pixel value less than or equal to the threshold is set to 0 (considered as background). The binary segmentation of the red and blue channels is completed, and the binary images are recorded as R _bin and B _bin respectively;

Then calculate the gradient enhanced image G(x,y) of the grayscale image I _grey after fuzzy denoising and other preprocessing, use the Sobel operator to obtain the horizontal and vertical gradients and then obtain the gradient magnitude image M(x,y);

The watershed change algorithm is used to process the gradient image, find the area with gradient mutation in the image, and obtain the segmentation area by constructing a topological map and setting the threshold segmentation value range, and then convert it into a binary mask;

Finally, the region of interest is comprehensively delineated, and the above three binary images are logically operated to obtain the hard segmentation ROI (i, j) of the region of interest (ROI) of the cervical tissue section;

Visual prior guided image generation stage: The image-level boundary distance representation obtained by combining the above two stages of cervical tissue image tissue block boundary distance calculation stage and tissue block visual prior region of interest (ROI) hard segmentation stage And ROI hard division ROI(i,j), through weighted calculation to obtain the visual prior guidance map I _activate ;

S2. Instance-level feature extraction of cervical tissue images:

In view of the advantage that the image encoder Vision Transformer (ViT) model can effectively capture the contextual dependency between adjacent patches, and in order to fit the refined diagnosis scenario of cervical tissue pathology, the present invention focuses on the development law of lesions at different levels of cervical tissue during the differentiation process and its spatial distribution sequence characteristics in the instance-level feature extraction stage of cervical tissue images; in view of the irregular shape characteristics of the tissue blocks of cervical tissue sections, the traditional rough segmentation method in the horizontal and vertical directions may not fully consider the regional coherence and development process of the lesion area. Therefore, the present invention proposes a grid sampling method based on visual priors, which introduces a PE module and uses a visual prior guidance map I _activate to generate a visual prior position encoding feature F _position to achieve a more refined tissue structure-guided division of the cervical tissue area, forming a tissue image sequence with biological significance that reflects the changes in tissue levels, replacing the simple division Slide strategy that only relies on a fixed step size in previous multi-instance learning tasks, so that the model can deeply explore the evolution trajectory of lesions between different levels of cervical tissue and fully capture the spatial coherence characteristics of the lesion area;

Then, the pre-trained CLIP image encoder (ViT) is used to extract the cervical tissue slice image feature F _{image_patch} at 40x magnification, and the generated visual prior position encoding feature F _position is concatenated with the cervical tissue slice image feature F _{image_patch} . Then, the cervical tissue image instance-level feature F _{image_instance} is obtained through the fully connected layer (FC). The formula is as follows:

F _position = LN(MHSA(I _activate W _PE )) (1)

F _{image_instance} = Concat(F _{image_patch} ,F _position )W _feat (2)

S3. Key instance feature extraction of cervical tissue images:

When processing cervical tissue slice images at 40 times microscopic magnification, taking into account the huge amount of data and the complexity of the calculation of image-by-image and text matching, and considering that the pathological changes of cervical tissue are continuous, the pathological features of adjacent slices in the same part are highly similar. Even if some slices are skipped for analysis, the impact on pathological grading diagnosis is relatively limited. Therefore, in the current stage, the present invention uses the attention mechanism to calculate the instance-level features F _{image_instance} of the cervical tissue image extracted from the S2 stage. The instance-level features contain spatial information, and then through the multi-head attention mechanism (Multi-HeadSelf-Attention), the purpose is to mine the spatial correlation and contextual correlation between instances, and thus generate attention scores. These scores are used to screen out the most representative key features F _{TopK_instance} and simplify redundant calculations. The specific method is as follows:

First, the instance-level feature F _{image_instance} of the cervical tissue image is layer-normalized. Then, the multi-head attention module (MHSA) is introduced to calculate the attention score α _{image_instance} and add it to the instance-level feature F _{image_instance} to obtain the visually guided weighted instance feature F _{image_instance} ′. Then, the class probability P _{image_instance} of the instance is obtained through the pooling layer (MaxPooling) and the fully connected layer (FC). According to the score, the top K instance-level features are selected as the key instance features F _{TOPK_instance} . The formula is as follows:

α _{image_instance} =MHSA(LN(F _{image_instance} )) (3)

F _{image_instance} ′＝α _{image_instance} F _{image_instance} (4)

P _{image_instance} =σ(Pool(F _{image_instance} ′)W _{image_instance} ) (5)

Then, the visually guided weighted instance feature F _{image_instance} ′ is randomly masked to form a feature sequence F _mask . By designing a position decoder PD, the feature F _{image_instance} ′ containing spatial and position information is decoded. The foreground and background information of the visual guidance map I _activate is used as supervision information to reconstruct the mask position, and then the visual guidance map I _activate ′ is updated to improve the spatial distribution semantic consistency and generalization ability. The distribution loss L _position is calculated and then continuously updated and iterated. The formula is as follows:

(F′,F _p ′)＝LN(MHSA(F _mask ))W _PD (8)

S4. Constructing instance-level textual description of cervical tissue images:

In order to achieve image-text semantic alignment and improve the generalization of image classification tasks, the present invention uses ChatGPT4 to generate a series of instance-level text descriptions of the grading locations of cervical tissue pathological lesions and their positive and abnormal features. These text descriptions are converted into text features through the pre-trained CLIP text encoder, and the similarity is calculated with the key instance feature F _{TOPK_instance} of cervical tissue extracted in the S3 stage to establish the association between instance-level images and texts;

S5. Alignment of instance-level image features and text features:

In order to reduce the computational level of CLIP model training in the task of semantic alignment of instance-level histopathological images and texts, the present invention designs a lightweight training strategy: the weights of the pre-trained CLIP image encoder and text encoder are frozen, and the training focuses on optimizing the learnable parameter Tokens of the image and text; the image Token T _{image_token} is obtained by passing the extracted key instance features through a lightweight network (LightNet) composed of two layers of multi-layer perceptrons (MLPs), and the text Token T _{text_token} is set to learn for each instance and capture the specific language description information of different pathological types to learn a more appropriate text description; therefore, the image T _{image_token} and the text T _{text_token} are concatenated to form a learnable parameter sequence T _token , the number of which can be designed by oneself, for example, L Tokens are set as empirical setting values, denoted as T _token ={t ₁ ,t ₂ ,…,t _L }, and the corresponding instance-level label is denoted as Y ={y ₁ ,y ₂ ,…,y _L }), the cross-entropy loss is used to measure the difference between the prediction results and the labels, and the back-propagation error is used to update these tokens. The formula is as follows:

T _token = Concat(LightNet(F _{TOPK_instance} ),T _{text_token} ) (10)

Among them, _pi is the probability that the model predicts the corresponding category label for the i-th Token, that is,

p _i =P(t _i |F _{TOPK_instance} ,T _token );

Next, the pre-trained text encoder is used to obtain text features, and the cosine similarity is calculated with the key instance features of cervical tissue obtained in S3. The objective function formula is as follows:

L _contrastive = (L _text2image + L _image2text )/2 (15)

Where _Ii is the embedding vector of the i-th text description, _Tj is the embedding vector of the j-th slice image, the text-to-image contrast loss is denoted as _Ltext2image , the image-to-text contrast loss is denoted as _Limage2text , and the overall similarity contrast loss is denoted as _Lcontrastive ;

S6. Instance-level feature aggregation:

By calculating the similarity scores between instance-level image features and text features, the key instance features extracted from S3 are weighted and aggregated to form bag-level features F _Bag , so that the features of lesions at different levels and of different types can be effectively integrated and expressed in the global scope of the panoramic cervical image.

S7. Establish a lesion prompt library based on the whole slice image (WSI) level:

Using the cervical histopathological lesion illustrations and feature descriptions of lesions of various levels from CIN1 to CIN3 accumulated in the cervical histopathology atlas, an intelligent prompt library for lesion description is created to include a standardized and intelligent professional knowledge system: the ChatGPT4 model re-describes the description information in the atlas to narrow the gap between medical professional knowledge and deep learning semantic understanding; the pre-trained CLIP image encoder and text encoder are used to extract the corresponding image features F _key and text features F _value : image features and text feature pairs are constructed to form multiple key-value pairs (Key-Value), recorded as F _{key_value} ;

S8. Package-level image feature retrieval text feature:

In order to further reduce the computational level of the CLIP model in the task of aligning pathological lesion features with text semantics, we consider converting the image-text matching task into a retrieval task in a training-free way: the bag-level feature F _Bag in S6 and the image feature F _key in S7 are used to calculate the similarity score A _{Bag_key} , and the key-value pair F _{key_value} is aggregated with F _value through label propagation to form a prediction F _Bag ′ to realize knowledge retrieval. The formula is as follows:

A _{Bag_key} = exp(-β(1-F _Bag F _key ^T )) (16)

F _Bag ′＝A _{Bag_key} F _value (17)

S9. Lesion diagnosis decision:

According to the prediction F _Bag ′ obtained in the S8 stage and the original bag feature F _Bag, the prediction features are obtained through the lightweight network LightNet composed of two layers of MLP. The LightNet weight is recorded as W _c . The two are weighted summed by residual connection to obtain the final prediction result logits _Bag . This result obtains both instance-level data information and Few shot knowledge. The prediction result calculation formula is as follows:

Finally, the level range of the lesion is determined based on the category prediction, and the corresponding description information of the lesion prompt library built by S7 is used as the final diagnosis basis, thereby assisting doctors to make more accurate diagnosis conclusions of cervical cancer and lesions.

Effects of the Invention

The present invention focuses on the field of cervical tissue pathology diagnosis. Aiming at the two major challenges of insufficient large-scale refined annotation data and diversity of production styles, a few-sample classification method for cervical panoramic images based on visual guidance and language prompts is proposed. First, starting from the low-magnification panoramic image of the cervix, color normalization is used to eliminate image color differences, and the tissue boundary is determined by contour detection. On this basis, the boundary distance is calculated as visual prior knowledge using the pathological principles and morphological characteristics of cervical tissue to locate the key diagnostic area. In the ROI hard segmentation stage, the OTSU threshold method and gradient enhancement technology are used, combined with the watershed change algorithm, to achieve effective segmentation of the dense epithelial and glandular epithelial cell areas in the cervical tissue, forming a visual prior region of interest (ROI). Furthermore, the invention integrates the results of boundary distance calculation and ROI hard segmentation to generate a visual prior guidance map to guide the refined segmentation of tissue image sequences. The pre-trained Vision Transformer (ViT) model is used to extract instance-level features, especially the introduction of visual prior position encoding features, which overcomes the limitations of traditional methods in locating lesion areas and can better capture the evolution law and spatial distribution characteristics of lesions between tissue levels. At the same time, the invention innovatively uses ChatGPT4 to generate instance-level text descriptions of cervical tissue lesions, and combines the CLIP model for image-text alignment, which enhances the semantic understanding and generalization capabilities of the model. By designing a lightweight training strategy, focusing on the mining and selection of key instance features, the complexity and computational cost of large-scale data processing are effectively reduced. Finally, the invention achieves weighted aggregation of key features by calculating the similarity between instance-level image features and text features, and constructs a lesion prompt library at the full-slice image level through histopathological atlases, and uses a training-free retrieval method for image-text matching, which greatly improves the diagnostic efficiency. The entire diagnostic decision-making process integrates instance-level data information and a small amount of sample knowledge to ensure the accuracy and reliability of graded diagnosis of cervical lesions. The significance of the invention technology is that by integrating visual and language information, it breaks through the limitations of traditional single image analysis methods, effectively improves the recognition accuracy and diagnostic efficiency of cervical cancer and other lesions, and provides certain support for clinical practice. Experiments have shown that this method can be quickly and effectively trained with a small amount of cervical tissue pathology data. It can provide accurate lesion grading diagnosis results, provide diagnostic basis, and provide lesion area prediction results, which meets the needs of clinical diagnosis scenarios.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 Technical roadmap of the few-sample classification method for cervical panoramic images based on visual guidance and language prompts;

Fig. 2 Supplementary diagram of the structure of the few-sample classification method of cervical panoramic images based on visual guidance and language prompts;

Figure 3: Visual prior guidance effect diagram;

Figures 4 to 9 are diagrams showing the diagnostic grading effects;

Specific implementation methods

Detailed ways

In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

As shown in Figures 1 and 2, the few-sample classification method for cervical panoramic images based on visual guidance and language prompts provided in this paper includes the following stages:

S1. Cervical tissue panoramic image visual priori guidance map generation stage:

Step 1: Preprocessing of cervical panoramic images:

Step 101, obtaining and scaling a cervical panoramic image thumbnail;

Step 102, perform color normalization: convert to Lab color space, perform normalization and Z-score standardization, and then convert back to RGB color space;

Step 103, image binarization and outer contour detection: perform image binarization processing, extract the image outer contour line through edge detection and connected domain marking, and segment and uniformly adjust the slice image size based on it;

Step 2: Calculation of cervical tissue image boundary distance:

Step 201, calculating the centroid of the tissue block: determining the centroid position according to the contour area and the first-order moment;

Step 202, determine the pixel distance representation: by calculating the distance ratio from each pixel point in the contour to the straight line where the centroid is located, obtain the image-level boundary distance representation in two directions;

Step 3: Hard segmentation of ROI of cervical tissue image:

Step 301, channel binarization: using the OTSU threshold method to perform binarization processing on the red and blue channels;

Step 302, gradient enhancement and segmentation: calculate the gradient image by using the Sobel operator, apply the watershed change algorithm to segment the gradient mutation area, and form a hard segmentation mask of the ROI;

Step 4: Generate a visual priori guidance map: Combine the boundary distance representation obtained in step 2 with the ROI hardening sub-mask obtained in step 3, and generate a visual priori guidance map through weighted and logical operations;

S2, cervical tissue image instance-level feature extraction stage:

Step 5, grid sampling and visual prior position encoding: align the visual prior guidance map with the panoramic image of cervical tissue and high-magnification tissue slices, use bilinear interpolation to perform upsampling and slicing operations, and construct the visual prior guidance position encoding through the position encoder;

Step 6: Image feature extraction: Use the pre-trained CLIP image encoder (ViT) model to extract image features of tissue sections at 40x magnification;

Step 7: Feature fusion: concatenate the visual prior position encoding features with the image features, and generate instance-level features through a fully connected layer (FC);

S3, Cervical tissue key instance feature extraction stage:

Step 8: Instance-level feature processing: perform layer normalization on instance-level features, calculate attention scores through the multi-head attention mechanism (MHSA), and generate visually guided weighted instance features;

Step 9: Key feature screening: Use the pooling layer and the fully connected layer to obtain the category probability of the instance, and select the top K instance-level features as key instance features based on the probability value;

S4: Constructing instance-level text prompt description of cervical tissue:

Step 10: Text description generation: Use ChatGPT4 to generate instance-level text description information prompts for cervical tissue pathological lesion sites and positive abnormal features;

Step 11: Text feature conversion: These text descriptions are converted into text features through the pre-trained CLIP text encoder;

S5, instance-level image-text pair alignment stage:

Step 12: Feature alignment and optimization: Calculate the similarity between key instance features and corresponding text features, and update model parameters through lightweight training strategy to optimize image-text alignment effect;

S6, instance-level feature aggregation stage:

Step 13: Weighted feature aggregation: perform weighted aggregation based on the similarity scores between key instance features and text features to form a package-level feature that reflects the overall lesion features;

S7. Construction of the lesion prompt library at the whole-slice image level:

Step 14: Establishment of lesion library: Collect images of cervical tissue pathological lesions of various levels from CIN1 to CIN3 and their feature descriptions, and use ChatGPT4 to normalize the descriptions; extract image features and text features of lesion examples respectively, and build a lesion prompt library in the form of key-value pairs;

S8. Package-level image feature retrieval text feature:

Step 15, image and text retrieval: use the package-level features and the image features in the lesion prompt library to calculate the cosine similarity, and obtain the prediction results through label propagation aggregation;

S9. Lesion diagnosis decision:

Step 16: Prediction and decision: Combine the prediction results with the original package features, generate the final prediction features through a lightweight network, and combine the prediction results with the lesion prompt library information to make a diagnostic decision on the lesion grade range.

In the visual prior guidance map generation stage, the embodiment of the present invention generates a visual prior guidance map through preprocessing, boundary distance calculation and ROI hard segmentation method to guide subsequent processing; then, with the help of the pre-trained CLIP image encoder (ViT), high-precision image features are extracted, and feature fusion is performed with the visual prior position encoding, and then the key instance features are screened out through the multi-head attention mechanism; ChatGPT4 is used to generate instance-level text descriptions corresponding to lesion features, and the text information is converted into text features that can be compared with image features through the CLIP text encoder, so as to achieve fine alignment optimization between images and texts; then the key instance features and text features are weightedly aggregated to generate package-level features that can comprehensively reflect the cervical lesion situation; based on this, a cervical tissue lesion image feature library covering levels CIN1 to CIN3 is created using tissue pathology maps, and the package-level features are used to calculate the similarity with the data in the library, and the prediction results are obtained through label propagation aggregation technology; finally, the lesion prediction results are combined with the instance-level prediction results to achieve lesion diagnosis decisions; the entire process aims to achieve accurate recognition and standardized description of cervical pathology images with the help of AI models to assist pathological diagnosis.

The embodiments of the present invention are described in detail below:

The embodiment of the present invention uses 400 private cervical panoramic images to apply the algorithm of the present invention for training, and uses 30 cervical panoramic images of different lesion levels to verify the pathological diagnosis effect of the algorithm of the present invention.

As shown in Figure 1 and Figure 2, the model training steps include the following parts:

S1. Generation of visual priori guidance map of panoramic image of cervical tissue, specifically implemented as follows:

Step 1: Preprocessing of panoramic images of cervical tissue at low magnification:

Step 101, using the OpenSlide tool to obtain a thumbnail of the panoramic image of the cervix at a magnification of 2.5x;

Step 102: Perform dye normalization in the cervical panoramic image training set:

First, convert RGB to Lab color space. The formula is as follows:

L＝0.2126×R+0.7152×G+0.0722×B (19)

a＝0.5×(R-G)+128 (20)

b＝0.5×(R-B)+128 (21)

Here R, G, and B represent the red, green, and blue components of each pixel in the RGB image respectively;

Then perform Lab channel normalization and then standardize with Z-sorce, that is, subtract the mean from the value of each channel and divide it by the standard deviation. The mean and standard deviation of the Lab channel can be obtained by calculating the statistics of the training data set. The formula is as follows:

Where _Li represents the brightness value of the i-th pixel in the Lab image, and N represents the total number of pixels. Similarly, the mean and standard deviation of channel a and channel b can be calculated;

Convert the normalized Lab image back to the RGB color space to get the normalized RGB image. The formula is as follows:

R＝L+1.402×(b-128) (24)

G＝L-0.3441×(a-128)-0.7141×(b-128) (25)

B＝L+1.772×(a-128) (26)

Step 103: perform image binarization and outer contour detection:

After obtaining the dyed normalized image, the image is binarized using the OpenCV tool library, and then the outer contour is detected. This method is based on image edge detection and connected domain marking technology. It accurately detects the outer contour of the object by finding the connected domain in the image and extracting the outer contour line.

Then the image is segmented according to the largest connected domain in the detected outer contour line C, and the segmented images are scaled to a consistent size, such as 1024×1024;

Step 2, calculating the boundary distance of the tissue block of the cervical tissue image;

Step 201, calculate the centroid of the tissue block, and determine the centroid position based on the contour area and the first-order moment:

According to the outer contour of the tissue slice obtained in the first step, the area A and the first-order moments M ₁₀ and M ₀₁ enclosed by the contour C are calculated, and the formula is expressed as follows:

A _i = contourArea(C _i ) (27)

Among them, _Ci represents the i-th contour, x and y are the horizontal and vertical coordinates of the pixel respectively;

Then calculate the centroid (cX, cY) based on the contour area and the first-order moment. The formula is as follows:

Step 202: Determine the pixel distance representation, by calculating the distance ratio from each pixel point in the contour to the straight line where the centroid is located, and obtain the image-level boundary distance representation in two directions:

According to the tissue contour C _i , find the four vertices P ₁ (x ₁ ,y ₁ ), P ₂ (x ₂ ,y ₂ ), P ₃ (x ₃ ,y ₃ ), P ₄ (x ₄ ,y ₄ ) of the minimum circumscribed rectangle R of the contour, expressed as:

{P ₁ ,P ₂ ,P ₃ ,P ₄ } = minAreaRect(C _i ) (32)

Then calculate the equations of the straight lines l ₁ and l ₂ passing through the centroid and parallel to the sides of the circumscribed quadrilateral, and find the distances dist ₁ and dist ₂ from each pixel point in the contour to the straight lines. The formulas are as follows:

Among them, (x _p ,y _p ) is the pixel point within the contour _Ci ;

The ratio of the distance value of the pixel inside the contour to the distance of the point on the contour is calculated as the distance representation of the pixel point. In this way, the distance representation of the pixel point closer to the contour is closer to 1, and the distance representation of the pixel point outside the contour is recorded as -1. In this way, the image-level distance representation in two directions is formed, which is recorded as With/>

Step 3: Hard segmentation of the visual prior region of interest (ROI) of the cervical tissue image tissue block:

Step 301: Channel binarization:

First, the red (R) and blue (B) channels in the tissue block of the cervical tissue image are binarized using the Otsu threshold (OTSU) global threshold method. For the red channel grayscale value _Rp and blue channel grayscale value _Bp of any pixel point p, the optimal thresholds _TR and _TB of each channel are calculated to maximize the inter-class variance between the foreground (chromatin dense area) and the background. The calculation formula is as follows:

Where ω ₀ and ω ₁ are the pixel proportions of the background and foreground, respectively, and μ ₀ and μ ₁ are the average grayscale values of the background and foreground, respectively;

Then, according to the calculated threshold, the pixel value of each channel is compared with its threshold. The pixel value greater than the threshold is set to 1 (considered as chromatin-dense area), and the pixel value less than or equal to the threshold is set to 0 (considered as background). The binary segmentation of the red and blue channels is completed, and the binary images are recorded as R _bin and B _bin respectively;

Step 302: Gradient enhancement and segmentation:

Calculate the gradient enhanced image G(x,y) of the grayscale image I _grey after fuzzy denoising and other preprocessing, use the Sobel operator to obtain the horizontal and vertical gradients and then obtain the gradient magnitude image M(x,y), the formula is as follows:

The watershed change algorithm is applied to process the gradient image to find the area where the gradient changes suddenly in the image. By constructing a topological map and setting the threshold segmentation range, the segmentation area is obtained and converted into a binary mask:

Mask _w = Filter(Watered(I _grey ,M(x,y))) (36)

Finally, the region of interest is comprehensively delineated, and the above three binary images are logically operated to obtain the hard segmentation ROI (i, j) of the region of interest (ROI) of the cervical tissue section:

Mask _C (i,j)＝R _bin (i,j)∨B _bin (i,j) (37)

ROI(i,j)＝Mask _C (i,j)∧Mask _W (i,j) (38)

Step 4: Generate visual prior guidance map of cervical tissue image tissue block:

Combining the image-level boundary distance representations I _dist1 , I _dist2 and ROI hard segmentation ROI(i,j) obtained in the above two stages of the cervical tissue image tissue block boundary distance calculation stage and the tissue block visual prior region of interest (ROI) hard segmentation stage, the visual prior guidance map I _activate is obtained through weighted calculation, and the formula is as follows:

Among them, α, β, and γ are hyperparameters with values between 0 and 1. Softmax maps the vector into a probability distribution to ensure that each element takes a value between 0 and 1.

S2. Instance-level feature extraction of cervical tissue images. The specific steps are as follows:

Step 5: Grid sampling and visual prior position encoding:

First, the visual prior guidance map I _activate is aligned with the cervical tissue slices; a panoramic image of the cervical tissue is obtained at a magnification of 5x and a set of cervical tissue slices is obtained at a magnification of 40x respectively; then the visual prior guidance map I _activate obtained in step 2 is upsampled by bilinear interpolation method, and aligned with the panoramic image of the cervical tissue at a magnification of 5x. The present invention is implemented by the tool class UpSample in PyTorch, and its principle is as follows:

Among them, Q ₁₁ (x ₁ ,y ₁ ),Q ₁₂ (x ₁ ,y ₂ ),Q ₂₁ (x ₂ ,y ₁ ),Q ₂₂ (x ₂ ,y ₂ ) are four adjacent points of the original image;

The panoramic image of cervical tissue at 5x magnification is sliced with the interpolated weight map, and aligned with the tissue slice at 40x magnification, and the slice is upsampled again, and the weight and slice are mapped to construct the prior visual guidance slice set I _{activate_patch} ;

Then, the visual prior-guided position encoding F _position is constructed through the position encoder. The encoder is constructed by a fully connected layer (FC), a multi-head attention layer (MHSA), and a layer normalization (LN). The calculation process is shown as follows:

F _position = LN(MHSA(I _{activate_path} W _PE )) (42)

Step 6, image feature extraction: Use the pre-trained CLIP image encoder (ViT) to extract the image features F _{image_patch} of cervical tissue slices at 40x magnification;

Step 7: Concatenate the generated visual prior position encoding feature F _position with the cervical tissue slice image feature F _{image_patch} , and then obtain the cervical tissue image instance-level feature F _{image_instance} through a fully connected layer (FC). The formula is as follows:

F _{image_instance} = Concat(F _{image_patch} ,F _position )W _feat (43)

S3. Key instance feature extraction of cervical tissue images:

Step 8, instance-level feature processing: perform layer normalization on the instance-level feature F _{image_instance} of the cervical tissue image, and then introduce the multi-head attention module (MHSA) to calculate the attention score α _{image_instance} and add it to the instance-level feature F _{image_instance} to obtain the visually guided weighted instance feature F _{image_instance} ′, the formula is as follows:

α _{image_instance} =MHSA(LN(F _{image_instance} )) (44)

F _{image_instance} ′＝α _{image_instance} F _{image_instance} (45)

Step 9, key feature screening: The visually guided weighted instance feature F _{image_instance} ′ is respectively passed through the pooling layer (MaxPooling) and the fully connected layer (FC) to obtain the instance category probability P _{image_instance} . According to the score, the top K instance-level features are selected as the key instance features F _{TOPK_instance} . The formula is as follows:

P _{image_instance} =σ(Pool(F _{image_instance} ′)W _{image_instance} ) (46)

Then, the visually guided weighted instance feature F _{image_instance} ′ is randomly masked to form a feature sequence F _mask . By designing a position decoder PD, the feature F _{image_instance} ′ containing spatial and position information is decoded. The foreground and background information of the visual guidance map I _activate is used as supervision information to reconstruct the mask position, and then the visual guidance map I _activate ′ is updated to improve the spatial distribution semantic consistency and generalization ability. The distribution loss L _position is calculated and then continuously updated and iterated. The formula is as follows:

(F′,F _p ′)＝LN(MHSA(F _mask ))W _PD (49)

S4. Constructing instance-level textual description of cervical tissue images:

Step 10, text description generation: ChatGPT4 is used to generate a series of text prompt information for the location of cervical tissue pathological lesions and their positive and abnormal characteristics. ChatGPT4 describes the three types of information, namely cell type, cell morphology description, and whether there is an abnormality, and integrates them into instance-level text description information prompts. The examples are as follows:

“([Cervical epithelial basal layer cells],[Columnar or cuboidal,tightly attached to the basement membrane,with nuclei that are elongated and relatively uniform in size,arranged in a regular pattern along the basement membrane],[Normal])”,“([Intermediate and superficial layers of cervical epithelial],[Intermediate layer cells begin to flatten and pack densely,without significant thickening,exhibiting uneven chromatin distribution,irregular nuclear membranes,darker staining.],[Abnormal])”,“([Cervical glandular epithelium],[Columnar cells form the glandular structures,withnuclei positioned at their bases,displaying disordered arrangement,large and irregular nuclei,increased cytoplasm,uneven chromatin distribution,and darklystained nuclei.],[Abnormal])”,“([Cervical stroma],[Located beneath the epithelial layer is the connective tissue, which contains sparse smooth muscle cells arranged loosely. The cells exhibit uniformly sized nuclei with lightstaining, contributing to an overall lighter hue.],[Unkown])”,…;

Step 11: Text feature conversion: The text description is converted into text features through the pre-trained CLIP text encoder, and the cosine similarity is calculated with the key instance feature F _{TOPK_instance} of cervical tissue extracted in the S3 stage to establish the association between instance-level images and texts;

S5. Alignment of instance-level image features and text features:

Step 12: Feature alignment and optimization:

In this embodiment, the weights of the pre-trained CLIP image encoder and text encoder are frozen, and 10 tokens are set as empirical setting values. The token is obtained by setting the text token T _{text_token} and the key instance feature F _{TOPK_instance} to concatenate the image token obtained by two layers of MLP, denoted as T _token = {t ₁ , t ₂ , …, t ₁₀ }, and the corresponding instance-level label is denoted as Y = {y ₁ , y ₂ , …, y ₁₀ }). The cross-entropy loss is used to measure the difference between the prediction result and the label, and the back-propagation error is used to update these tokens. The formula is as follows:

T _token = Concat(LightNet(F _{TOPK_instance} ),T _{text_token} ) (51)

Among them, _pi is the probability that the model predicts the corresponding category label for the i-th Token, that is,

p _i =P(t _i |F _{TOPK_instance} ,T _token );

According to step 11, its objective function is constructed as follows:

L _contrastive = (L _text2image + L _image2text )/2 (56)

Where _Ii is the embedding vector of the i-th text description, _Tj is the embedding vector of the j-th slice image, the text-to-image contrast loss is denoted as _Ltext2image , the image-to-text contrast loss is denoted as _Limage2text , and the overall similarity contrast loss is denoted as _Lcontrastive ;

S6. Instance-level feature aggregation:

Step 13: Weighted feature aggregation: By calculating the similarity scores between instance-level image features and text features, the key instance features extracted from S3 are weightedly aggregated to form bag-level features F _Bag ;

S7. Construct a lesion prompt library based on the whole slice image (WSI) level:

Step 14: Establishment of lesion library: By systematically collecting the descriptions and legends of cervical histopathology lesions in the cervical histopathology atlas, we regard them as authoritative expert knowledge reserves, and further use the advanced artificial intelligence model ChatGPT4 to deeply integrate and sort out these professional knowledge, aiming to transform the traditional subjective description method of medical experts into a standardized and intelligent description system, thereby effectively bridging the gap in the knowledge transfer process. The examples are as follows:

“([CIN1],[Slight nuclear enlargement,mild chromatin condensation,and slightly enlarged nucleoli.Cells show minor disarray but maintain a single-layer structure,mainly affecting the lower third of the cervical epithelium,…],[Abnormal])”,“([CIN2],[The lesion primarily involves the intermediate to deep layers of the cervical epithelium,where despite the basal layer structure typically remaining relatively intact,cells immediately adjacent to the basal layer exhibit pronounced abnormalities,manifesting as enlarged nuclei,increased nuclear staining intensity,heightened nucleo-cytoplasmic ratio…],[Abnormal])”,“([CIN3],[The lesion involves the deeplayers and potentially the entire epithelium of the cervix,accompanied bysignificant increases in cellular atypia.Although the basal cell layerstructure may be locally preserved,there are marked changes suggestive of malignant potential in adjacent regions as well as in intermediate to deeplayers of cells.This is characterized by notably enlarged and irregularly shaped nuclei, with intense and dense nuclear staining, severe imbalance in the nucleus-to-cytoplasm ratio, prominently abnormal nucleolar structures, and can even manifest as multinucleation or bizarre nuclei…],[Abnormal])”,…;

Then use the pre-trained CLIP image encoder and text encoder to extract the corresponding image features F _key and text features F _value : construct image features and text features to form multiple key-value pairs (Key-Value), recorded as F _{key_value} ;

S8. Package-level image feature retrieval text feature:

Step 15, image and text retrieval: Calculate the similarity score A _{Bag_key} between the bag-level feature F _Bag in S6 and the image feature F _key in S7, and use the key-value pair F _{key_value} to aggregate with F _value through label propagation to form the prediction F _Bag ′ to realize knowledge retrieval. The formula is as follows:

A _{Bag_key} = exp(-β(1-F _Bag F _key ^T )) (57)

F _Bag ′＝A _{Bag_key} F _value (58)

S9. Lesion diagnosis decision:

Step 16, prediction and decision: According to the prediction F _Bag ′ obtained in the S8 stage and the original bag feature F _Bag , the prediction feature is obtained through the lightweight network LightNet composed of two layers of MLP. The LightNet weight is recorded as W _c . The two are weighted summed by residual connection to obtain the final prediction result logits _Bag . This result obtains both instance-level data information and Few-shot knowledge. The prediction result calculation formula is as follows:

The level range of the lesion is determined based on the category prediction, and the corresponding description information of the lesion prompt library constructed by S7 is used as the final diagnosis basis.

Finally, the algorithm diagnosis results were demonstrated using three WSIs of different lesion levels, and the results are shown in Figures 4 to 9. It is clear that the visual guidance prior plays a certain guiding role in lesion identification. The combination of visual and text models is applied to the refined diagnosis of cervical tissue pathology. This method successfully meets the needs of clinical practice, not only accurately providing diagnostic results, but also revealing the detailed basis supporting the diagnostic conclusions.

The present invention may also have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of the present invention.

Claims (6)

1. A few-sample classification method for panoramic cervical images based on visual guidance and language prompts, characterized by including multiple stages such as visual prior guidance, instance-level feature extraction, key instance screening, text description construction and image-text alignment, and lesion grade determination: The method comprises the steps of: S1. Cervical tissue panoramic image visual priori guidance map generation stage: Step 1: Preprocessing of cervical panoramic images: obtaining and scaling cervical panoramic image thumbnails, performing dye normalization, image binarization, and outer contour detection; Step 2: Calculation of the boundary distance of the cervical tissue image: Calculate the centroid of the tissue block and determine the pixel distance representation; Step 3: Hard segmentation of ROI of cervical tissue image: Binarization of red and blue channels is performed using the OTSU threshold method, gradient image is calculated using the Sobel operator, and the watershed change algorithm is applied to segment the gradient mutation area to form a hard segmentation mask of ROI; Step 4: Generate a visual priori guidance map: Combine the boundary distance representation obtained in step 2 with the ROI hardening sub-mask obtained in step 3, and generate a visual priori guidance map through weighted and logical operations; S2, cervical tissue image instance-level feature extraction stage: Step 5, grid sampling and visual prior position encoding: align the visual prior guidance map with the panoramic image of cervical tissue and high-magnification tissue slices, use bilinear interpolation to perform upsampling and slicing operations, and construct the visual prior guidance position encoding through the position encoder; Step 6: Image feature extraction: Use the pre-trained CLIP image encoder (ViT) model to extract image features of tissue sections at 40x magnification; Step 7: Feature fusion: concatenate the visual prior position encoding features with the image features, and generate instance-level features through a fully connected layer (FC); S3, Cervical tissue key instance feature extraction stage: Step 8: Instance-level feature processing: perform layer normalization on instance-level features, calculate attention scores through the multi-head attention mechanism (MHSA), and generate visually guided weighted instance features; Step 9: Key feature screening: Use the pooling layer and the fully connected layer to obtain the category probability of the instance, and select the top K instance-level features as key instance features based on the probability value; S4: Constructing instance-level text prompt description of cervical tissue: Step 10: Text description generation: Use ChatGPT4 to generate instance-level text description information prompts for cervical tissue pathological lesion sites and positive abnormal features; Step 11: Text feature conversion: convert the text description into text features through the pre-trained CLIP text encoder; S5, instance-level image-text pair alignment stage: Step 12: Feature alignment and optimization: Calculate the similarity between key instance features and corresponding text features, and update model parameters through lightweight training strategy to optimize image-text alignment effect; S6, instance-level feature aggregation stage: Step 13: Weighted feature aggregation: perform weighted aggregation based on the similarity scores between key instance features and text features to form a package-level feature that reflects the overall lesion features; S7. Construction of the lesion prompt library at the whole-slice image level: Step 14: Establishment of lesion library: Collect images of cervical tissue pathological lesions of various levels from CIN1 to CIN3 and their feature descriptions, and use ChatGPT4 to normalize the descriptions; extract image features and text features of lesion examples respectively, and build a lesion prompt library in the form of key-value pairs; S8. Package-level image feature retrieval text feature: Step 15, image and text retrieval: use the package-level features and the image features in the lesion prompt library to calculate the cosine similarity, and obtain the prediction results through label propagation aggregation; S9. Lesion diagnosis decision: Step 16: Prediction and decision: Combine the prediction results with the original package features, generate the final prediction features through a lightweight network, and combine the prediction results with the lesion prompt library information to make a diagnostic decision on the lesion grade range. 2. The method for classifying a small number of cervical panoramic images based on visual guidance and language prompts as claimed in claim 1, characterized in that the method for generating a visual priori guidance map of a panoramic image of cervical tissue described in steps 4 to 7 in S2, the method is based on the specific structure and pathological characteristics of cervical tissue, and it is considered that the distribution information of the dark-stained area of epithelial tissue has a natural priori property, and the specific steps are as follows: Step 4: Preprocessing of panoramic images of cervical tissue at low magnification: Step 401, obtaining a thumbnail of a panoramic image of the cervix at a magnification of 2.5x; Step 402: Perform dye normalization on the cervical panoramic image training set: First, convert RGB to Lab color space. The formula is as follows: L＝0.2126×R+0.7152×G+0.0722×B a＝0.5×(R-G)+128 b＝0.5×(R-B)+128 Here R, G, and B represent the red, green, and blue components of each pixel in the RGB image respectively; Then perform Lab channel normalization and then standardize with Z-sorce, that is, subtract the mean from the value of each channel and divide it by the standard deviation. The mean and standard deviation of the Lab channel can be obtained by calculating the statistics of the training data set. The formula is as follows: Where _Li represents the brightness value of the i-th pixel in the Lab image, and N represents the total number of pixels. Similarly, the mean and standard deviation of channel a and channel b can be calculated; Convert the normalized Lab image back to the RGB color space to get the normalized RGB image. The formula is as follows: R＝L+1.402×(b-128) G = L - 0.3441 × (a-128) - 0.7141 × (b-128) B＝L+1.772×(a-128) Step 403: perform image binarization and outer contour detection: After obtaining the dyed normalized image, the image is binarized and then the outer contour is detected. This method is based on image edge detection and connected domain marking technology. It accurately detects the outer contour of the object by finding the connected domain in the image and extracting the outer contour line. Then the image is segmented according to the largest connected domain in the detected outer contour line C, and the segmented images are scaled to a consistent size, such as 1024×1024; Step 5, calculating the boundary distance of the tissue block of the cervical tissue image; Step 501, calculate the centroid of the tissue block, and determine the centroid position based on the contour area and the first-order moment: According to the outer contour of the tissue slice obtained in the first step, the area A and the first-order moments M ₁₀ and M ₀₁ enclosed by the contour C are calculated, and then the centroid (cX, cY) is calculated based on the contour area and the first-order moment. The formula is as follows: Among them, _Ci represents the i-th contour, x and y are the horizontal and vertical coordinates of the pixel respectively; Step 502: Determine the pixel distance representation, by calculating the distance ratio from each pixel point in the contour to the straight line where the centroid is located, and obtain the image-level boundary distance representation in two directions: According to the tissue contour C _i , find the four vertices P ₁ (x ₁ ,y ₁ ), P ₂ (x ₂ ,y ₂ ), P ₃ (x ₃ ,y ₃ ), P ₄ (x ₄ ,y ₄ ) of the minimum circumscribed rectangle R of the contour, expressed as: {P ₁ ,P ₂ ,P ₃ ,P ₄ } = min Area Rect(C _i ) Then calculate the equations of the lines passing through the centroid and parallel to the sides of the circumscribed quadrilateral, and find the distances from each pixel point in the contour to the line, dist ₁ , dist ₂ , (x _p , y _p ) is a pixel point in the contour _Ci , and the formula is as follows: The ratio of the distance value of the pixel inside the contour to the distance of the point on the contour is calculated as the distance representation of the pixel point. In this way, the distance representation of the pixel point closer to the contour is closer to 1, and the distance representation of the pixel point outside the contour is recorded as -1. In this way, the image-level distance representation in two directions is formed, which is recorded as With/> Step 6: Visual prior region of interest (ROI) hard segmentation of cervical tissue image tissue block: Step 601: Channel binarization: First, the red (R) and blue (B) channels in the tissue block of the cervical tissue image are binarized using the Otsu threshold (OTSU) global threshold method. For the red channel grayscale value _Rp and blue channel grayscale value _Bp of any pixel point p, the optimal thresholds _TR and _TB of each channel are calculated to maximize the inter-class variance between the foreground (chromatin dense area) and the background. The calculation formula is as follows: Where ω ₀ and ω ₁ are the pixel proportions of the background and foreground, respectively, and μ ₀ and μ ₁ are the average grayscale values of the background and foreground, respectively; Then, according to the calculated threshold, the pixel value of each channel is compared with its threshold. The pixel value greater than the threshold is set to 1 (considered as chromatin-dense area), and the pixel value less than or equal to the threshold is set to 0 (considered as background). The binary segmentation of the red and blue channels is completed, and the binary images are recorded as R _bin and B _bin respectively; Step 602: Gradient enhancement and segmentation: Calculate the gradient enhanced image G(x,y) of the grayscale image I _grey after fuzzy denoising and other preprocessing, use the Sobel operator to obtain the horizontal and vertical gradients and then obtain the gradient magnitude image M(x,y), the formula is as follows: The watershed change algorithm is applied to process the gradient image to find the area where the gradient changes suddenly in the image. By constructing a topological map and setting the threshold segmentation range, the segmentation area is obtained and converted into a binary mask: Mask _w = Filter(Watered(I _grey ,M(x,y))) Finally, the region of interest is comprehensively delineated, and the above three binary images are logically operated to obtain the hard segmentation ROI (i, j) of the region of interest (ROI) of the cervical tissue section: Mask _C (i,j)＝R _bin (i,j)∨B _bin (i,j) ROI(i,j)＝Mask _C (i,j)∧Mask _W (i,j) Step 7: Generate visual prior guidance map of cervical tissue image tissue block: The image-level boundary distance representation obtained by combining the above two stages of cervical tissue image tissue block boundary distance calculation stage and tissue block visual prior region of interest (ROI) hard segmentation stage is And ROI hard division ROI(i,j), through weighted calculation to obtain the visual prior guidance map I _activate , the formula is as follows: Among them, α, β, and γ are hyperparameters with values between 0 and 1. Softmax maps the vector into a probability distribution to ensure that each element takes a value between 0 and 1. 3. The method for classifying cervical panoramic images with a small number of samples based on visual guidance and language prompts as claimed in claim 1, characterized in that the visually guided grid sampler technology described in step S2 is a division strategy for the irregular shape characteristics of cervical tissue and the coherence of the lesion area, so that the model can more deeply explore the evolution law and spatial connection of the lesion between different levels of the tissue, and the specific steps are as follows: Step 5: Grid sampling and visual prior position encoding: First, the visual prior guidance map I _activate is aligned with the cervical tissue slices; a panoramic image of cervical tissue is obtained at a magnification of 5x and a set of cervical tissue slices is obtained at a magnification of 40x respectively; then the visual prior guidance map I _activate obtained in step 2 is upsampled by bilinear interpolation and aligned with the panoramic image of cervical tissue at a magnification of 5x. The principle is as follows: Among them, Q ₁₁ (x ₁ ,y ₁ ),Q ₁₂ (x ₁ ,y ₂ ),Q ₂₁ (x ₂ ,y ₁ ),Q ₂₂ (x ₂ ,y ₂ ) are four adjacent points of the original image; The panoramic image of cervical tissue at 5x magnification is sliced with the interpolated weight map, and aligned with the tissue slice at 40x magnification, and the slice is upsampled again, and the weight and slice are mapped to construct the prior visual guidance slice set I _{activate_patch} ; Then, the visual prior-guided position encoding F _position is constructed through the position encoder. The encoder is constructed by a fully connected layer (FC), a multi-head attention layer (MHSA), and a layer normalization (LN). The calculation process is shown as follows: F _position = LN(MHSA(I _{activate_path} W _PE )) Step 6, image feature extraction: Use the pre-trained CLIP image encoder (ViT) to extract the image features F _{image_patch} of cervical tissue slices at 40x magnification; Step 7: Concatenate the generated visual prior position encoding feature F _position with the cervical tissue slice image feature F _{image_patch} , and then obtain the cervical tissue image instance-level feature F _{image_instance} through a fully connected layer (FC). The formula is as follows: F _{image_instance} = Concat(F _{image_patch} ,F _position )W _feat. 4. The method for classifying cervical panoramic images with a small number of samples based on visual guidance and language prompts according to claim 1, characterized in that the key instance feature extraction method described in steps 8 to 9 in S3 comprises the following specific steps: Step 8, instance-level feature processing: perform layer normalization on the instance-level feature F _{image_instance} of the cervical tissue image, and then introduce the multi-head attention module (MHSA) to calculate the attention score α _{image_instance} and add it to the instance-level feature F _{image_instance} to obtain the visually guided weighted instance feature F _{image_instance} ′, the formula is as follows: α _{image_instance} =MHSA(LN(F _{image_instance} )) F _{image_instance} ′＝α _{image_instance} F _{image_instance} Step 9, key feature screening: The visually guided weighted instance feature F _{image_instance} ′ is respectively passed through the pooling layer (MaxPooling) and the fully connected layer (FC) to obtain the instance category probability P _{image_instance} . According to the score, the top K instance-level features are selected as the key instance features F _{TOPK_instance} . The formula is as follows: P _{image_instance} =σ(Pool(F _{image_instance} ′)W _{image_instance} ) Then, the visually guided weighted instance feature F _{image_instance} ′ is randomly masked to form a feature sequence F _mask . By designing a position decoder PD, the feature F _{image_instance} ′ containing spatial and position information is decoded. The foreground and background information of the visual guidance map I _activate is used as supervision information to reconstruct the mask position, and then the visual guidance map I _activate ′ is updated to improve the spatial distribution semantic consistency and generalization ability. The distribution loss L _position is calculated and then continuously updated and iterated. The formula is as follows: F _mask = [f ₁ ^m ,f ₂ ^m ,...,f _n ^m ] (F′, F _p ′)＝LN(MHSA(F _mask ))W _PD 5. The method for classifying cervical panoramic images with a small number of samples based on visual guidance and language prompts according to claim 1, characterized in that the instance-level text prompt learning method described in steps 10 to 12 in S4 to S5 is summarized as follows: Step 10, text description generation: ChatGPT4 is used to generate a series of text prompt information for the location of cervical tissue pathological lesions and their positive and abnormal characteristics. ChatGPT4 is used to describe the three types of information, namely, cell type, cell morphology description, and whether there is an abnormality, and integrate them into instance-level text description information prompts; Step 11: Text feature conversion: The text description is converted into text features through the pre-trained CLIP text encoder, and the cosine similarity is calculated with the key instance feature F _{TOPK_instance} of cervical tissue extracted in the S3 stage to establish the association between instance-level images and texts; Step 12: Feature alignment and optimization: In order to reduce the computational level of CLIP model training in the task of semantic alignment of instance-level histopathological images and texts, the present invention designs a lightweight training strategy: the weights of the pre-trained CLIP image encoder and text encoder are frozen, and the training focuses on optimizing the learnable parameter Tokens of the image and text; the image Token T _{image_token} is obtained by passing the extracted key instance features through a lightweight network (LightNet) composed of two layers of multi-layer perceptrons (MLPs), and the text Token T _{text_token} is set to learn for each instance and capture the specific language description information of different pathological types to learn a more appropriate text description; therefore, the image T _{image_token} and the text T _{text_token} are concatenated to form a learnable parameter sequence T _token , the number of which can be designed by oneself, for example, L Tokens are set as empirical setting values, denoted as T _token ={t ₁ , t ₂ ,…, t _L }, and the corresponding instance-level label is denoted as Y ={y ₁ , y ₂ ,…, y _L }), and the cross-entropy loss (Cross-Entropy Loss) is used to measure the difference between the prediction results and the labels, and the back propagation error is used to update these tokens. The formula is as follows: T _token = Concat(LightNet(F _{TOPK_instance} ),T _{text_token} ) Among them, _pi is the probability that the model predicts the corresponding category label for the i-th Token, that is, p _i =P(t _i |F _{TOPK_instance} ,T _token ); Next, the pre-trained text encoder is used to obtain text features, and the cosine similarity is calculated with the key instance features of cervical tissue obtained in S3. The objective function formula is as follows: L _contrastive = (L _text2image + L _image2text )/2 where _Ii is the embedding vector of the i-th text description, _Tj is the embedding vector of the j-th slice image, the text-to-image contrastive loss is denoted as _Ltext2image , the image-to-text contrastive loss is denoted as _Limage2text , and the overall similarity contrastive loss is denoted as _{Lcontrastive.} 6. The method for classifying cervical panoramic images with a small number of samples based on visual guidance and language prompts according to claim 1, characterized in that the package-level text prompt learning method described in steps 13 to 16 in S6 to S9 is summarized as follows: Step 13: Weighted feature aggregation: By calculating the similarity scores between instance-level image features and text features, the key instance features extracted from S3 are weightedly aggregated to form bag-level features F _Bag ; Step 14, lesion library establishment: by systematically collecting the descriptions and legends of cervical histopathology graded lesions in the cervical histopathology atlas, we regard them as authoritative expert knowledge reserves, and further use the advanced artificial intelligence model ChatGPT4 to deeply integrate and sort out these professional knowledge, aiming to convert the traditional subjective description method of medical experts into a standardized and intelligent description system, so as to effectively bridge the gap in the knowledge transfer process, and then use the pre-trained CLIP image encoder and text encoder to extract the corresponding image features F _key and text features F _value : construct image features and text features to form multiple key-value pairs (Key-Value), recorded as F _{key_value} ; Step 15, image and text retrieval: Calculate the similarity score A _{Bag_key} between the bag-level feature F _Bag and the image feature F _key , and use the key-value pair F _{key_value} to aggregate with F _value through label propagation to form the prediction F _Bag ′ to realize knowledge retrieval. The formula is as follows: A _{Bag_key} = exp(-β(1-F _Bag F _key ^T )) F _Bag ′＝A _{Bag_key} F _value Step 16, prediction and decision: According to the obtained prediction F _Bag ′ and the original bag feature F _Bag , the prediction feature is obtained through the lightweight network LightNet composed of two layers of MLP. The LightNet weight is recorded as W _c . The two are weighted summed by residual connection to obtain the final prediction result logits _Bag . This result obtains both instance-level data information and Few-shot knowledge. The prediction result calculation formula is as follows: The grade range of the lesion is determined based on the category prediction, and combined with the corresponding description information in the lesion prompt library as the basis for the final diagnosis.