CN116168403A

CN116168403A - Medical data classification model training method, classification method, device and related medium

Info

Publication number: CN116168403A
Application number: CN202310073285.0A
Authority: CN
Inventors: 刘伟华; 左勇; 刘磊; 林超超
Original assignee: Athena Eyes Co Ltd
Current assignee: Athena Eyes Co Ltd
Priority date: 2023-01-17
Filing date: 2023-01-17
Publication date: 2023-05-26

Abstract

The invention relates to the field of data classification, and discloses a medical data classification model training method, a classification method, a device, computer equipment and a storage medium, wherein the classification method comprises the following steps: acquiring medical text data and medical image data; inputting the medical text data and the medical image data into a medical data classification model, wherein the medical data classification model is the model; based on a feature extraction module, text feature extraction is carried out on the medical text data to obtain text features, and image feature extraction is carried out on the medical image data to obtain image features; based on a transducer module, attention calculation, fusion processing and classification processing are sequentially carried out on the text features and the image features, so that classification results are obtained, and the classification accuracy of medical data classification is improved by adopting the method and the device.

Description

Medical data classification model training method, classification method, device and related medium

Technical Field

The present invention relates to the field of data classification, and in particular, to a medical data classification model training method, a medical data classification device, a medical data classification computer device, and a medical data storage medium.

Background

With the development of artificial intelligence technology, technological revolution and industrial revolution are affecting various industries. The medical field is also an important field of research on artificial intelligence technology at home and abroad at present, and particularly in recent years, various information processing technologies are gradually applied to medical data classification.

The current medical data classification mainly includes classification based on a rule, classification based on a machine learning or deep learning, classification based on a knowledge graph, and the like. The classification based on the rule mode refers to a process of performing association rule analysis on medical data by utilizing a medical database to obtain a classification result. The classification based on the machine learning or the deep learning means a process of extracting text features of medical data or extracting image features of medical data by using the machine learning or the deep learning, and analyzing the text features or the image features to obtain a classification result. The classification based on the knowledge graph means that the medical data is subjected to entity extraction, and the classification result is rapidly determined based on the knowledge graph.

However, the method has the following problems: (1) The medical rule is manually constructed in a rule-based mode, and the consistency of rule marking of the same disease cannot be achieved due to uneven professional knowledge levels of different medical professionals, so that the problem of low classification accuracy of the rule-based classification method is caused; (2) The mode based on machine learning or deep learning and the mode based on the knowledge graph are all to obtain a classification result by analyzing single-mode information, but the problem of low classification accuracy is caused by the fact that the single-mode information contains less medical information.

Therefore, the existing medical data classification has the technical problem of low classification accuracy.

Disclosure of Invention

The embodiment of the invention provides a medical data classification model training method, a medical data classification device, computer equipment and a storage medium, so as to improve the classification accuracy of medical data classification.

In order to solve the above technical problems, an embodiment of the present application provides a medical data classification model training method, including:

constructing an initial medical data classification model based on a feature extraction module and a transducer module, wherein the feature extraction module comprises an image feature extraction unit and at least two text feature extraction units, the weight of the image feature extraction unit is dynamically changed according to a preset rule, and the weight of the text feature extraction unit is fixed;

acquiring medical image training data and medical text training data, and inputting the medical image training data and the medical text training data into the initial medical data classification model;

based on each text feature extraction unit, text feature extraction is carried out on the medical text training data respectively, and text features corresponding to each text feature extraction unit are obtained;

based on the image feature extraction unit, extracting image features from the medical image training data to obtain image features;

taking the image features and all the text features as input of the transducer module, and carrying out feature fusion and classification processing based on the transducer module to determine classification results;

and if the classification result meets the preset condition, taking the obtained model as a medical data classification model.

In order to solve the above technical problems, an embodiment of the present application provides a classification method, including:

acquiring medical text data and medical image data;

inputting the medical text data and the medical image data into a medical data classification model, wherein the medical data classification model is the model;

based on a feature extraction module, text feature extraction is carried out on the medical text data to obtain text features, and image feature extraction is carried out on the medical image data to obtain image features;

and based on a transducer module, performing attention calculation, fusion processing and classification processing on the text features and the image features in sequence to obtain classification results.

To solve the above technical problem, an embodiment of the present application provides a medical data classification model training apparatus, including:

the model construction module is used for constructing an initial medical data classification model based on the feature extraction module and the transducer module, wherein the feature extraction module comprises an image feature extraction unit and at least two text feature extraction units, the weight of the image feature extraction unit is dynamically changed according to a preset rule, and the weight of the text feature extraction unit is fixed;

the data acquisition module is used for acquiring medical image training data and medical text training data and inputting the medical image training data and the medical text training data into the initial medical data classification model;

the text feature extraction module is used for respectively extracting text features of the medical text training data based on each text feature extraction unit to obtain text features corresponding to each text feature extraction unit;

the image feature extraction module is used for extracting image features of the medical image training data based on the image feature extraction unit to obtain image features;

the classification result determining module is used for taking the image features and all the text features as the input of the transducer module, and carrying out feature fusion and classification processing based on the transducer module to determine classification results;

and the medical data classification model determining module is used for taking the obtained model as a medical data classification model if the classification result meets the preset condition.

In order to solve the above technical problem, an embodiment of the present application further provides a classification device, including:

the data acquisition module is used for acquiring medical text data and medical image data;

the input module is used for inputting the medical text data and the medical image data into a medical data classification model, wherein the medical data classification model is the model;

the feature extraction module is used for extracting text features of the medical text data based on the feature extraction module to obtain text features, and extracting image features of the medical image data to obtain image features;

and the classification result acquisition module is used for sequentially performing attention calculation, fusion processing and classification processing on the text features and the image features based on the transducer module to obtain a classification result.

In order to solve the above technical problem, an embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the steps of the medical data classification model training method when executing the computer program, or implements the steps of the classification method when executing the computer program.

To solve the above technical problem, embodiments of the present application further provide a computer readable storage medium storing a computer program, where the computer program implements the steps of the medical data classification model training method described above when executed by a processor, or implements the steps of the classification method described above when executed by the processor.

The medical data classification model training method, the medical data classification device, the computer equipment and the storage medium provided by the embodiment of the invention construct an initial medical data classification model based on the feature extraction module and the transducer module; acquiring medical image training data and medical text training data, and inputting the medical image training data and the medical text training data into the initial medical data classification model; based on each text feature extraction unit, text feature extraction is carried out on the medical text training data respectively, and text features corresponding to each text feature extraction unit are obtained; based on the image feature extraction unit, extracting image features from the medical image training data to obtain image features; taking the image features and all the text features as input of the transducer module, and carrying out feature fusion and classification processing based on the transducer module to determine classification results; and if the classification result meets the preset condition, the obtained model is used as a medical data classification model, the medical data classification model is trained by the method, so that medical image data and medical text data are comprehensively analyzed based on the medical data classification model, the dynamic change of the weight of the medical image data is controlled, the weight of the medical text data is fixed, the multi-mode feature analysis is performed, and the classification accuracy of medical data classification is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of one embodiment of a medical data classification model training method of the present application;

FIG. 2 is a flow chart of one embodiment of a classification method of the present application;

FIG. 3 is a schematic structural view of one embodiment of a medical data classification model training apparatus according to the present application;

FIG. 4 is a schematic structural view of one embodiment of a sorting apparatus according to the present application;

FIG. 5 is a schematic structural diagram of one embodiment of a computer device according to the present application.

Detailed Description

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description and claims of the present application and in the description of the figures above are intended to cover non-exclusive inclusions. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Referring to fig. 1, fig. 1 shows a medical data classification model training method provided in an embodiment of the present application, including:

s101, constructing an initial medical data classification model based on a feature extraction module and a transducer module, wherein the feature extraction module comprises an image feature extraction unit and at least two text feature extraction units, the weight of the image feature extraction unit is dynamically changed according to a preset rule, and the weight of the text feature extraction unit is fixed.

In step S101, the above-described feature extraction module refers to a module for extracting features of medical data.

The above-mentioned transducer module is a module for performing deep learning on the extracted image features and text features by using a transducer system.

The medical data includes medical text data and medical image data.

Specifically, the image feature extraction unit is used for extracting feature information of medical image data, and the text feature extraction unit is used for extracting feature information of medical text data.

Specifically, the text feature extraction unit includes, but is not limited to, an electronic medical record feature extraction unit and a knowledge graph feature extraction unit, wherein the electronic medical record feature extraction unit is used for extracting text features of the electronic medical record, and the knowledge graph feature extraction unit is used for extracting text features based on the knowledge graph.

The electronic medical record feature extraction unit may be a robberta model (brute force optimization BERT method), and the text features corresponding to the electronic medical record are obtained by inputting the user basic information and the user complaint information in the electronic medical record into the pre-trained robberta model.

The knowledge graph feature extraction unit may link a disease of a diagnosis result of a doctor in the electronic medical record to a disease entity of the knowledge graph, and acquire nodes within two hops with the disease as a center as a subgraph. And generating disease text data according to the actual node relation by using all the structured information in the subgraph, inputting the disease text data into a RoBerta model which is trained to obtain hidden layer characteristics, and taking the obtained hidden layer characteristics as text characteristics corresponding to the knowledge graph priori.

The preset rule refers to a rule for controlling the weight of the image feature extraction unit. For example, the weight of the control image feature extraction unit is changed in the following sequence (0.2,0.4,0.6,0.8,1.0).

The weight of the text feature extraction unit is a fixed value, for example, the weight in the text feature extraction unit is controlled to be fixed at 0.4.

The medical image data is affected by the quality of the image, and the feature extraction is required to be carried out on the medical image data through different weights so as to ensure the accuracy of the extracted image features, and convolution operation is required to be carried out on the extracted image features, so that the image features and the text features keep the same dimension, the accuracy is ensured, and meanwhile, the smooth fusion of the image features and the text features is ensured, so that the classification accuracy of medical data classification is improved.

S102, acquiring medical image training data and medical text training data, and inputting the medical image training data and the medical text training data into an initial medical data classification model.

In step S102, the medical image training data and the medical text training data are associated data.

S103, based on each text feature extraction unit, text feature extraction is carried out on the medical text training data respectively, and text features corresponding to each text feature extraction unit are obtained.

In step S103, specifically, based on the electronic medical record feature extraction unit, feature extraction is performed on the electronic medical record in the medical text training data, so as to obtain the electronic medical record feature. And based on the knowledge graph feature extraction unit, extracting text features of the medical text training data to obtain knowledge graph features.

S104, based on the image feature extraction unit, image feature extraction is carried out on the medical image training data, and image features are obtained.

In step S104, specifically, image feature extraction is performed on the medical image training data based on the residual neural network, and a continuous feature sequence is generated.

And carrying out convolution processing on the continuous feature sequence to obtain image features, wherein the dimensions of the image features are the same as those of the text features.

The residual neural network is preferably a ResNet residual neural network.

Based on the residual neural network, extracting image features of the medical image training data according to weights set by preset rules, and generating a continuous feature sequence, wherein the continuous feature sequence is continuous features extracted from the medical image training data.

The medical image is affected by the quality of the image, and the feature extraction is required to be carried out on the medical image training data through different weights so as to ensure the accuracy of the extracted image features, and meanwhile, the convolution operation is required to be carried out on the extracted image features, so that the image features and the text features keep the same dimension, the accuracy is ensured, and meanwhile, the smooth fusion of the image features and the text features is ensured, so that the classification accuracy of medical data classification is improved.

S105, taking the image features and all text features as input of a transducer module, and carrying out feature fusion and classification processing based on the transducer module to determine classification results.

In step S105, specifically, the image features and all text features are taken as input of the transducer module. And respectively carrying out attention calculation on the image features and the text features based on the attention layer of the transducer module to obtain attention image features and attention text features. And carrying out feature fusion on the attention image features and the attention text features to obtain fusion features. And carrying out pooling and classification treatment on the fusion characteristics, and determining classification results.

S106, if the classification result meets the preset condition, taking the obtained model as a medical data classification model.

In step S106, the preset condition refers to the convergence of the loss function. The loss function can be specifically adjusted according to actual conditions.

Preferably, the present application employs a cross entropy loss function.

In this embodiment, the medical data classification model is obtained through training by the method, so that the medical image data and the medical text data are comprehensively analyzed based on the medical data classification model, the medical text data weight is fixed by controlling the dynamic change of the medical image data weight, the multi-mode feature analysis is performed, and the classification accuracy of medical data classification is improved.

Referring to fig. 2, fig. 2 shows a classification method according to an embodiment of the present invention, and the method is applied to the medical data classification model in fig. 1 as an example, and is described in detail as follows:

s201, acquiring medical text data and medical image data.

S202, inputting medical text data and medical image data into a medical data classification model, wherein the medical data classification model is the model.

S203, based on the feature extraction module, text feature extraction is carried out on the medical text data to obtain text features, and image feature extraction is carried out on the medical image data to obtain image features.

S204, based on a transducer module, attention calculation, fusion processing and classification processing are sequentially carried out on the text features and the image features, and classification results are obtained.

In step S201, the medical text data and the medical image data are associated data. That is, when the medical text data is related to lung cancer, the above medical image data may be a lung CT image.

In step S203, specifically, image feature extraction is performed on the medical image data based on the image feature extraction unit in the feature extraction module, so as to obtain image features; and based on each text feature extraction unit in the feature extraction module, extracting text features of the medical text data to obtain text features corresponding to each text feature extraction unit.

It should be appreciated that the number of text features extracted by the feature extraction module corresponds to the number of text feature extraction units.

In the embodiment, the medical image data and the medical text data are comprehensively analyzed through the medical data classification model, the weight of the medical text data is fixed and unchanged through controlling the dynamic change of the weight of the medical image data, the multi-mode feature analysis is performed, and the classification accuracy of medical data classification is improved.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.

Fig. 3 shows a schematic block diagram of a medical data classification model training apparatus in one-to-one correspondence with the medical data classification model training method of the above embodiment. As shown in fig. 3, the medical data classification model training apparatus includes a model construction module 31, a data acquisition module 32, a text feature extraction module 33, an image feature extraction module 34, a classification result determination module 35, and a medical data classification model determination module 36. The functional modules are described in detail as follows:

the model construction module 31 is configured to construct an initial medical data classification model based on a feature extraction module and a transducer module, where the feature extraction module includes an image feature extraction unit and at least two text feature extraction units, the weights of the image feature extraction units dynamically change according to a preset rule, and the weights of the text feature extraction units are fixed.

The data acquisition module 32 is configured to acquire medical image training data and medical text training data, and input the medical image training data and the medical text training data into an initial medical data classification model.

The text feature extraction module 33 is configured to perform text feature extraction on the medical text training data based on each text feature extraction unit, so as to obtain text features corresponding to each text feature extraction unit.

The image feature extraction module 34 is configured to perform image feature extraction on the medical image training data based on the image feature extraction unit, so as to obtain image features.

The classification result determining module 35 is configured to take the image feature and all text features as input of the transducer module, and perform feature fusion and classification processing based on the transducer module to determine a classification result.

The medical data classification model determining module 36 is configured to take the obtained model as a medical data classification model if the classification result meets a preset condition.

In some optional implementations of this embodiment, the text feature extraction unit includes an electronic medical record feature extraction unit and a knowledge graph feature extraction unit, where the electronic medical record feature extraction unit is configured to extract text features of the electronic medical record, and the knowledge graph feature extraction unit is configured to extract text features based on the knowledge graph.

In some alternative implementations of the present embodiment, the text feature extraction module 33 includes:

the electronic medical record feature extraction unit is used for carrying out feature extraction on the electronic medical record in the medical text training data based on the electronic medical record feature extraction unit to obtain the electronic medical record feature.

The knowledge graph feature extraction unit is used for extracting text features of the medical text training data based on the knowledge graph feature extraction unit to obtain knowledge graph features.

In some alternative implementations of the present embodiment, the image feature extraction module 34 includes:

and the image feature extraction unit is used for extracting image features of the medical image training data based on the residual neural network to generate a continuous feature sequence.

And the convolution processing unit is used for carrying out convolution processing on the continuous feature sequence to obtain image features, wherein the dimensions of the image features are the same as those of the text features.

In some alternative implementations of the present embodiment, the classification result determination module 35 includes:

and the input unit is used for taking the image characteristics and all the text characteristics as the input of the transducer module.

And the processing unit is used for respectively carrying out attention calculation on the image characteristics and the text characteristics based on the attention layer of the transducer module to obtain attention image characteristics and attention text characteristics.

And the fusion unit is used for carrying out feature fusion on the attention image features and the attention text features to obtain fusion features.

And the classification unit is used for carrying out pooling and classification processing on the fusion characteristics and determining classification results.

For specific limitations on the medical data classification model training apparatus, reference may be made to the above limitations on the medical data classification model training method, and no further description is given here. The above-described modules in the medical data classification model training apparatus may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

Fig. 4 shows a schematic block diagram of a classification apparatus in one-to-one correspondence with the classification method of the above embodiment. As shown in fig. 4, the classification apparatus includes a data acquisition module 41, an input module 42, a feature extraction module 43, and a classification result acquisition module 44. The functional modules are described in detail as follows:

the data acquisition module 41 is used for acquiring medical text data and medical image data.

The input module 42 is configured to input the medical text data and the medical image data into a medical data classification model, where the medical data classification model is the model.

The feature extraction module 43 is configured to perform text feature extraction on the medical text data based on the feature extraction module to obtain text features, and perform image feature extraction on the medical image data to obtain image features.

The classification result obtaining module 44 is configured to sequentially perform attention calculation, fusion processing, and classification processing on the text feature and the image feature based on the transducer module, so as to obtain a classification result.

For specific limitations of the sorting apparatus, reference may be made to the above limitations of the sorting method, which are not repeated here. The respective modules in the above-described sorting apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 5, fig. 5 is a basic structural block diagram of a computer device according to the present embodiment.

The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It is noted that only a computer device 4 having a component connection memory 41, a processor 42, a network interface 43 is shown in the figures, but it is understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.

The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.

The memory 41 includes at least one type of readable storage medium including flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or D interface display memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 4. Of course, the memory 41 may also comprise both an internal memory unit of the computer device 4 and an external memory device. In this embodiment, the memory 41 is typically used for storing an operating system and various application software installed on the computer device 4, such as program codes for controlling electronic files, etc. Further, the memory 41 may be used to temporarily store various types of data that have been output or are to be output.

The processor 42 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute a program code stored in the memory 41 or process data, such as a program code for executing control of an electronic file.

The network interface 43 may comprise a wireless network interface or a wired network interface, which network interface 43 is typically used for establishing a communication connection between the computer device 4 and other electronic devices.

The present application also provides another embodiment, namely, a computer readable storage medium storing an interface display program, where the interface display program is executable by at least one processor to cause the at least one processor to perform the steps of the medical data classification model training method as described above, or to cause the at least one processor to perform the steps of the classification method as described above.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method described in the embodiments of the present application.

It is apparent that the embodiments described above are only some embodiments of the present application, but not all embodiments, the preferred embodiments of the present application are given in the drawings, but not limiting the patent scope of the present application. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a more thorough understanding of the present disclosure. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing, or equivalents may be substituted for elements thereof. All equivalent structures made by the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the protection scope of the application.

Claims

1. A medical data classification model training method, characterized in that the medical data classification model training method comprises:

2. The medical data classification model training method of claim 1, wherein the text feature extraction unit comprises an electronic medical record feature extraction unit and a knowledge-graph feature extraction unit, wherein the electronic medical record feature extraction unit is configured to extract text features of an electronic medical record, and the knowledge-graph feature extraction unit is configured to extract text features based on a knowledge graph.

3. The method for training a medical data classification model according to claim 2, wherein the step of extracting text features from the medical text training data based on each text feature extraction unit to obtain text features corresponding to each text feature extraction unit comprises:

based on the electronic medical record feature extraction unit, extracting features of the electronic medical record in the medical text training data to obtain electronic medical record features;

and based on the knowledge graph feature extraction unit, extracting text features of the medical text training data to obtain knowledge graph features.

4. The medical data classification model training method of claim 1, wherein the step of extracting image features from the medical image training data based on the image feature extraction unit includes:

based on a residual neural network, extracting image features of the medical image training data to generate a continuous feature sequence;

5. The medical data classification model training method of claim 1, wherein the step of using the image features and all the text features as inputs to the transducer module and performing feature fusion and classification processing based on the transducer module, and determining classification results comprises:

taking the image features and all the text features as inputs of the transducer module;

based on the attention layer of the transducer module, respectively carrying out attention calculation on the image features and the text features to obtain attention image features and attention text features;

carrying out feature fusion on the attention image feature and the attention text feature to obtain a fusion feature;

and carrying out pooling and classification treatment on the fusion characteristics, and determining classification results.

6. A method of classification, the method comprising:

acquiring medical text data and medical image data;

inputting the medical text data and the medical image data into a medical data classification model, wherein the medical data classification model is the model of any one of claims 1 to 5;

7. A medical data classification model training apparatus, characterized in that the medical data classification model training apparatus comprises:

8. A sorting apparatus, the sorting apparatus comprising:

an input module for inputting the medical text data and the medical image data into a medical data classification model, wherein the medical data classification model is a model according to any one of claims 1 to 5;

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the medical data classification model training method according to any one of claims 1 to 5 when executing the computer program or the classification method according to claim 6 when the processor executes the computer program.

10. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the medical data classification model training method of any of claims 1 to 5, or wherein the computer program when executed by a processor implements the classification method of claim 6.