KR102693667B1 - Apparatus and method for predicting discharge of inpatients - Google Patents

Abstract

A device for predicting discharge of a patient according to one embodiment may include a processor for applying a machine learning model to input data including medical data collected from a date of admission to a first time point during a period of hospitalization of a patient hospitalized at a first time point, thereby obtaining a first likelihood score of discharge of the patient within a target period from the first time point, and predicting whether the patient will be discharged from the first time point within the target period based on the obtained first likelihood score. The processor may, in response to a case in which the patient is hospitalized at a second time point after the first time point, update medical data collected during the period of hospitalization based on the medical data collected from the first time point to the second time point, and obtain a second likelihood score of discharge of the patient within the target period from the second time point by applying the machine learning model to input data including the updated medical data, and predicting whether the patient will be discharged from the second time point within the target period based on the obtained second likelihood score.

Description

{APPARATUS AND METHOD FOR PREDICTING DISCHARGE OF INPATIENTS}

Hereinafter, a technology related to a device and method for predicting discharge of a hospitalized patient is disclosed.

Effective resource management in hospitals can improve the quality of healthcare services by reducing the labor-intensive burden of personnel, reducing the waiting time of hospitalized patients, and ensuring optimal treatment time. Utilization of hospital processes requires effective bed management, and a patient's hospitalization longer than the optimal treatment time can hinder bed management. Predicting the length of stay of a patient can help make wise decisions about bed management.

Utilization of expensive and scarce human and material resources may be essential for efficient operation of hospital processes. Hospitals may be required to improve overall management efficiency while handling various resources such as medical staff and medical staff schedule management, bed management, and clinical pathway management. Effective resource management of hospitals can improve the quality of care by reducing the labor-intensive burden of personnel, reducing the waiting time of hospitalized patients, and securing optimal treatment time. Hospital resource management may include bed management. In most hospitals today, clinicians can manually check the status of patients and decide whether to continue hospitalization or discharge them. Based on the aforementioned decision, medical staff and staff can determine the bed capacity in the near future and schedule patients. The number of patients hospitalized for various chronic and acute diseases such as cardiovascular disease (CVD) is steadily increasing, and readmission or complications due to inadequate treatment may occur. If patients are hospitalized longer than the optimal treatment time, effective bed management may become difficult. It may be important to accurately predict the length of hospitalization of patients and carefully decide on discharge.

Many studies have focused on the efficiency of hospital resources, and most of them can suggest algorithms or models to improve bed management. Integer linear programs can be proposed to investigate bed planning and solve optimization problems. Simulated bed occupancy schedules can be explained. In addition, bed simulation for surgical patients can be studied using Monte Carlo simulation to figure out ICU capacity. In particular, the expected length of stay (LOS) is one of the information required for bed management, and LOS can be predicted based on electronic health records (EHR). Machine learning (ML)-based models can be used to predict LOS, long-term hospitalization, and unplanned readmissions, and to find biomarkers for critical illness. Recently, many studies on interpretable or explainable artificial intelligence (XAI) can be conducted. In one of the XAI studies, a model can be developed to predict acute diseases and provide both results and interpretation. Compared to EHR, research utilizing computer vision results of imaging techniques can be promoted more actively because it can directly visualize important parts of the image. To support bed management, ML-based predictive models can be developed to provide daily discharge probabilities and “individual descriptors” that visualize the affected features.

delete

Korean Patent Publication No. 10-2021-0113042 International Publication WO2021/028961 United States Patent Application Publication No. US2021/0005321

A machine learning (ML)-based prediction model can be developed to predict the discharge probability of hospitalized patients with cardiovascular diseases (CVDs). The results of the prediction model can be evaluated, and the major risk factors of hospitalized patients can be explained for personalized treatment. Bed schedules can be managed efficiently, and long-term hospitalization patients can be detected in advance. The utilization of hospital processes can be improved, and the quality of medical services can be improved.

Patients with chronic and acute diseases, including cardiovascular diseases, may have high rates of hospitalization, readmission, and complications. There may be alternatives to transfer to other hospitals to resolve treatment or delay in admission that causes serious problems. Hospitals should continuously seek fundamental measures to reduce waiting times, and efficient bed management is one of them. Due to the diversity of diseases, it may be more advantageous to find common risk factors and implement bed management for specific departments or diseases (e.g., clustered specific wards) and then expand to the hospital level. Developing an ML-based model and predicting the discharge of patients hospitalized with CVD can determine available bed capacity in the near future and discover risk factors. By providing persuasive discharge information such as expected discharge date for each individual and cardiovascular disease risk factors, it can assist medical staff in manual accurate bed management in practice.

To evaluate the results of prediction and explain the major risk factors of hospitalized patients for individual treatment, individual descriptors are proposed. Although patients have the same disease and have common variables indicating the disease, each patient may have different characteristics, medical history, circumstances, and treatments. It may be required to identify and monitor unique individual variables for each patient. The results of the ML-based prediction model may include information on the daily discharge of patients as well as the contribution of features such as feature importance. The daily discharge probability of each patient and the features that affect each patient during the hospitalization period can be visualized. Individual descriptors can help medical teams and patients obtain reasonable grounds for the results of the ML-based model, understand the conditions in detail, and prepare for treatment in advance. Individual analysis can focus on each patient, and the identified meaningful features can be used in other studies as a basis for pre-identifying variables that affect hospitalization.

It can help to manage bed reservations efficiently and detect long-term patients in advance. Bed management can refer to the process of designating patients with the highest probability of discharge, securing the number of available beds, and assigning beds to patients waiting after admission reservations. Since the process is complex and usually performed manually, it can be intended to support the process by providing the expected LOS and discharge probability returned from the model and recognizing bed capacity in the near future. Not only patients with a high probability of discharge, but also patients with a consistently low probability of discharge can be detected. In other words, the reasons for long-term hospitalization of high-risk patients can be identified and analyzed and provided to the management team.

In summary, an ML-based model can be developed to predict whether a patient hospitalized with CVD will be discharged within 3 days. Based on the model, individual descriptors can be proposed, and a bed management simulation including the discharge probability and the affected features such as demographics, prescribed medications, and treatments can be illustrated. It can help improve the efficient utilization of hospital resources and improve the quality of healthcare services.

However, technical challenges are not limited to the technical challenges described above, and other technical challenges may exist.

You can set the cohort criteria and extract data from CardioNet, a manually curated database specializing in CVD. You can process the data to reindex the date index, merge the current features with the past features from 3 years ago, and impute missing values to create a suitable dataset. You can train and evaluate ML-based predictive models to find sophisticated models. You can predict the probability of discharge within 3 days and explain the results by identifying, quantifying, and visualizing the features of the model.

We developed an ML-based predictive model to predict the probability of discharge within 3 days for each cardiovascular disease patient and obtain individual LOS.

A device for predicting discharge of a patient according to an embodiment of the present invention may include a processor for applying a machine learning model to input data including medical data collected from a date of hospitalization to a first time point during a period of hospitalization of a patient hospitalized at a first time point, thereby obtaining a first likelihood score that the patient will be discharged from the hospital within a target period from the first time point, and predicting whether the patient will be discharged from the hospital within the target period from the first time point based on the obtained first likelihood score.

The processor may obtain the first likelihood score by applying the machine learning model to input data including medical data regarding one or more combinations of operation, procedure, Picture Archiving and Communication System (PACS), diagnosis, medication, laboratory, and physical collected from the date of admission to the first time point for the patient.

The processor, in response to a case where the patient is hospitalized at a second point in time after the first point in time, updates medical data collected during the hospitalization period based on medical data collected from the first point in time to the second point in time, and applies the machine learning model to input data including the updated medical data to obtain a second likelihood score that the patient will be discharged from the hospital within a target period from the second point in time, and predicts whether the patient will be discharged from the hospital within the target period from the second point in time based on the obtained second likelihood score.

The above processor can obtain the first likelihood score by applying the machine learning model to input data including medical data collected during the hospitalization period together with medical data collected during a predefined period prior to the hospitalization period.

The processor may obtain the first likelihood score by applying the machine learning model to input data including medical data collected during the period of hospitalization and medical data regarding one or more combinations of diagnosis, medication, laboratory, physical, and length of stay (LOS) in an intensive care unit (ICU) collected for the patient during a predefined period prior to the period of hospitalization.

The above processor may select one or more features of the features of the collected data as features of the input data of the machine learning model based on feature importance of each feature for a temporary machine learning model trained based on all features of the collected data, train the machine learning model based on the selected features, and obtain the first likelihood score by applying the machine learning model to the input data including the selected features.

The above processor can select one or more features as inputs of the machine learning model by applying a recursive feature elimination with cross validation (RFECV) technique to the features of the input data.

The above processor can obtain the first likelihood score by applying an extreme gradient boost (XGB) model to the input data.

The processor may select one or more of the features based on a feature influence corresponding to a score induced by each feature of the input data with respect to the acquired first likelihood score.

The device for predicting discharge of a patient according to one embodiment may further include a display that displays feature influences of the selected one or more features with respect to the acquired first likelihood score.

A method for predicting discharge of a patient according to one embodiment may include the steps of: obtaining a first likelihood score of a patient being discharged from the hospital within a target period from the first time point by applying a machine learning model to input data including medical data collected from a date of hospitalization to the first time point during the hospitalization period of a patient hospitalized at a first time point; and predicting whether the patient will be discharged from the hospital within the target period from the first time point based on the obtained first likelihood score.

The step of obtaining the first likelihood score may include the step of obtaining the first likelihood score by applying the machine learning model to input data including medical data regarding one or more combinations of operation, procedure, Picture Archiving and Communication System (PACS), diagnosis, medication, laboratory, and physical collected from the date of admission to the first time point for the patient.

A method for predicting discharge of a patient according to one embodiment may further include: in response to a case where the patient is hospitalized at a second point in time after the first point in time, updating medical data collected during the hospitalization period based on the medical data collected from the first point in time to the second point in time; obtaining a second likelihood score that the patient will be discharged from the hospital within a target period from the second point in time by applying the machine learning model to input data including the updated medical data; and predicting whether the patient will be discharged from the hospital within the target period from the second point in time based on the obtained second likelihood score.

The step of obtaining the first likelihood score may include the step of obtaining the first likelihood score by applying the machine learning model to input data including medical data collected during the hospitalization period together with medical data collected during a predefined period prior to the hospitalization period.

The step of obtaining the first likelihood score may include the step of obtaining the first likelihood score by applying the machine learning model to input data including medical data collected during the period of hospitalization and medical data regarding one or more combinations of diagnosis, medication, laboratory, physical, and length of stay (LOS) in an intensive care unit (ICU) collected for the patient during a predefined period prior to the period of hospitalization.

A method for predicting discharge of a patient according to one embodiment may further include: selecting one or more features of the collected data as features of the input data of the machine learning model based on feature importance of each feature to a temporary machine learning model trained based on all features of the collected data; and training the machine learning model based on the selected features; wherein the step of obtaining the first likelihood score may include obtaining the first likelihood score by applying the machine learning model to the input data including the selected features.

The step of selecting one or more features as features of the input of the machine learning model may include the step of selecting one or more features as input of the machine learning model by applying a recursive feature elimination and cross validation (RFECV) technique to features of the input data.

The step of obtaining the first likelihood score may include a step of obtaining the first likelihood score by applying an extreme gradient boost (XGboost) model to the input data.

A method for predicting discharge of a patient according to one embodiment may further include a step of selecting one or more of the features based on a feature influence corresponding to a score induced by each feature of the input data with respect to the obtained first likelihood score.

A method for predicting discharge of a patient according to one embodiment may further include a step of displaying feature influences of the selected one or more features with respect to the obtained first likelihood score.

A method for predicting discharge of a patient according to one embodiment may further include the step of displaying likelihood scores for multiple time points during the hospitalization period.

A computer program according to one embodiment may be stored in a computer-readable recording medium to execute any one of the above-described methods in combination with hardware.

Five ML-based models can be tested using five-fold cross-validations. The final model, Extreme Gradient Boosting (XGB), can achieve an average AUROC (area under receiver operating characteristic) score of 0.865 higher than other models (e.g., logistic regression, random forest, support vector machine, and multilayer perceptron). Feature reduction (e.g., feature selection) can be performed, feature importance can be expressed, and prediction results can be evaluated. One of the results, the individual explainer, can provide hospital discharge scores and daily feature influence scores to medical staff and patients. To utilize the results, simulated bed management can be visualized.

The present invention can propose individual descriptors based on the developed ML-based predictive model that provides discharge probability and relative contribution of features. The device and method according to the present invention can assist the medical team and patients in identifying individual and common risk factors of CVD and hospital administrators in improving the management of beds and other resources.

Figure 1 illustrates the operation of a device for predicting discharge of a patient according to one embodiment.
Figure 2 illustrates the overall flow of a method for predicting patient discharge according to one embodiment.
Figure 3 illustrates medical data according to one embodiment.
Figure 4 illustrates a preprocessing process for obtaining medical data from raw medical data according to an embodiment.
Figure 5 illustrates labeling of medical data according to one embodiment.
FIG. 6 illustrates obtaining a probability score and predicting whether or not to discharge by a processor according to one embodiment.
Figure 7 illustrates cross-validation performed in training a machine learning model according to one embodiment.
Figure 8 shows a ROC curve for comparing the performance of multiple machine learning models according to one embodiment.
Figure 9 illustrates selecting features to be applied to a machine learning model based on feature importance according to one embodiment.
Figure 10 illustrates the performance of machine learning models on input data including selected features according to one embodiment.
Figure 11 illustrates a waterfall chart representing feature influence according to one embodiment.
Figure 12 shows predicted likelihood scores at multiple time points during a patient's hospitalization period according to one embodiment.
Figure 13 shows the simulated impact on bed management with the prediction model and individual descriptors applied according to one embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be implemented in various forms. Therefore, the actual implemented form is not limited to the specific embodiments disclosed, and the scope of the present disclosure includes modifications, equivalents, or alternatives included in the technical idea described in the embodiments.

Although the terms first or second may be used to describe various components, such terms should be construed only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

When it is said that a component is "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, the terms "comprises" or "have" should be understood to specify the presence of a described feature, number, step, operation, component, part, or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. When describing with reference to the attached drawings, identical components are given the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

Figure 1 illustrates the operation of a device for predicting patient discharge according to one embodiment. The device (100) for predicting patient discharge may include a processor (110) and a display (120).

The processor (110) can obtain a likelihood score (also expressed as discharge probability or likelihood in this specification) of a patient's discharge using a machine learning model. The likelihood score can represent the likelihood that the patient will be discharged within a target period (e.g., 3 days) from the predicted time point. The processor (110) can predict whether the patient will be discharged within the target period from the predicted time point based on the obtained likelihood score. The operation of the processor that obtains the likelihood score and predicts whether to be discharged using a machine learning model is described in detail in FIG. 6 below.

The display (120) can display whether the patient is expected to be discharged. The display (120) can display predicted probability scores at multiple points in time through a graph (121). In addition, the display (120) can display feature influence, which indicates the degree to which each feature influences the probability score, as a waterfall chart (122). The operation of the display (120) is described in detail in FIGS. 11 to 13 below.

In this specification, a feature (e.g., a medical feature) may represent an individual item and/or category that classifies information related to a patient's medical condition. A medical feature may include one or a combination of two or more of a surgical feature, a treatment feature, a medical image transmission system feature, a diagnosis feature, a medication feature, a test feature, a body feature, and an intensive care unit length of stay feature. Each of the aforementioned medical features may include a plurality of sub-features. Individual medical features are described below in FIG. 3.

FIG. 2 illustrates the overall flow of a method for predicting patient discharge according to one embodiment. In data processing, cohort criteria can be set, and an appropriate data set can be generated by processing data. In AI Model Evaluation, an elaborate model can be found by training and evaluating a machine learning-based prediction model (e.g., a machine learning model). In prediction and explanation, the probability of discharge within a target period (e.g., 3 days) can be predicted, and the results of the model can be explained by identifying, quantifying, and visualizing features.

Figure 3 illustrates medical data according to one embodiment.

Medical data (310) according to one embodiment is data having a format that can be input to a machine learning model, and can be acquired through preprocessing of raw medical data. For reference, the medical data (310) can be an input format of the machine learning model itself, and as described later in FIG. 9, data including some features among the features of the medical data can be an input format of the machine learning model.

Medical data (310) may include first partial medical data including past medical features (311) and second partial medical data including current medical features (312).

Past medical features (e.g., past features (311) shown in FIG. 3) may include features representing information corresponding to the patient's medical history prior to the patient's hospitalization. For example, past medical features may represent features obtained from raw medical data collected prior to the patient's hospitalization date.

The first partial medical data (e.g., historical partial medical data) collected during a predefined period prior to the hospitalization period may include medical information corresponding to the historical medical features described above. For example, the first partial medical data may include historical medical features of the patient's past (e.g., for three years prior to the date of hospitalization). The first partial medical data may include data corresponding to historical medical features of one or more combinations of diagnosis, medication, laboratory, physical information, and length of stay in an intensive care unit (LOS) (ICU), for example.

The present medical feature (e.g., the present feature (312) shown in FIG. 3) may include features representing information corresponding to a date after the hospitalization date. For example, the present medical feature may represent features obtained from raw medical data collected during the hospitalization period.

The second partial medical data collected during the hospitalization period (e.g., current partial medical data) is data including medical information after the date of the patient's hospitalization, and may include medical information corresponding to the current medical feature. The second partial medical data may include data corresponding to the current medical features of one or more combinations of, for example, operation, procedure, PACS, diagnosis, medication, laboratory, and physical information.

The second part medical data may be updated at regular intervals (e.g., daily) until the patient is discharged. For example, the device for predicting the discharge of the patient may collect additional medical information at predetermined intervals. The additional medical information may include information additionally collected after the patient's hospitalization, for example, a diagnosis that the patient receives during the hospitalization. The device for predicting the discharge of the patient may update the current medical features of the second part medical data in the medical data based on the additional medical information described above.

The device for predicting patient discharge can generate the aforementioned medical data by combining the aforementioned current partial medical data and past partial medical data.

For reference, the length of stay is the continuous length of stay for the currently hospitalized patient, and for patients who have been hospitalized multiple times, it can represent the period from the most recent date of admission. For example, if the patient is admitted on Day 1, discharged on Day 2, and readmitted on Day 3, the length of stay between Day 1 and Day 2 and the length of stay after Day 3 can be two non-consecutive (e.g., separate) lengths of stay. If it is predicted whether the patient will be discharged at a point in time after Day 3, the length of stay for that patient can represent the period after Day 3, which is the most recent date of admission. The period between Day 1 and Day 2 can be included in the period prior to the length of stay, not the length of stay.

Figure 4 illustrates a preprocessing process for obtaining medical data from raw medical data according to an embodiment.

Raw medical data refers to data containing medical information about patients collected by one or more entities, and may represent data before any preprocessing is applied to the collected medical information.

For example, raw medical data can be extracted from CardioNet, Inc., a manually curated electronic health record (EHR) database specializing in CVD. CardioNet may consist of, for example, 572,811 patients who visited Asan Medical Center in Seoul, Korea for CVD between January 1, 2000 and December 31, 2016. Collection of CardioNet may be approved by the AMC institutional review board if informed consent is waived. There may be 27 tables such as visitation, demographic, diagnosis, medication, and laboratory. Most of the tables in CardioNet can have common variables such as patient's identification (PAID), patient's encounter number (INNO), visit or admission date (INDT), and discharge date (OUDT). The KEY column, which is a concatenation of PAID and INNO, can connect the visit table with other tables. Through KEY, variables from each table to be analyzed can be extracted.

Of the 572,811 patients in CardioNet, 84,251 records of 63,261 anonymous patients admitted to Cardiology or Thoracic Surgery can be obtained. Furthermore, in order to develop a practical and usable model, it can be focused on predicting discharge within a target period (e.g., 3 days) and detecting long-term patients. Long-term patients of more than 30 days can be managed separately by Asan Medical Center (AMC). Therefore, the length of stay can be set between 3 and 30 days.

Data extracted from CardioNet may contain the following variables across multiple tables:

- Visit table: PAID, INNO, KEY, INDT, OUDT, visit type, department, Intensive Care Unit (ICU) length of stay (LOS)

- Diagnosis table: International Classification of Diseases (ICD)-10th diagnosis code

- Laboratory table: Pathology examination date and code, examination results

- Physical information table: patient's age, height, weight, systolic and diastolic blood pressure, respiratory rate, pulse rate, body mass index, body surface area, and measurement date.

- Medication table: Prescription date and code

- Procedure table: Date and code of order

- Operation table: Date and code of surgery or treatment

- Picture Archiving and Communication System (PACS) table: Order date and code

- Transfusion table: Order date and code

For reference, the ICU list is as follows: ACU (Acute Care Unit), CCU (Coronary Care Unit), CSICU (Cardiac Surgery ICU), MICU (Medical ICU), NICU (Neonatal ICU), NRICU (Neurological ICU), NSICU (Neurosurgical ICU), PICU (Pediatric ICU), and SICU (Surgical ICU).

The visit table and other tables of the raw medical data according to one embodiment may contain only one piece of information per row, and it may be difficult for the ML model to learn all the data at once. Therefore, the device can obtain features of a new data set (e.g., medical data) by performing preprocessing including one-hot encoding (OHE) of clinically important orders and codes. Through the preprocessing, the device can access aggregated records by date for each patient.

For reference, as described above in FIG. 3, the tables for Diagnosis, Medication, Examination, and Body can be used for both past features and present features. For example, in the case of the Diagnosis table, the information included in the Diagnosis table after the date of admission (e.g., the period of admission) and the information of a predefined period prior to the period of admission (e.g., 3 years prior to the date of admission) can be used to generate present features (or present partial medical data) and past features (or past partial medical data), respectively. The tables for Surgery, Procedure, and PACS can be used for present features. The LOS of the ICU can be used for past features.

In step (410), the device for predicting patient discharge can select codes having a high frequency from the raw data (select top frequent codes). For example, if there are too many values that the code variables of the prescription included in the medication table can have, training and/or inference of a machine learning model that distinguishes all values of the code variables of the prescription can result in inefficiency. The device for predicting patient discharge can select codes having a high frequency and change all remaining codes into one code (for example, a code indicating "other") in order to limit the number of values that the codes can have to a predetermined number or less.

For example, for the diagnosis and surgery table, all values of ICD-10 codes and surgery codes can be sliced at the third digit to be converted into 3-digit codes. This may be because the string after the fourth digit may represent a lower level of the 3-digit code. All frequency counts of the values are sorted in descending order and the first 99 codes can be selected. The remaining codes (e.g., unselected codes) can be transformed into "other" features.

At step (420), the device can perform one-hot encoding (OHE). For example, for the diagnosis and surgery table, OHE can be performed on all 100 codes. Features in the form of "Z_code" such as "Z_DICD", "Z_OPCD" can refer to "Others" of each original table.

At step (430), the device can fill values. For example, in the case of the diagnosis and surgery tables, a total of 100 codes can be obtained for each table and the date index value can be filled with 1 if there is valid prescribed or ordered data, otherwise 0.

At step (440), the device can perform imputation for missing values.

For example, in tables that exclude features related to examination, body information, and date, null values can be replaced with 0. The value type of most other features can be computed as a frequency, so they can be null or integer.

For example, to handle missing values of continuous data types of features in the Examination and Body tables, the data set can first be disjoint based on KEY so that individual admissions are not mixed. KEY can refer to one admission of a patient. Null values can be filled in chronologically (e.g., from past to present). Afterwards, the remaining null values can be filled in chronologically (e.g., from present to past) to handle cases where outcomes were not measured at the beginning of the admission. Null values can be imputed for each admission of an individual patient. Finally, to fill in values where all features are not aligned or measured, the remaining null values can be filled with the most frequent value for each feature.

While the above has been described primarily as an example for the diagnosis and surgery tables, similar preprocessing can be applied to other tables. For example, similar to the diagnosis and surgery tables, the values in the PACS table can be transformed into 100 features. For another example, similar to the diagnosis table, in the medication and treatment table, the 99 most frequent codes and "Other" can be obtained by performing OHE, and the corresponding data can be filled in. For another example, in the examination table, the 60 most frequent examination codes that were examined by more than 50% of the total patients can be selected. OHE of the values can be performed and the values can be filled in with the results corresponding to each examination. If the patient was examined multiple times in a day, the data set can be filled in with the average of the results.

However, preprocessing is not limited to what was described above, and some steps may be omitted or added for each table.

The device for predicting discharge of a patient according to one embodiment may omit the step (410) of selecting frequent codes during the preprocessing process for the table. For example, in the case of the blood transfusion table, all 27 available codes may be used. The value may be filled with the number of daily or one-time prescriptions considering the severity of each patient's disease. For another example, the body table may have 10 codes, and all codes may be used.

A device for predicting discharge of a patient according to another embodiment may perform additional steps in addition to steps (410 to 440) in the preprocessing process.

Although omitted in FIG. 4, a device for predicting patient discharge can merge and link multiple tables. In a primary table (e.g., a visit table) of raw medical data (e.g., CardioNet) according to one embodiment, there may be multiple main columns (e.g., PAID, INNO, INDT, OUDT) and variables related to visits. Each row may represent a single hospitalization case for each patient. In order to create a new data set format with a date index for the period between admission and discharge, the index may be reset. For example, a row with INDT of 2021.02.01 and OUDT of 2021.02.10 may have a LOS of 10 days. One row of the visit table may be converted into 10 rows with 10 date indices. After preprocessing all values corresponding to PAID, INNO, and date indices of other tables, the tables may be merged and linked to create a new data set for model training.

In addition, the device for predicting patient discharge can remove features or generate additional features. For example, the device for predicting patient discharge can remove OUDT containing future information after creating a new data set. For another example, in order to distinguish and recognize time information of a date according to type, a total of 10 date-related features can be generated. INDT and date index can be split into integer features such as year, month, day, and day of the week. A feature indicating whether the date index is a public holiday and another feature indicating LOS from the date index can be generated by subtracting INDT from the date index.

The above mainly describes the preprocessing process for obtaining current features from medical data as an example, but the preprocessing process for obtaining past features from raw medical data can be performed similarly. In order for the ML model to learn data deeply, the features of the medical data can include past medical features about the patient's medical history (e.g., medical history) along with day-by-day features (e.g., current medical features). When the date index of each hospitalization starts from the date of admission (INDT), some past medical features can be obtained from key information of the hospital visit records 3 years prior to the date of admission.

For example, a device for predicting patient discharge can obtain past medical features from raw medical data collected before the period of hospitalization. Similarly to the current medical features, OHE can be performed on the past medical features and the values can be populated. The past medical features of the medical data can be populated with a sum value or a recent value corresponding to each feature. For example, the length of stay in each intensive care unit in the visit table can be summed. If there is a past diagnosis record for 100 diagnosis codes, each value can be summed. If there is a record for 100 medication codes, the number of prescriptions per day or at a time can be summed. For another example, physical information and recent test results within 3 years can be used for a total of 70 codes.

Figure 5 illustrates labeling of medical data according to one embodiment.

Supervised learning algorithms for classification may require labels such as True or False to indicate the correct answer. The target criteria for labeling as True are shown in Figure 5.

In Fig. 5, Day 1 can be the admission date (INDT), Day N can be the discharge date (OUDT), and one circle can represent each day during the hospital stay. Day N (e.g., the discharge date) can be excluded from the dataset because of information such as discharge procedure that can provide hints to the ML model. Although the accuracy of discharge prediction can be high up to two days before the discharge date, it can be useful to predict in advance the target period (e.g., 3 days) when using the real model. Therefore, the dates from 1 day before the discharge date (OUDT) to 3 days before the discharge date (OUDT) can be labeled as 1 (e.g., true or positive), and the dates from the admission date (INDT) to 4 days before the discharge date (OUDT) can be labeled as 0 (e.g., false or negative).

In the medical data according to an embodiment, various variables of the original table (e.g., raw medical data) can be transformed into 10 date-related features, 597 current features, and 279 past features. Medical data of 669,667 rows with 886 features can be generated from 84,251 records for 63,261 hospitalized patients with CVD. Medical data consisting of 669,667 records with 886 features including diagnosis codes, laboratory test results, physical information, medications, procedures, surgeries, PACS, and blood transfusions can be generated. The patients may have been admitted to the department of cardiology or thoracic surgery, and the LOS of the patients may be between 3 and 30 days. The mean age of the patients may be 61.03 years, and the standard deviation may be 13.42 years. Medical data may consist of 38% females (e.g., 254,254 rows) and 62% males (e.g., 415,413 rows).

FIG. 6 illustrates obtaining a probability score and predicting whether or not to discharge by a processor according to one embodiment.

In step (610), the processor can obtain medical data by preprocessing raw medical data.

As described above in FIG. 3, the medical data may include partial medical data (e.g., second partial medical data or current partial medical data) collected during the hospitalization period (a1) and partial medical data (e.g., first partial medical data or past partial medical data) collected during a predefined period (a2) prior to the hospitalization period, based on the collected period.

At step (620), the processor can obtain a first likelihood score by applying a machine learning model to the input data.

The input data may include at least a portion of the medical data. The medical data may include a plurality of features. The processor may select one or more of the features of the medical data as features of the input data. The selection of features of the input data is described in detail in FIG. 9 below.

The likelihood score may represent the likelihood that the patient will be discharged within a predefined target period from the predicted time point. For example, the first likelihood score may represent the likelihood that the patient will be discharged within a target period (p1) from the predicted time point (d1).

For reference, predicting discharge for a patient by the processor may be performed repeatedly according to a predefined cycle. The cycle of discharge prediction according to one embodiment may be the same as the cycle of updating medical data described below. In this specification, the cycle of discharge prediction and the cycle of updating medical data are mainly described as being one day, but are not limited thereto. The cycle of discharge prediction according to another embodiment may be longer than the cycle of updating medical data or may be a multiple of the cycle of updating medical data.

In step (630), the processor can predict whether the patient will be discharged based on the patient's first likelihood score. The patient's discharge predicted based on the first likelihood score can specifically indicate whether the patient will be discharged within a target period from the first time point. For example, the processor can predict whether the patient will be discharged by comparing the first likelihood score with a threshold score.

At step (640), the processor may update the medical data collected during the hospitalization period in response to the patient being hospitalized at the second time point. The second time point may represent a time point at which one period of updating the medical data has elapsed from the first time point. However, the present invention is not limited thereto, and the second time point may represent a time point at which one or more periods have elapsed from the first time point. Due to additional medical information generated between the first time point (d1) and the second time point (d2), a portion of the medical data (e.g., current partial medical data or current features of the medical data) may be changed (e.g., updated).

In step (650), the processor can obtain a second likelihood score by applying a machine learning model to the updated input data. The second likelihood score can represent the likelihood that the patient will be discharged from the hospital within the target period from the second time point. In order to output the second likelihood score from the machine learning model, the input data can include at least a portion of medical data acquired from the date of admission to the second time point during the hospitalization period. The second likelihood score at the second time point can be output based on the input data including the medical data collected up to the second time point during the hospitalization period.

In step (660), the processor can predict whether the patient will be discharged based on the patient's second likelihood score. The patient's discharge predicted based on the second likelihood score can specifically indicate whether the patient will be discharged within a target period (p2) from the second time point (d2).

Figure 7 illustrates cross-validation performed in training a machine learning model according to one embodiment.

The training data can be labeled as either discharge or admission. Discharge can be labeled as positive (e.g., 1) and admission can be labeled as negative (e.g., 0). To evaluate and compare the performance of the models, metrics including accuracy, sensitivity (or recall for positive), specificity, precision, positive predictive value (PPV), negative predictive value (NPV), false positive rate (FPR), and true positive rate (TPR) can be used. When monitoring the model training and validation, F1-Score can be used to reflect imbalanced subjects, receiver operating characteristic (ROC) curve can be used to find the optimal threshold, and area under ROC (AUROC) score can be used for comparison.

To avoid overfitting of ML-based models and reduce biased results, stratified 5-fold cross-validation can be performed as shown in Fig. 7. The 63,261 PAIDs can be randomly shuffled and split into 5 groups of about 12,000 people. This may be because we try not to split a single patient's record into training (e.g., dotted box in Fig. 7) and test sets (e.g., diagonally hatched box in Fig. 7). The first group can be the test set and the remaining groups can be the training set of fold 1. Folds 1 to 5 can be generated in a similar way to ensure equal splitting of imbalanced subjects (e.g., the true labels of the dataset consist of 62.4% for label 0 and 37.6% for label 1 in all folds). 25% of the training set can be split into a validation set to tune the hyperparameters. As a result, the data set in each fold can be split into about 133,000 rows for the test set and 535,000 rows for the training set (i.e., including the validation set). ML-based models can be trained and tested on all five folds.

The device for predicting discharge of a patient according to an embodiment can experiment with five machine learning models to find the most suitable model. For example, the logistic regression (LR) model can be set as a baseline for performance estimation. A support vector machine (SVM), a random forest (RF), a multi-layer perceptron (MLP), and Extreme Gradient Boosting (XGBoost) can be selected as machine learning models for comparison. The device for predicting discharge according to an embodiment can perform hyperparameter tuning for each model through a random search.

The device for predicting discharge according to an embodiment of the present invention may select XGB, which is one of the GBM (Gradient-Boosting Algorithm) models, as the final model. GBM may include an ensemble method that combines several weak classifiers (e.g., trees). The main idea of GBM may be to focus on and weight incorrectly predicted results. While XGB is being trained, one tree learns a data set and assigns weights to incorrectly predicted records with errors, and the next tree of the same model may repeat the process of learning the data set with assigned weights and assigning weights.

For reference, GBM is an explainable machine learning model that can quantify the contribution of features to prediction results such as feature importance. In particular, XGB can have the advantages of regularization and performance. XGB can perform parallel processing, can be regulated to prevent overfitting, can be widely used in structured data learning, and can have excellent prediction performance.

Feature importance can list the features and contribution scores of features that the model considers important in the process of training data by a tree-based algorithm model. XGB can be considered as the final model because it can access the internals of the model including the decision-making process as well as the high performance of XGB. By approaching the tree, the specific features and influences of the features that contributed to the daily discharge prediction of each patient can be explained.

Figure 8 shows a ROC curve for comparing the performance of multiple machine learning models according to one embodiment.

[Table 1] Evaluation by AUROC score of 5-fold cross validation for each model

5 Using cross-validation, five ML-based models can be tested, and the AUROC scores for each fold can be shown in Table 1. The highest AUROC score for each fold is shown in bold, and the "Support" column in Table 1 represents the number of each true value label. In Fig. 8, the ROC curve plot can be shown. The area under the curve can represent the AUROC, which has a value between 0 and 1. The closer the AUROC score is to 1, the higher the performance of the model can be. XGB can get the highest and relatively stable score in all folds.

[Table 2] Comparison of five ML-based models with metrics results

Table 2 compares the evaluation results of five ML-based models. All scores in Table 2 are the mean and standard deviation of the results in five folds, and the highest score for each metric can be bolded. In terms of Specificity, LR and SVM were the highest at 0.828, but XGB could be the highest in the remaining metrics. In particular, even when the labels of the dataset are imbalanced, XGB can score more than 0.7 in predicting label 1. Therefore, XGB can be selected as the final model for predicting the probability of discharge.

Figure 9 illustrates selecting features to be applied to a machine learning model based on feature importance according to one embodiment.

Feature importance can indicate the importance of a feature in input data for a machine learning model. Feature importance can be calculated based on the degree to which the prediction error increases compared to the original data when the value of the feature is replaced with a random value.

The graph (900) can represent relative feature importance sorted by gain score of XGB. The gain score can represent the average gain of all splits where the feature is used. All features used in the machine learning model according to one embodiment can be replaced by the names used in Asan Medical Center (AMC). Most of the features that affect the model, except for the date-related features, can be found in all tables. Features in the treatment table can be substantially related to clinically important situations. For example, a term marked (D) can be more likely to mean a more severe condition than others. The remaining features can also be associated with CVD or include primary examinations and prescriptions during hospitalization.

Feature importance can only indicate the importance of a feature to a machine learning model, and can be distinguished from feature influence, which will be described later in Fig. 11. As will be described later, feature influence is a value indicating the degree to which the value of a feature influences an output (e.g., likelihood score). Since feature importance can explain the model and may be difficult to explain each patient, it may not be sufficient to be used as an individual descriptor for prediction (e.g., likelihood score). Depending on the condition of the patient, different features may affect the daily discharge probability each time. An individual descriptor that provides features that affect the daily discharge probability during the hospitalization period for each patient can be proposed. The individual descriptors are described in detail in Figs. 11 to 13.

A processor according to one embodiment may select features of input data from among features of medical data based on feature importance.

In step (910), the processor can calculate feature importance for the temporary machine learning model. The temporary machine learning model can represent a machine learning model trained with data including all features of the preprocessed medical data. The feature importance can represent the importance of the corresponding feature for the machine learning model. For example, the feature importance can be calculated based on the degree to which the prediction error of the temporary machine learning model increases compared to the original data when the value of the corresponding feature in the data is replaced with a random value.

In step (920), the processor may select features of the input data of the machine learning model based on the calculated feature importance. For example, the processor may sort the calculated feature importance in descending order and select top features corresponding to a predefined number. For another example, the processor may select features having a feature importance greater than a predefined threshold feature importance.

Through feature selection of input data, the input format of the machine learning model can be determined as a data format that includes the selected features.

At step (930), the processor can train the machine learning model based on the selected features. The processor can extract the selected features from the medical data and obtain training data composed of the extracted features. Instead of being trained based on all features of the medical data, the processor can obtain the machine learning model by training based on the selected features.

In step (940), the processor can obtain a likelihood score by applying a machine learning model to the selected features. The machine learning model can be applied to input data consisting of the selected features. In one embodiment, the input data can be obtained by extracting the selected features from medical data including all features for the patient. In another embodiment, the input data can be obtained by collecting only the features selected as inputs of the machine learning model from the patient.

Figure 10 illustrates the performance of machine learning models of input data including selected features according to one embodiment. Too many features may have a negative impact on model performance. Therefore, selecting an appropriate number of features may be required.

In one embodiment, recursive feature elimination with cross-validation (RFECV) can be performed, and the goal of RFECV can be to remove features with low feature importance one at a time while identifying the optimal number of features by comparing model performances. RFECV can return ranks and names of all features. By applying RFECV to the final model XGB, about 150 features with rank 1 can be identified. 5-fold cross-validation can be performed using the same dataset with the same parameters to compare performance.

The embodiments to be compared in Fig. 10 may include an embodiment based on all 886 features (denoted as XGB 886), an embodiment based on 150 features selected by RFECV (denoted as XGB RFE 150), and an embodiment based on top 50 features of feature importance of the model trained with 150 features selected by RFECV (denoted as XGB RFE & FI 50). The numbers in parentheses of the legend in Fig. 10 may represent the AUROC scores for each embodiment.

[Table 3] Evaluation by AUROC score of 5-fold cross validation for feature selection

Table 3 together with Figure 10 can show that the performance difference between the model using all features, the model using 150 features, and the model using 50 features is only about 1 to 2.5% in terms of AUROC score. In one embodiment, even when applying feature reduction of 83.1% to 94.4%, the maximum performance difference can be shown to be only 2.5%. The number of features can be appropriately adjusted considering the situation of each hospital or the characteristics of the data.

The prediction model can classify data as 0 or 1 based on the threshold score. The optimal threshold score can be the score that can simultaneously maximize the sum of sensitivity and precision. In the ROC curve, TPR and FPR can be proportional to each other, but sensitivity and precision can have a trade-off. Reducing false negatives (FN) increases sensitivity, and reducing false positives (FP) increases precision. It may be required to appropriately adjust the threshold score at the decision point of hospital operation.

Note that the optimal threshold score can be adjusted depending on the hospital situation, but there may be ambiguity in decision-making due to the likelihood score near the threshold score. Additional techniques can be used to reduce this ambiguity in decision-making. For example, a technique using a weighted average can be used to make the results more conservative but reliable. Rather than directly using the likelihood score (e.g., probability) returned by the model, it may be more useful to weight the results before the prediction time so that the past results are at least partially reflected at the prediction time. It may be just as important to produce reliable results as it is to explain the model and its internal features.

Daily discharge probabilities and date-wise feature influences during hospitalization can be presented through individual descriptors for prediction.

Figure 11 illustrates a waterfall chart representing feature influence according to one embodiment.

Individual explainers can be presented to help interpret the prediction results of XGB using a waterfall chart. A waterfall chart is a type of bar chart, also known as a bridge or cascade chart, that can portray relative values that calculate the difference between adjacent values. It can represent the gradual direction of the final discharge probability and the degree of positive or negative influence.

An individual explainer can display feature influence for the obtained likelihood score. The feature influence can correspond to the score caused by each feature for the likelihood score. The feature influence can be expressed as a contribution that quantifies the degree to which the feature contributed to the likelihood score. By displaying the feature influence for the likelihood score, the display can identify features that had a significant influence on causing the likelihood score, and can also visually present the magnitude of the influence to the user.

In one embodiment, to estimate the values of individual descriptors, desired records can be predicted with trained XGB and contributions of all features can be obtained. For example, the contributions can represent feature influence obtained by aggregating the scores contributed by each feature to all trees. Subsequently, the logistic value of feature influence ( ) and the relative values required for the descriptors can be calculated.

The processor can select one or more features from among the plurality of features based on feature influence. The processor can sort the features in descending order of feature influence and select features that are sorted higher than a predefined number. The selected features can be displayed. For example, the number of features to be displayed can be selected to be 15, and the remaining 871 features can all be combined and displayed simultaneously as "Other" in the descriptor.

In Fig. 11, the x-axis of the plot is a score from 0 to 1, and the y-axis can represent the contribution and the value that affected the likelihood score at the prediction time point (e.g., time point 1) (e.g., the 1st likelihood score). The intercept of the general diagonal hatched box at the bottom of the y-axis can be a modified value that reflects the imbalance in the number of each true value label. The discharge probability, which is the gray box at the top of the y-axis, can represent the likelihood score. The width of each box corresponding to a feature can represent the absolute value of each score. The actual score can be displayed on the right side of the plot. The absolute value can decrease from bottom to top, and can also represent a decrease in the contribution to the discharge probability. Note that the box of “Others” can be relatively wide because it is the sum of the scores of about 800 features excluding the features below it. The dotted boxes can represent each score of a feature that positively contributed to the discharge probability. The dotted boxes indicate that the scores can be shifted to the right in the graph. Conversely, the diagonally hatched boxes indicate scores for negatively contributing features and the scores can be shifted to the left in the graph.

In summary, from bottom to top, there are features that contributed to the prediction on the y-axis, with the dotted box on the right representing positive and the diagonal hatched box on the left representing negative.

Graph (1110) may represent the feature influence for a likelihood score of 0.004 obtained on Date: 7, and graph (1120) may represent the feature influence for a likelihood score of 0.811 obtained on Date: 12. In graph (1110), (D)ARTERIAL MONITORING = 1.0 and (D)INFUSION PUMP = 3.0 may have a negative influence on the likelihood score. In contrast, in graph (1120), (D)INFUSION PUMP = 0 may have a positive influence on the discharge probability. Since arterial monitoring and infusion pump are mainly prescribed to critically ill patients, both may consist of mostly 0 in the data set. Displaying the values of the features along with the features may help the medical staff to intuitively interpret the plot.

Individual descriptors may or may not have features that appear in the feature importance plot (as illustrated in Figure 9). This may suggest that it is necessary to identify features that contribute to individual patients rather than just managing the feature importance for the overall model.

Figure 12 shows predicted likelihood scores at multiple time points during a patient's hospitalization period according to one embodiment.

The display can display multiple predicted likelihood scores for a hospitalized patient over time. The multiple likelihood scores are predicted likelihood scores at different points in time, and can be displayed over time to indicate the changing likelihood of the patient's discharge.

For example, as shown in Fig. 12, the sample data set can be a record of a patient with PAID of 228,443, INNO of 2, who was admitted for 13 days and discharged on the 14th day. A plot of the likelihood score of the patient can be shown in Fig. 12. The x-axis of the plot can represent the days of the patient's hospitalization excluding the discharge date (indicated as day 14), and the y-axis can represent the likelihood score (e.g., the probability of discharge). The optimal threshold score of the model can be 0.39, which is indicated by the horizontal dotted line. The circles and triangles can represent the true labels 1 and 0, respectively, and the sizes of the circles and triangles can be proportional to the likelihood scores. The patterns in the plot can represent the outcomes predicted by the model. The dots can represent positive predictions (e.g., label 1; predicted as discharge) and the diagonal hatching can represent negative predictions (e.g., label 0; predicted as admission).

For the sample in Fig. 12, the model can accurately predict discharge within 3 days. However, adjusting the threshold score may change the prediction results for days 11 and 12. For example, if the threshold score increases, label 1 may only apply to days 12 and 13. Increasing the threshold score can be useful when trying to reduce false positives (FPs) despite increasing false negatives (FNs).

Figure 13 shows the simulated impact on bed management with the prediction model and individual descriptors applied according to one embodiment. The discharge probability of all patients in each ward can be recognized daily, and the most important features and the values of the features that affect the discharge probability score can be identified at once. Since the individual descriptors imply inferences about not only discharge but also long-term discharge, they can be useful for interpreting both high and low discharge probabilities. Similarly, information can be obtained based on the expected discharge date of each patient in the near future, such as bed capacity. In order to efficiently utilize human and material resources of the hospital, future bed information can help reduce hospital costs by improving bed management and admission reservations.

Research on bed management that requires hospital processes and biomarker detection for patient-tailored treatment can be actively conducted. The present invention can propose an ML-based prediction model to identify discharge dates for better bed management and to identify risk factors related to discharge and CVD. However, since environmental variables differ for each hospital, an algorithm that can comprehensively consider these may be required. The present invention can contribute to improving the algorithm and supporting medical services. The expectations of the prediction model are described below.

The model according to the present invention can be extended from the ward level to the hospital level bed management. The present invention can contribute to reducing the labor-intensive work of medical staff and the waiting time of patients.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and software applications running on the OS. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone, but those skilled in the art will appreciate that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, a processing device may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

The software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and/or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal waves, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over network-connected computer systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium.

The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination, and the program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known to and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described with limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the described embodiments. For example, even if the described techniques are performed in a different order than the described method, and/or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or are replaced or substituted by other components or equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also included in the scope of the claims described below.

Claims (23)

In a device for predicting patient discharge,
A processor for obtaining, from raw medical data including medical information on a plurality of patients, medical data collected from a date of hospitalization to the first time point during the hospitalization period of a patient hospitalized at a first time point, applying a machine learning model to input data including the medical data to obtain a first likelihood score that the patient will be discharged from the hospital within a target period from the first time point, and predicting whether the patient will be discharged from the hospital within the target period from the first time point based on the obtained first likelihood score;
Including,
The above processor,
Obtaining the raw medical data including multiple tables,
For the above multiple tables, in response to a case where the number of values that each feature can have exceeds a predetermined number, values having a frequency greater than a threshold frequency are maintained and values having a frequency less than the threshold frequency are replaced with a common value.
Merge the plurality of tables based on at least one feature among the patient ID, patient encounter number, or admission date included in each of the plurality of tables, and
From the result of merging the above multiple tables, the medical data aggregated by date for the patient is obtained.
A device for predicting patient discharge.
In the first paragraph,
The above processor,
By applying the machine learning model to input data including medical data regarding one or more combinations of operation, procedure, Picture Archiving and Communication System (PACS), diagnosis, medication, laboratory, and physical collected from the date of admission to the first time point for the patient, the first likelihood score is obtained.
A device for predicting patient discharge.
In the first paragraph,
The above processor,
In response to the patient being hospitalized at a second point in time after the first point in time, the medical data collected during the hospitalization period is updated based on the medical data collected from the first point in time to the second point in time,
By applying the machine learning model to the input data including the updated medical data, a second likelihood score of the patient being discharged from the hospital within the target period from the second time point is obtained,
Based on the second probability score obtained above, predicting whether the patient will be discharged from the hospital within the target period from the second time point.
A device for predicting patient discharge.
In the first paragraph,
The above processor,
Obtaining the first likelihood score by applying the machine learning model to input data including medical data collected during the said hospitalization period together with medical data collected during a predefined period prior to the said hospitalization period.
A device for predicting patient discharge.
In the first paragraph,
The above processor,
By applying the machine learning model to input data including medical data collected during the said hospitalization period and medical data regarding one or more combinations of diagnosis, medication, laboratory, physical, and length of stay (LOS) in an intensive care unit (ICU) collected for the patient during a predefined period prior to the said hospitalization period, the first likelihood score is obtained.
A device for predicting patient discharge.
In the first paragraph,
The above processor,
Based on the feature importance of each feature for a temporary machine learning model trained on the basis of all features of the collected data, one or more features among the features of the collected data are selected as features of the input data of the machine learning model,
The above machine learning model is trained based on the above selected features,
Obtaining the first likelihood score by applying the machine learning model to the input data including the selected features.
A device for predicting patient discharge.
In the first paragraph,
The above processor,
Selecting one or more features as inputs to the machine learning model by applying the recursive feature elimination with cross validation (RFECV) technique to the features of the input data.
A device for predicting patient discharge.
In the first paragraph,
The above processor,
By applying the XGB (extreme gradient boost) model to the above input data, the first likelihood score is obtained.
A device for predicting patient discharge.
In the first paragraph,
The above processor,
Selecting one or more features among the features of the input data based on the feature influence corresponding to the score induced by each feature of the input data for the first possibility score obtained above.
A device for predicting patient discharge.
In Article 9,
A display showing the feature influence of one or more of the selected features for the first probability score obtained above.
A device for predicting discharge of a patient including:
In the first paragraph,
Display showing probability scores for multiple time points during the above hospitalization period
A device for predicting discharge of a patient including:
A method for predicting patient discharge performed by a device for predicting patient discharge,
A step of obtaining medical data collected from the date of hospitalization to the first time point during the hospitalization period of a patient hospitalized at a first time point, from raw medical data including medical information on multiple patients;
A step of obtaining a first likelihood score of the patient being discharged from the hospital within a target period from the first time point by applying a machine learning model to input data including the above medical data; and
A step of predicting whether the patient will be discharged from the hospital within the target period from the first time point based on the first probability score obtained above.
Including,
The step of obtaining the medical data from the above raw medical data is:
A step of obtaining the raw medical data including a plurality of tables;
For the above multiple tables, in response to a case where the number of values that each feature can have exceeds a predetermined number, a step of maintaining values having a frequency greater than a threshold frequency and replacing values having a frequency less than the threshold frequency with a common value; and
A step of merging the plurality of tables based on at least one feature among a patient ID, a patient encounter number, or a hospitalization date included in each of the plurality of tables;
A step of obtaining the medical data aggregated by date for the patient from the result of merging the plurality of tables,
A method for predicting patient discharge.
In Article 12,
The step of obtaining the above first possibility score is:
A step of obtaining the first likelihood score by applying the machine learning model to input data including medical data regarding one or more combinations of operation, procedure, Picture Archiving and Communication System (PACS), diagnosis, medication, laboratory, and physical collected from the date of admission to the first time point for the patient,
A method for predicting patient discharge.
In Article 12,
In response to a case where the patient is hospitalized at a second point in time after the first point in time, a step of updating medical data collected during the hospitalization period based on medical data collected from the first point in time to the second point in time;
A step of obtaining a second likelihood score of the patient being discharged from the hospital within the target period from the second time point by applying the machine learning model to input data including the updated medical data; and
A step of predicting whether the patient will be discharged from the hospital within the target period from the second time point based on the second probability score obtained above.
A method for predicting discharge of a patient including:
In Article 12,
The step of obtaining the above first possibility score is:
A step of obtaining the first likelihood score by applying the machine learning model to input data including medical data collected during the said hospitalization period together with medical data collected during a predefined period prior to the said hospitalization period,
A method for predicting patient discharge.
In Article 12,
The step of obtaining the above first possibility score is:
A step of obtaining the first likelihood score by applying the machine learning model to input data including medical data collected during the period of hospitalization and one or more combinations of diagnosis, medication, laboratory, physical, and length of stay (LOS) in an intensive care unit (ICU) collected for the patient during a predefined period prior to the period of hospitalization, comprising:
A method for predicting patient discharge.
In Article 12,
A step of selecting one or more features of the collected data as features of the input data of the machine learning model based on the feature importance of each feature for a temporary machine learning model trained based on all features of the collected data; and
A step of training the machine learning model based on the selected features.
Including more,
The step of obtaining the above first possibility score is:
Comprising a step of obtaining the first likelihood score by applying the machine learning model to the input data including the selected features.
A method for predicting patient discharge.
In Article 17,
The step of selecting one or more of the above features as features of the input of the machine learning model comprises:
A method comprising: selecting one or more features as inputs to the machine learning model by applying a recursive feature elimination and cross validation (RFECV) technique to the features of the input data.
A method for predicting patient discharge.
In Article 12,
The step of obtaining the above first possibility score is:
A step of obtaining the first likelihood score by applying an extreme gradient boost (XGboost) model to the input data,
A method for predicting patient discharge.
In Article 12,
A step of selecting one or more features among the features of the input data based on the feature influence corresponding to the score induced by each feature of the input data for the first possibility score obtained above.
A method for predicting discharge of a patient including:
In Article 20,
A step of displaying the feature influence of one or more of the selected features for the first possibility score obtained above.
A method for predicting discharge of a patient including:
In Article 15,
Step of displaying probability scores for multiple time points during the above hospitalization period
A method for predicting discharge of a patient including:
A computer program stored on a computer-readable recording medium for executing the method of any one of claims 12 to 22 in combination with hardware.

Images