JP2021124989A - Medical agent identification system and medical agent identification sensor - Google Patents

Abstract

To allow a medical agent identification system to identify medical agents from images taken under various different conditions.SOLUTION: A medical agent identification system 10 includes: a learning data storage unit 60 for storing, as learning data 62, how various types of medical agents 12 look in taken images 22; and a medical agent identification unit 30 for identifying the medical agents 12 in the taken images 22. The medical agent identification unit 30 identifies medical agents 12 in the images 22 by using an AI model 50. The AI model 50 is created on the basis of learning data 62 stored in the learning data storage unit 60 and returns the taken images 22 as input data and returns information of the medical agents 12 corresponding to the taken images as output data.SELECTED DRAWING: Figure 1

Description

The present invention relates to a drug identification system that identifies a photographed drug by performing image recognition of a photographed image of the drug. The present invention also relates to a drug identification sensor that identifies a drug contained in a photographed image by performing image recognition processing on the photographed image.

In facilities that handle drugs, such as pharmacies, mechanical identification of drugs may be required. For example, mechanical identification of a drug is performed in order to determine whether or not the drug prepared based on the prescription actually matches the prescription content of the prescription by a subject other than the worker (pharmacist, etc.). A device to perform (pharmaceutical inspection device) may be used. By using such a device, it is possible to prevent work mistakes in delivering the drug to the patient.

Patent Document 1 discloses a weighing device applied to a drug inspection device. In this weighing device, in addition to a weighing unit that measures the weight of the drug to be measured, an imaging means (camera) for capturing an image of the measurement target is provided, and based on the captured image data, the weighing device is provided. Whether or not the drug to be measured matches the type of drug specified in the prescription is determined.

Japanese Patent No. 5744776

However, in the weighing device disclosed in Patent Document 1, an appropriate determination cannot be made unless the drug to be measured is imaged under appropriate imaging conditions. Specifically, it is necessary for the imaging means to position the measurement target in an imaging target region where the measurement target can be appropriately imaged from above. Therefore, a positioning means for positioning the container holding the measurement target at an appropriate position for measurement is provided. Further, in order to prevent the measurement target from being shaded by the disturbance light, it is also necessary to shield the measurement target from the disturbance light by housing the container for holding the measurement target and the imaging means together in the housing.

As described above, the weighing device disclosed in Patent Document 1 is provided with a mechanism for placing the drug to be measured under appropriate imaging conditions (position, lighting conditions, etc.), and these must be used. It is not possible to make a proper judgment. This is because the weighing device of Patent Document 1 discriminates the drug type based on the image data of the imaged drug and the image data of the drug as the master data, and the master data and the imaging conditions are different. This is because the image data obtained by imaging cannot be properly discriminated.

Therefore, the weighing device disclosed in Patent Document 1 requires work (positioning, storage in a housing, etc.) for placing the drug under appropriate imaging conditions when capturing an image of the drug. In order to use this weighing device, it is necessary to learn a dedicated operation method, and the operation itself is very time-consuming. Therefore, it was not possible to easily identify the drug mechanically. In addition, this weighing device has become a large-scale device equipped with a dedicated mechanism for placing the drug under appropriate imaging conditions, and cannot be a portable device.

Therefore, an object of the present invention is to make it possible to identify a drug from images taken by photographing under various conditions without placing the drug under specific appropriate imaging conditions.

In order to solve the above problems, a drug identification system as an example of an embodiment according to the present invention is used in a drug identification system that identifies a photographed drug by performing image recognition of a photographed image of the drug. The drug identification unit includes a learning data storage unit that stores as learning data how each of the images appears in the captured image, and a drug identification unit that identifies the drug contained in the photographed image. The drug identification unit is the learning data. An AI model created based on the learning data stored in the storage unit, and the captured image input using the AI model that returns the captured image as input data and the information of the corresponding drug as output data. It is characterized by identifying the drug contained in.

Further, preferably, when the drug or the drug package for packaging the drug contains an identification index indicating the drug identification information, the drug identification unit identifies the drug based on the drug identification information indicated by the identification index. To create drug imaging example data in which the identified drug information and the captured image information at that time are combined, and transmit the drug imaging example data to the learning data storage unit, the learning data storage unit When the drug imaging example data is received from the drug identification unit, the drug information and the captured image information included in the drug imaging example data are stored as an example of the appearance of the drug in the captured image, and the learning data is stored. Should be updated.

Further, preferably, when it is determined by the AI model that the captured image contains the identification index, the drug identification unit is based on the identification information of the drug corresponding to the identification index determined by the AI model. It is advisable to identify the drug and create drug imaging example data by combining the information of the identified drug and the information of the photographed image at that time.

It is also preferable that each of the drug identification units has a plurality of local devices that can access the main device via the network, and the drug imaging example data in the drug identification performed by the local device. Is created, the drug imaging example data is transmitted to the main device, and the main device updates the learning data stored by itself based on the drug imaging example data transmitted from the local device. It is good to do it.

Further, preferably, the drug identification unit of the local device identifies the drug contained in the captured image by using the local AI model incorporated in the local device, and the drug contained in the captured image even by using the local AI model. If it is not possible to identify the drug, it is preferable to access the main device via the network and identify the drug using the main AI model created based on the learning data stored in the main device.

Further, preferably, when the photographed drug is a ethical drug, the optical mark marked on the drug packaging of the ethical drug and indicating information on the ethical drug may be used as the identification index.

Further, preferably, when the photographed drug is a drug dispensed based on the prescription information input at the discretion of the doctor, the information of one or more drugs identified by the drug identification unit is the prescription information. It is advisable to determine whether or not they match.

Further, preferably, when the photographed drug is a drug managed based on inventory information that fluctuates according to information on the drug received and discharged at the pharmacy, one or more drugs identified by the drug identification unit. It is advisable to determine whether or not the drug information matches the inventory information.

Further, preferably, when the photographed drug is a drug dispensed to the patient and should be taken based on the medication schedule instructed by the medical institution, one identified by the drug identification unit. It is advisable to determine whether or not the above drug information matches the medication schedule.

Further, it is preferable that the learning data storage unit and the drug identification unit are configured by separate devices, and the drug identification unit can access the learning data storage unit via a network.

Further, the drug identification sensor as another example of the embodiment according to the present invention is a processing unit including a storage element and an arithmetic element incorporating an AI model created based on the learning data obtained by the above-mentioned drug identification system. And an image capturing element capable of capturing an image, and by performing image recognition processing on the captured image captured by the image capturing element in the processing unit, a drug contained in the captured image. It is characterized in that the identification is performed.

According to the drug identification system as an example of the embodiment according to the present invention, information on how various drugs appear in the photographed images, that is, various photographed images and those photographed images are reflected. The correspondence between the drug information (type and quantity) and the correspondence is stored (stored) in the learning data storage unit as learning data. Then, using the AI model created based on the learning data, the drug contained in the input captured image is identified. Here, if the AI model is created from sufficiently accumulated training data (captured a large amount of captured images), the images captured under various shooting conditions (lighting conditions, position, distance, angle, etc.) can be used. , The drug information (type and quantity) shown in the image can be properly identified. Therefore, the drug does not have to be placed under certain proper imaging conditions. Therefore, the operator who intends to identify the drug does not need to perform the work for placing the drug under appropriate imaging conditions, and can easily identify the drug.

When an identification index is written on the drug or drug package, the drug identification unit identifies the drug based on the identification information indicated by the identification index, and the information on the identified drug and the information of the photographed image at that time are obtained. If the learning data is updated based on the drug shooting example data in combination with, even if the shot image is taken by a person who does not have specialized knowledge about the drug, what kind of shot image and the shot image will be It is information that can properly update (accumulate) the learning data, which is linked with the correct information about whether or not the drug is contained. For this reason, information about what is the correct answer, even for those who do not have specialized knowledge (for example, a patient who has received a drug dispensed at a pharmacy and is neither an AI engineer nor a medical personnel). It can contribute to the accumulation of learning data including, so-called teacher data. Therefore, in this drug identification system, it is possible to capture images taken by a person who does not have expertise in AI and drugs to promote the accumulation of learning data on drugs, so that the user of the system (shooting of drugs) can proceed. As the number of people who perform the training increases, the speed of accumulating learning data increases. In addition, the accuracy of the AI model created based on the learning data will improve as the number of system users increases.

In addition, when it is determined by the AI model that the captured image contains the identification index, the drug identification and the drug imaging example data are obtained based on the identification information of the drug corresponding to the identification index determined by the AI model. If the data is created, the drug imaging example data can be obtained only by photographing the drug, and the learning data can be updated more efficiently.

Further, when the drug identification system is a system having a plurality of local devices that can access the main device via a network, the local device functions as a device that can be carried by an individual, for example, a drug identification unit. By using a smartphone on which a predetermined application is installed, it becomes possible to collect drug imaging example data from a large number of local devices, and the storage speed of learning data is further improved.

In addition, when identifying the drug contained in the captured image using the local AI model incorporated in the local device, the local device does not need to access the main device if the drug can be identified by the local AI model. Therefore, the frequency of communication via the network can be minimized, the average speed of drug identification can be improved, and the amount of communication in the entire system can be suppressed.

In addition, when the photographed drug is a ethical drug, the optical mark on the ethical drug packaging, which indicates information about the ethical drug, is used as an identification index to form the ethical drug packaging. It is possible to identify drugs using an identification index that has a predetermined coding method for drug information, such as barcodes that are required to be displayed, and the accuracy of drug identification based on the identification index. Is improved.

In addition, when the photographed drug is a drug dispensed based on the prescription information input by the judgment of the doctor, whether or not the information of one or more drugs identified by the drug identification unit matches the prescription information. By determining whether or not, it is possible to prevent the delivery of a drug different from the contents of the prescription when the drug is delivered to the patient at a dispensing pharmacy or the like.

In addition, when the photographed drug is a drug managed based on inventory information that fluctuates according to information on the drug received and discharged at the pharmacy, the drug identification unit identifies one or more drugs. By determining whether or not the information matches the inventory information, it is possible to collate the inventory information with the information of the drug actually existing in the pharmacy at the time of inventory of the pharmacy.

In addition, when the photographed drug is a drug dispensed to the patient and should be taken based on the medication schedule instructed by the medical institution, one or more drugs identified by the drug identification unit. By determining whether the drug information matches the medication schedule, it is possible to confirm whether the patient is taking the drug correctly according to the medication schedule.

Further, if the learning data storage unit and the drug identification unit are configured by separate devices, and the drug identification unit can access the learning data storage unit via the network, the system configuration is free. The degree increases. For example, the learning data storage unit and the drug identification unit can be provided at a distance apart from each other, or the respective devices can be miniaturized.

Further, the drug identification sensor as another example of the embodiment according to the present invention can identify the drug without placing the drug under specific imaging conditions, and is large-scale in addition to the processing unit and the imaging element. Since it does not require a complicated mechanism, it can be a small portable device. Further, since this drug identification sensor discriminates a drug by using an AI model created based on learning data efficiently accumulated by the above-mentioned drug identification system, highly accurate drug identification becomes possible.

The block diagram which shows schematic structure of the drug identification system as an example of the Embodiment which concerns on this invention. The figure which shows the tablet alone which the identification index was described. FIG. 2 shows a PTP sheet on which an identification index is written, which wraps the tablet of FIG. 2A. The figure which shows the whole PTP sheet of FIG. 2B. The figure which shows the back side of the PTP sheet of FIG. 2C. The figure which shows the individual packaging of the powder which marked the identification index. The figure which shows the PTP sheet which described the identification index which wraps a capsule. The figure which shows an example of the identification index table. The figure which shows an example of the drug imaging example data. The flowchart which shows the procedure of drug identification and learning data update in the drug identification system of FIG. A block diagram schematically showing the configuration of a drug identification system in which a plurality of local devices access the main device via a network. The figure which shows typically the structure of the drug identification sensor as another example of the Embodiment which concerns on this invention.

The block diagram of FIG. 1 schematically shows the configuration of the drug identification system 10 as an example of the embodiment according to the present invention. As an example of the drug 12 to be identified in the drug identification system 10, FIG. 1 shows a tablet 12a alone and a tablet PTP sheet 12b (10 tablets per tablet).

The drug identification system 10 shown in FIG. 1 identifies a monitor 14 for displaying the identification result, a camera 20 for capturing an image of the drug 12, and a drug 12 contained in the captured image 22 captured by the camera 20. The drug identification unit 30, the identification index table storage unit 40 that stores the identification index table 42 (FIG. 3) described later, and the AI model 50 that returns the information of the drug 12 corresponding to the captured image 22 as input data (AI model 50). As an entity, it includes a storage unit that stores the data of the AI model 50) and a learning data storage unit 60 that stores the learning data 62 that is the basis for creating the AI model 50. These components may be integrated into one device, or may be divided into a plurality of devices and arranged. For example, the drug identification system 10 may be constructed by one smartphone, with the screen of the smartphone as the monitor 14, the camera incorporated in the smartphone as the imager 20, and the memory and the processor as other components. Alternatively, the drug identification system 10 is constructed on a network (such as the Internet), and each of a plurality of devices connected via the network functions as each component. The drug identification system 10 may be available from devices at various locations.

(Basic operation)
A user (worker) who intends to perform an identification operation using the drug identification system 10 first photographs a drug 12 (a region containing the drug 12) with a camera 20 and acquires a photographed image 22. Then, the captured image 22 is input to the drug identification unit 30. The drug identification unit 30 performs a predetermined process for inputting the captured image 22 into the AI model 50 (such as converting it into a data format accepted by the AI model 50), and then inputs the captured image 22 into the AI model 50. Hand over as.

The AI model 50 used in the drug identification system 10 is a model that describes an input / output relationship in which when some input data is received, output data corresponding to the input data is returned, and is like a kind of function. Have a job. Specifically, when the AI model 50 receives the captured image 22 as input data, the AI model 50 returns the information of the drug 12 (included) in the captured image 22 as output data. The AI model 50 can output various information related to the drug 12, such as a product name, a dosage form, an active ingredient, and an efficacy, as output data. Here, for the sake of explanation, it is assumed that the product name of the drug 12 and the quantity shown in the captured image 22 are output as output data.

The AI model 50 is created based on the learning data 62 stored in the learning data storage unit 60. The learning data 62 is a collection of a plurality of exemplary data, with the captured image 22 captured in the past and information on what the drug 12 captured in the captured image 22 was as a set of exemplary data. Is. Here, in the photographed image 22 taken in the past, the drug 12 is in various states (the position, distance, angle, packaging state, lighting condition at the time of shooting, resolution, etc. of the drug 12 are different for each photographed image 22). It is reflected. In other words, the learning data storage unit 60 stores as learning data 62 how the various agents 12 appear in the captured image 22.

The AI model 50 created based on such learning data 62 takes the photographed image 22 photographed under various imaging conditions as input data, and accurately outputs the information of the drug 12 contained in the photographed image 22. It is possible to do.

Then, the drug identification unit 30 receives the output data from the AI model 50, and the information (here, the product name and quantity) of the drug 12 shown in the photographed image 22 identified by the AI model 50 is used as the identification result on the monitor 14. Display on. At this time, if a plurality of types of agents 12 are shown in the captured image 22, the agent identification unit 30 can identify all of the plurality of types of agents 12. Further, for the same type of drug 12, it is possible to identify how much the drug 12 shown in the photographed image 22 is. Then, the monitor 14 can display information on the type (for example, product name) and quantity of each of the plurality of types of agents 12 shown in the captured image 22. For example, when the drug 12 to be identified is the tablet 12a alone and the tablet PTP sheet 12b shown in FIG. 1, the product name of the tablet 12a is "Example X 5 mg" and the product name of the tablet PTP sheet 12b is "Example A 1 mg". Then, information such as "<Drug 1: Product name / ExampleX 5 mg, Quantity / 1 tablet>-<Drug 2: Product name / ExampleA 1 mg, Quantity / 1 sheet>" is displayed on the monitor 14. The display of such an identification result may be displayed together with the captured image 22. In particular, the drug in the captured image 22 and the identification result display of the drug may be superimposed and displayed as a kind of AR image.

(About identification index)
The above is the basic operation of the drug identification system 10. In the drug identification system 10 of the present embodiment, it is possible to identify the drug 12 by using an "identification index" in addition to the identification by the AI model 50. It has become. The identification index is written on the drug 12 or the drug package that packages the drug 12 in the form of engraving, printing, or the like, and indicates what the drug 12 is, that is, the identification information of the drug 12. As an identification index, an optical mark such as a bar code or a two-dimensional code (which displays coded information by combining a plurality of parts having different optical properties such as a light refractive index and a reflection spectrum). In addition, a display in which the product name itself or an identification code (for example, a symbol string consisting of a company code of a pharmaceutical company and a product code defined by the company) is written as a character string / symbol string can be used. Although the information to be shown is described as it is (the information is not encoded), such a display can also be used as an identification index. Specifically, an identification code stamped on the drug 12 itself, a character string in which the product name of the drug 12 is printed as it is on the drug package (for example, "Example A 1 mg"), or a barcode printed on the drug package (for example, "Example A 1 mg"). For example, the GS1 data bar specified in Japanese Industrial Standards X0509, which is required to be displayed on medical drugs, can be used as an identification index.

2A-2F show specific examples of the identification index. In addition to these identification indicators, the company logo of the pharmaceutical company, the recycling type mark of the packaging, the medicinal effect, the usage, etc. may be displayed on the drug and the packaging of the drug, but these are omitted here for identification. Only indicators are shown. FIG. 2A shows one tablet 70a alone, and the tablet 70a is engraved with the identification code 71a of "XY123" on the surface thereof. FIG. 2B shows a PTP sheet piece 73a from which a part (two tablets) of the PTP sheet for packaging the tablet 70a of FIG. 2A is separated, and the PTP sheet piece 73a is at a position where the tablet 70a is arranged. In addition to the identification code 71a printed nearby, the product name 72a "ExampleA 1mg" is printed as it is as a character string.

FIG. 2C shows the entire PTP sheet 75a (for 10 tablets) before the PTP sheet piece 73a of FIG. 2B is separated. Similar to FIG. 2B, the identification code 71a and the product name 72a are printed near the tablet 70a, and the product name 72a is also printed on the end portion (upper end portion in the drawing) of the PTP sheet 75a.

FIG. 2D shows the back side of the PTP sheet 75a shown in FIG. 2C. On the back side, the product name 72a is printed for each area of two tablets separated by a separating line, and the barcode 76a is printed on the end (lower end in the figure) of the PTP sheet 75a. There is. The barcode 76a represents a number having a predetermined number of digits by a combination of a line and a blank. The number represented by the GS1 data bar, which is required to be written on ethical drugs, is usually 10 digits or more, but here, for the sake of explanation, the barcode 76a is assumed to represent a 6-digit number. ..

FIG. 2E shows an individual package 73b in which the powder 70b (powder) is packaged in a nylon SP package (Strip Package). The identification code 71b (“ZZ789”) of the powder 70b, the product name 72b (“ExampleB 10%”), and the barcode 76b are printed on the individual package 73b.

FIG. 2F shows a PTP sheet piece 73c from which a part (3 tablets) of the PTP sheet for packaging the capsule 70c is separated, and the PTP sheet piece 73c has an identification code 71c (“VW456”) of the capsule 70c. "), The product name 72c ("Example C 200 "), and the barcode 76c are printed.

The identification information indicated by the identification index as shown in FIGS. 2A-2F is associated with the drug information (drug information), respectively. It can be said that the product names 72a, 72b, and 72c are identification indexes and represent drug information by themselves. The correspondence between the identification information and the drug information is managed in the drug identification system 10, and in the present embodiment, the correspondence is stored in the identification index table storage unit 40 as the identification index table 42 as shown in FIG. There is.

FIG. 3 shows an example of the identification index table 42. Here, the correspondence between the identification information and the drug information is shown for the tablet 70a of FIGS. 2A-2D, the powder 70b of FIG. 2E, and the capsule 70c of FIG. 2F. For example, for the tablet 70a, the product name 72a "Example A 1 mg", which is the drug information, is associated with the number "123456" represented by the barcode 76a and the identification code 71a "XY123" as the identification information (in the table). It is stored in the same row, that is, as data belonging to the same record. Similarly, with respect to the powder 70b, the product name 72b “Example B 10%” is stored in association with the number “456789” represented by the barcode 76b and the identification code 71b “ZZ789”. Regarding the capsule 70c, the product name 72c "Example C 200" is stored in association with the number "024680" represented by the barcode 76c and the identification code 71c "VW456". In addition to these information, the identification index table 42 can include various information related to the drug, such as "dosage form" (shape of drug) and "packaging" (how to package).

The identification index table 42 naturally contains information on a large number of drugs other than the tablets 70a, the powder 70b, and the capsules 70c. Especially for ethical drugs, it is desirable to include information on all products that can be identified. Such an identification index table 42 is prepared independently of the AI model 50 of FIG. 1, and documents submitted by each pharmaceutical company to a public institution, publicly available data, or data provided by the pharmaceutical company. Is created based on.

As an example of a drug other than the tablet 70a, the powder 70b, and the capsule 70c, the identification index table 42 shown in FIG. 3 contains information on the drug having the product name “ExampleX 5 mg”. This drug "Example X 5mg" is a bottled tablet as the "packaging" is "100 tablets per bottle". A barcode indicating the number "975310" is attached to the bottle as an identification index of the drug "Example X 5 mg", but the barcode is not attached to the drug in units of one tablet. On the other hand, the character string of the identification code "NM321" is engraved on each tablet. The identification index table 42 can also include information about various forms of the drug, such as ointments in tubes and liquids in ampoules.

(Identification using identification index)
The drug identification system 10 (FIG. 1) of the present embodiment uses an identification index as shown in FIGS. 2A-2F and an identification index table 42 as shown in FIG. 3, and captures an image 22. It is also possible to identify the drug 12 contained in.

That is, when the identification index is written on the drug 12 or the drug package to be imaged, the drug identification system 10 does not use the output data of the AI model 50 as the identification result as it is, but the identification information indicated by the identification index. The identification result can be obtained by specifying the drug information from. For example, if the drug identification unit 30 has a detection function for detecting an identification index consisting of characters or barcodes from the captured image 22 by image analysis, apart from the AI model 50, the identification index obtained by the detection function is provided. By collating the identification information shown in the above with the identification index table 42, the information of the drug 12 can be obtained without using the AI model 50.

That is, the detection and reading of the identification index may be performed by an image analysis program different from the AI model 50. Compared to the AI model 50, which detects the part where the identification index (bar code, character string, etc.) exists and specializes in reading only that part, it distinguishes from the entire appearance of the identification target. Therefore, depending on the shooting conditions (for example, when the illumination is dark as a whole but only the barcode portion is bright), it may be possible to perform identification with higher accuracy than the AI model 50.

However, the AI model 50 can also detect the identification index included in the captured image 22. That is, the AI model 50 can identify the drug 12 shown in the photographed image 22 based on the appearance (outline shape, color, etc.) other than the identification index, and can also determine the presence or absence of the identification index. If the AI model 50 determines that the identification index is included in the captured image 22, the AI model 50 notifies the drug identification unit 30 of the identification information of the drug 12 corresponding to the identification index. Then, the drug identification unit 30 collates the notified identification information with the identification index table 42, so that the drug 12 is identified based on the identification index while using the AI model 50.

When the information on the drug 12 is obtained based on the identification information as described above, the information on the drug 12 obtained based on the identification information is displayed on the monitor 14 as the identification result, and the identification result and the photographed image are displayed. 22 are combined and transmitted to the learning data storage unit 60. Specifically, as shown in FIG. 4, drug imaging example data 32a and 32b are created and transmitted to the learning data storage unit 60. In this case, that is, when the identification index is detected by the image analysis program or the AI model 50, the identification based on the identification information indicated by the identification index has priority over the identification result by the AI model 50.

(Update of learning data)
The drug imaging example data 32a and 32b include information on the drug 12 identified by the drug identification unit 30 based on the identification information, and captured images 22 obtained by photographing the drug 12 to be identified at that time. For example, if the identification target is one tablet 12a alone and one tablet PTP sheet 12b as shown in FIG. 1, as shown in FIG. 4, "one sheet (10) of a drug having a product name of Example A 1 mg. Drug identification unit 3 30 creates and transmits these drug imaging example data 32a and 32b to the learning data storage unit 60. Here, the photographed image 22 is divided into the drug imaging example data 32a in which "Example A 1 mg is one sheet" and the drug imaging example data 32b in which "Example X 5 mg is one tablet". The whole of the above may be transmitted to the learning data storage unit 60 as drug imaging example data in which "Example A 1 mg is one sheet and Example X 5 mg is one tablet".

When the learning data storage unit 60 receives the drug imaging example data 32a and 32b from the drug identification unit 30, the learning data storage unit 60 updates the learning data 62 based on the drug imaging example data 32a and 32b. Specifically, the combination of the information of the drug 12 (what kind of drug was photographed) and the information of the photographed image 22 (what kind of image) included in the drug imaging example data 32a and 32b is combined. , "This drug 12 (here," Example A 1 mg "and" Example X 5 mg ") may appear in such a form in the photographed image 22".

When the learning data 62 is updated, the AI model 50 is recreated (educated) at a predetermined timing based on the updated learning data 62. It is desirable that this re-creation be performed at a timing that does not interfere with the operation of the drug identification system 10. For example, if the learning data storage unit 60 itself can recreate the AI model 50 and it does not take much time to recreate it, even if the AI model 50 is recreated every time the learning data 62 is updated. good. On the other hand, if the time required to recreate the AI model 50 is long and the drug identification system 10 becomes unavailable during the re-creation work, the drug identification system 10 is not operating (system is stopped). ), It is preferable that the AI model 50 is recreated. For example, in order to recreate it, it is necessary to connect an external device (a device different from the component shown in FIG. 1) that analyzes the learning data 62 to the drug identification system 10, so that the AI model 50 cannot be used during that time. In such a case, it is desirable to recreate it while the system is stopped. In addition to this, the timing of re-creation can be set in various ways. For example, when a predetermined period has elapsed since the previous re-creation, or when the learning data 62 is accumulated to some extent (update of the learning data 62 is predetermined). It may be a timing such as when it is performed a number of times, when the amount of data added by the update exceeds a predetermined amount, and so on.

The AI model 50 recreated based on the updated learning data 62 has a higher ability to identify the drug 12 shown in the captured image 22 than the AI model 50 before the re-creation. For example, in the drug imaging example data 32a of FIG. 4, the back side of the PTP sheet is shown, the appearance of the drug alone does not appear, and the characters of the product name are small and difficult to read, but the barcode is in a readable state. It has become. Therefore, the drug identification unit 30 can identify that the photographed image is an image of "Example A 1 mg" based on the identification information indicated by the barcode. When the learning data 62 is updated using the drug imaging example data 32a at this time and the AI model 50 is recreated (educated) based on the learning data 62, the AI model 50 will be added to the input captured image of the drug alone in the future. Even if the figure does not appear and the product name is difficult to read, it is possible to identify that "Example A 1 mg" is shown if the photographed image is similar to the drug imaging example data 32a.

Further, by accumulating a large number of drug imaging example data 32a and 32b relating to the individual agents 12a and 12b, the AI model 50 in which the information is reflected can be obtained with respect to the captured image 22 in which the plurality of agents 12 are captured. It becomes possible to identify that the individual drugs 12a and 12b are reflected in it. That is, by collecting the drug imaging example data 32a and 32b relating to the individual drugs 12a and 12b, it is possible to identify a plurality of drugs 12 at the same time. On the other hand, even when the captured image 22 is not separated as the drug imaging example data 32a and 32b relating to the individual drugs 12a and 12b, but is accumulated in the form of "drug imaging example data showing a plurality of drugs". , It becomes possible to accurately and quickly identify a plurality of drugs for a photographed image similar to the drug imaging example data.

(Procedure for identification by drug identification system)
FIG. 5 shows as a flowchart a procedure for identifying the drug 12 and updating the learning data 62 performed by the drug identification system 10 of FIG. First, the captured image 22 captured by the user is input to the drug identification unit 30 (step S01). The drug identification unit 30 starts image recognition of the input captured image 22 (step S02), and determines whether or not the captured image 22 contains an identification index by the AI model 50 or a detection function different from the AI model 50. (Step S03).

When the identification index is included in the captured image 22 (step S03-YES), the drug identification unit 30 photographs the image from the identification index table 42 of the identification index table storage unit 40 based on the identification information indicated by the identification index. Information (drug information) regarding the drug 12 in the image 22 is acquired (step S04), and the drug information is used as the identification result. Then, the drug identification unit 30 creates drug imaging example data 32a and 32b based on the captured image 22 and the acquired drug information (step S05), and transmits the data to the learning data storage unit 60 (step S06). The learning data storage unit 60 that has received the drug imaging example data 32a and 32b updates the learning data 62 based on the drug imaging example data 32a and 32b (step S07).

On the other hand, when the identification index is not included in the captured image 22 (step S03-NO), the drug information returned as the output data by the AI model 50 receiving the captured image 22 as input data is used as the identification result (step S08). ).

Then, the drug identification unit 30 transmits the identification result obtained in step S04 or step S08 to the monitor 14. The monitor 14 that has received the identification result displays the identification result to the user (step S09).

As described above, the identification result of the drug 12 targeted by the user is displayed on the monitor 14, and the learning data 62 is updated if the identification index is included in the captured image 22. According to this drug identification system 10, an AI model 50 is created based on learning data 62 in which captured images 22 taken under various past conditions are accumulated, and identification is performed using the AI model 50. .. Therefore, the drug identification system 10 identifies the drug 12 in various postures (for example, the drug 12 in an oblique direction such as the tablet PTP sheet 12b shown in FIG. 1 or the drug package containing the drug 12) in that posture. It can be performed. Therefore, the identification can be performed without performing the work of placing the drug 12 under appropriate imaging conditions as in the conventional weighing device described in Patent Document 1, so that the user can easily perform the identification. can. To give one example of "putting it in an appropriate imaging condition", in the prior art, it was necessary to unify the background color of the drug 12 to a specific color (typically black) in the captured image 22. This is a part different from that color by unifying the background color to a specific color in the conventional technology when detecting "where in the image the drug is", that is, detecting the outline (edge) of the drug. Needed to be able to be detected as a region of presence of the drug. On the other hand, if the AI model 50 is well-educated as used in the present embodiment, the background of the image may be any color (even if a plurality of colors or objects are reflected). ) Edges can be detected, and the work of unifying the background color (for example, the work of arranging the drug 12 on a black mat prepared in advance) becomes unnecessary.

Further, if the captured image 22 includes the identification index, the learning data 62 is updated only by photographing the drug 12 as the user's work, so that the user has specialized knowledge about the drug 12 (accurate drug). Even a user who does not have the information) can contribute to the update (accumulation) of the learning data 62. Further, it is possible to accumulate the captured image 22 taken in the process of drug identification as a part of the learning data 62, and the accuracy of identification by the AI model 50 is improved by performing drug identification many times. Is possible.

Further, since the identification index can be identified by the AI model 50 even if the identification index is not detected, the appearance of the drug 12 can be identified by the AI model 50 even for the drug 12 whose product name or barcode cannot be known due to, for example, one-pack dispensing. , The identification can be performed based on the contour. In addition, since the product code engraved on the drug 12 alone can be used as an identification index, it is possible to identify the drug 12 alone taken out from the drug packaging, and other than ethical drugs. , Over-the-counter drugs without barcodes can also be identified. Since the identification code is uniquely determined by the manufacturer, the same identification code may be assigned to different drugs, but even in such cases, it is often possible to distinguish by the dosage form, color, and the presence or absence of a score line, and the identification code. It is possible to perform highly accurate identification by combining the reading of the above and the shape identification by the AI model 50.

(System via network)
FIG. 6 shows a drug identification system 80 in which a plurality of local devices 86 access the main device 84 via the network 82. In the drug identification system 80, the user directly operates the local device 86. The local device 86 may be a device having a different form, and in FIG. 6, as an example of the local device 86, a smartphone 86a, a dispensing inspection device 86b (a device installed installed in a dispensing pharmacy, etc.), and a desktop PC 86c are shown. Has been done. In the figure, each of these local devices 86 is connected to the network 82 by a line, but in reality, the main device 84 may be accessed by wireless communication.

Further, the main device 84 is connected to the network 82. The main device 84 is a device (such as a server device) to be accessed by each local device 86 via the network 82, and as shown in FIG. 6, a learning data storage unit 60 that stores learning data 62 and this learning It has a main AI model 52 created based on the data 62. Like the AI model 50 in the drug identification system 10 of FIG. 1, the main AI model 52 is created based on the learning data 62 in which information about how the drug appears in the photographed image is accumulated. , It has a function of returning the photographed image as input data and the corresponding drug information as output data.

Each local device 86 has a function corresponding to the drug identification unit 30 in the drug identification system 10 of FIG. For example, by installing an identification application that provides a drug identification function on the local device 86, the arithmetic unit (CPU or processor) of the local device 86 can be used as the drug identification unit 30. In addition, each local device 86 incorporates a local AI model 56. For example, if the above-mentioned identification application includes data constituting the local AI model 56, the local device 86 is in a state where the local AI model 56 is incorporated. The data constituting the local AI model 56 can be obtained, for example, by receiving the data of the main AI model 52 at that time from the main device 84 at the time of installing or updating the identification application.

Like the main AI model 52, the local AI model 56 also has a function of returning the captured image as input data and the corresponding drug information as output data. Although not shown in FIG. 6, each local device 86 also has a function of taking an image of the drug. For example, in the case of a smartphone 86a and a dispensing inspection device 86b, a camera incorporated therein is used, and in the case of a desktop PC 86c, a camera externally attached by a USB connection or the like is a means for obtaining a photographed image of the drug.

By using the local AI model 56, each local device 86 can identify the drug (included) in the photographed image taken by itself, and whether the photographed image contains an identification index. Whether or not it is determined, and if an identification index is included, the drug can be identified based on the identification information indicated by the identification index. Hereinafter, as an example, the function of the drug identification system 80 when the smartphone 86a functions as the local device 86 will be described, but it is basic except that the data input / output between the components of the system may be performed via the network 82. Drug identification is performed in the same flow as the flowchart of FIG.

If the user takes a photographed image of the drug with the camera of the smartphone 86a and inputs it to the local AI model 56, and the local AI model 56 returns the drug information as output data, the drug information is the identification result of the smartphone. It is displayed on the screen of 86a. When it is determined that the captured image contains an identification index, the drug is identified based on the identification information corresponding to the identification index. In the identification of a drug based on this identification information, if the data corresponding to the identification index table 42 of FIG. 3 is installed on the smartphone 86a together with the identification application, the drug information corresponding to the identification information can be obtained using the data. Can be done. Alternatively, if a drug DB88a (DB: database) in which drug information related to various drugs is recorded as shown in FIG. 6 is connected to the network 82 as corresponding to the identification index table storage unit 40 of FIG. The smartphone 86a may access the drug DB 88a via the network 82 and acquire information on the photographed drug from the data in the drug DB 88a.

When a drug is identified using the identification index, the smartphone 86a creates drug imaging example data that combines the captured image at that time and the identification result (information on the identified drug), and via the network 82. Is transmitted to the main device 84. It should be noted that this transmission does not have to be performed immediately after the drug is identified, but may be performed when there is an opportunity to access the main device 84. The main device 84 updates the learning data 62 stored in its own learning data storage unit 60 based on the drug imaging example data transmitted from the smartphone 86a (local device 86). When the learning data 62 is updated, the main AI model 52 is recreated (educated) at a predetermined timing. After that, when the request for updating the local AI model 56 is received from the smartphone 86a, the copy data of the recreated main AI model 52 is transmitted to the smartphone 86a.

By the way, the discriminating ability of the main AI model 52 improves as the learning data 62 is updated (accumulated), but the discriminating ability of the local AI model 56 remains constant unless the model itself is updated. Therefore, even if the main AI model 52 can identify what is shown in the input captured image, the local AI model 56 may not be able to identify it. Therefore, when the drug contained in the captured image cannot be identified by the identification by the local AI model 56 (what kind of drug is contained in the captured image cannot be identified), the smartphone 86a is used as the main device 84 via the network 82. Is accessed, the photographed image at that time is transmitted to the main device 84, and an "inquiry" is made about the drug shown in the photographed image. Upon receiving the "inquiry", the main device 84 inputs the captured image at that time into the main AI model 52, identifies the drug shown in the captured image, and transmits the identification result to the smartphone 86a. Make an "answer". As described above, it is preferable that the drug can be identified using the main AI model 52 even from the smartphone 86a, which is the local device 86, via the network 82.

When other devices such as the dispensing inspection device 86b and the desktop PC 86c work as the local device 86, the same procedure as in the case of the smartphone 86a described above is performed. In this way, when the drug identification system 80 has a main device 84 connected via the network 82 and a plurality of local devices 86, the main device 84 is a large number of local devices such as smartphones 86a that are widely available in the world. It is possible to collect drug imaging example data from the device 86, and it is expected that the accumulation speed of the learning data 62 will be improved. When collecting drug imaging example data from a large number of devices in this way, the local device 86 uses the drug imaging example data as the imaging date, shooting location, and itself in order to ensure data consistency and improve the efficiency of data analysis. The terminal identification number and the like may be added and sent to the main device 84.

Further, when the drug contained in the captured image is identified using the local AI model 56 incorporated in the local device 86, the local device 86 is the main device 84 if the drug can be identified only by the local AI model 56. Since it is not necessary to access to, the frequency of communication via the network 82 can be minimized, the average speed of drug identification can be improved, and the amount of communication in the entire system can be suppressed.

Each local device 86 may be equipped with a means for obtaining a captured image, a communication function for accessing the network 82, and a drug identification unit 30, but may not be equipped with the local AI model 56. In this case, each local device 86 transmits the captured image to the main device 84 and identifies the drug by using the AI model 52 of the main device 84, as in the case of making the above-mentioned "inquiry". The main device 84 that has received the captured image passes the output data that is the result of identification by the AI model 52 with respect to the captured image to each local device 86, so that the drug in each local device 86 is identified.

Further, in addition to the main device 84 and the local device 86, various components can be connected to the network 82. FIG. 6 shows, as an example, a state in which the prescription DB88b, the inventory DB88c, and the medication management DB88d are connected to the network 82.

(Dispensing inspection)
The prescription DB88b is a database that stores prescription information input at the discretion of a doctor. Prescription information is information on a drug that a doctor has determined to be prescribed to the patient as a result of the doctor diagnosing the patient at a medical institution such as a hospital. In a so-called dispensing pharmacy, when a patient submits a document (prescription) containing prescription information, a drug is dispensed based on the prescription information and delivered to the patient. At this time, in the work of "dispensing inspection" for confirming whether the drug to be delivered to the patient really matches the contents described in the prescription, if the prescription DB88b is connected to the network 82, the drug identification system 80 can be used. That is, the drug to be delivered to the patient is photographed with a smartphone 86a or a camera of the dispensing inspection device 86b, the drug to be delivered is identified, and the identification result is collated with the prescription DB88b based on the prescription information. It is possible to confirm whether the dispensed drug really matches the prescription information. Here, when there are a plurality of drugs to be dispensed, the drug identification system 80 collectively identifies the plurality of drugs, and whether the information on the plurality of drugs matches the prescription information in both type and quantity. It is preferable to determine. Alternatively, the drug identification system 80 may perform identification in a plurality of times and determine whether or not the total of the identification results matches the prescription information.

(Inventory control)
The inventory DB 88c is a database that stores inventory information that fluctuates according to information on drugs that have been received at a pharmacy and information on drugs that have been issued. In other words, inventory information is information that indicates what kind of drug should be left in the pharmacy at present. If the inventory DB 88c is connected to the network 82, the drug identification system 80 can be used for the work of confirming what kind of drug is currently left in the pharmacy, that is, the so-called inventory. For example, every time a drug remaining in a pharmacy is taken out, it is photographed with a camera of a smartphone 86a or the like to identify it, and when all the drugs are taken out, whether or not the total identification result up to that point matches the inventory information. By making a determination, it is possible to confirm whether or not there is a difference between the inventory information and the actually remaining drug. At this time, if there are actually few drugs remaining compared to the inventory information, there is a possibility that the drugs have been lost, and if there are many remaining drugs, extra warehousing or shipping in a form that is not recorded. There is a possibility that you have forgotten to take out the medicine you should do. By using the drug identification system 80, it is possible to determine the possibility that such an unexpected situation has occurred.

(Medication management)
The medication management DB88d is a database that manages the medication schedule instructed by the medical institution for the drugs dispensed to the patient. Specifically, for each type of dispensed drug, the date and time of dispensing, the total quantity at the time of dispensing, the quantity to be consumed per dose, and the desired dosing frequency (after each meal, etc.) Is recorded, and based on this information, it is possible to estimate how much of the dispensed drug should remain (how much was consumed) at the current time. If the medication management DB88d is connected to the network 82, it is possible to confirm the compliance status of the medication compliance by the patient and improve the medication adherence. That is, at the site of home care, the drug remaining in the patient's hand is photographed and identified, and whether or not the identification result matches the current number of remaining drugs estimated from the medication management DB88d. By determining, it is possible to confirm whether or not the patient is taking the drug according to the medication schedule. In addition, the staff of the medical institution asks the patient why he / she does not follow the medication schedule if the patient does not take the medicine according to the medication schedule, and if there is any reason why he / she cannot follow the medication schedule, the circumstances. It is expected that the medication adherence will be improved by reviewing the medication schedule according to the above.

(Billing process)
The administrator who operates the drug identification system 80 configured via the network 82 as shown in FIG. 6 charges the user who uses the drug identification system 80 a fee according to the usage status (billing). ) Is also possible. For example, when performing identification using the main AI model 52, when updating the local AI model 56, when accessing the prescription DB88b, inventory DB88c, medication management DB88d, etc., the user tries to use the resources on the network 82. At that time, it is possible to allow the use of the resource only to the user who has agreed to pay the fee set according to the resource usage pattern.

(Identification sensor)
In the drug identification system 10 of the present invention as shown in FIG. 1, the learning data 62 is updated every time the user identifies a drug, and the AI is recreated (educated) based on the learning data 62. The drug identification accuracy of the model 50 also improves according to the number of times the drug identification system 10 is used. As a result, by incorporating the AI model 50 whose drug identification accuracy has been improved to some extent into the drug identification sensor, it is possible to construct a drug identification sensor having high drug identification accuracy.

FIG. 7 schematically shows an example of the drug identification sensor 90. The drug identification sensor 90 is connected to a substrate 91 via a flexible cable 92 and an arithmetic element 93 (CPU, processor, PLD, etc.) and a storage element 95 (memory element, etc.) arranged on the substrate 91. It has an element 94 (such as a camera) and a connector 98 for connecting the substrate 91 and an external device.

The data of the AI model 50 created (fully educated) based on the learning data 62 sufficiently accumulated by repeating the drug identification in the drug identification system 10 of FIG. 1 is stored in the storage element 95. Will be done. In the drug identification sensor 90, the arithmetic element 93 and the storage element 95 function as a processing unit in which the AI model 50 is incorporated. The drug identification sensor 90 can identify the drug contained in the photographed image by performing image recognition processing on the photographed image photographed by the image capturing element 94 in the processing unit. Specifically, the captured image may be passed to the AI model 50 as input data, and the drug information returned by the AI model 50 as output data may be used as the identification result. The identification result can be displayed on a monitor or the like connected to the connector 98.

Since the drug identification sensor 90 of FIG. 7 incorporates the AI model 50 created based on the learning data 62 sufficiently accumulated in the drug identification system 10 of FIG. 1, drug identification can be performed with extremely high accuracy. It can be carried out. Further, since the minimum components constituting the drug identification sensor 90 are only the image capturing element 94 and the processing unit (memory element 95 and calculation element 93), the dimensions of the drug identification sensor 90 should be very compact. Can be done.

It is also possible to provide the drug identification sensor 90 with a learning data storage unit 60 as shown in FIG. 1, to identify the drug based on the identification index, and to update the learning data 62. Further, by connecting the drug identification sensor 90 to the network 82 as shown in FIG. 6 via the connector 98, the learning data updated by the drug identification sensor 90 or the drug imaging example data collected by the drug identification sensor 90 can be obtained. It is also possible to update the learning data 62 of the main device 84 by transmitting to the main device 84 and using the result of the drug identification performed by the drug identification sensor 90.

(User confirmation and correction of data)
In the embodiment of FIG. 1, when the drug is identified based on the identification information indicated by the identification index, the identification result is combined with the captured image 22 to create drug imaging example data 32a and 32b. Although it is transmitted to the learning data storage unit 60, the data may be confirmed and input by the user before the drug imaging example data 32a and 32b are transmitted to the learning data storage unit 60. For example, the monitor 14 displays the identification result and also displays the "confirm" button and the "correct" button. If the displayed identification result is correct, the user selects the "confirm" button (by an input means such as a mouse or a touch panel) to inform the drug identification unit 30 that the identification result is correct. If the displayed identification result is incorrect, the user selects the "correct" button to inform the drug identification unit 30 that the identification result is incorrect, and the drug 12 (by input means such as a keyboard or touch panel). Enter the correct information of and convey it to the drug identification unit 30. When the "confirm" button is selected, the drug identification unit 30 transmits the drug imaging example data 32a and 32b as they are to the learning data storage unit 60, assuming that the created identification result has been confirmed. When the "correct" button is selected, the identification result is corrected to the correct information input by the user, and the identification result is combined with the captured image 22 to recreate the drug imaging example data 32a and 32b, and then the learning data storage unit. Send to 60. In this way, even if there is an error in reading an identification index such as a barcode, the correct information can be transmitted to the learning data storage unit 60. In inputting the "correct data", the user represents an identification index that can be read by humans as it is, such as the identification code 71a or the product name 72a written on the drug or the drug package, as shown in FIG. 2B. It is also possible to enter the identification information. In particular, in medical drugs, such an identification index as a character string is written on the drug packaging, etc., and the number represented by the barcode 76a (FIG. 2D) is often printed as a character string near the barcode 76a. .. Therefore, it is possible for the user to input "correct data" even if he / she does not have specialized knowledge about the drug. In this way, even when the data is input by the user, the drug is identified based on the identification information by inputting the identification information indicated by the identification index.

Further, even when the identification is performed by the AI model 50 without using the identification index, it is possible for the user to confirm and input the data. For example, as in the above example, the "confirm" button and the "correct" button are displayed on the monitor 14 together with the identification result by the AI model 50. Then, for each button, the identification result is confirmed or corrected in the same manner as described above, and the confirmed or corrected identification result and the captured image 22 are combined and sent to the learning data storage unit 60. If the identification index is not detected and the AI model 50 cannot identify the data, the monitor 14 displays a display indicating that the identification cannot be performed (such as "unidentifiable") and a "data input" button. The correct data may be input by the user even when this button is pressed. In this way, even when the identification is performed by the AI model 50, or when the identification is impossible by the AI model 50 or the identification index, the drug imaging example including the captured image 22 at that time is included. The data 32a and 32b can be transmitted to the learning data storage unit 60. In the input of "correct data" in this case, the user re-photographs the part where the identification index is written with the camera 20 (for example, when the first shooting is the front side of the PTP sheet, the back side). The drug is identified again by reading the bar code with a bar code reader provided separately from the camera 20) or by reading the bar code on the drug to be identified or the identification index written on the drug package. The unit 30 may be made to detect it. In addition, the drug name written on the cosmetic box, which is the unit for selling and packaging the drug, is also an identification index written on a kind of drug packaging, and the identification index written on the individual drug packaging such as a PTP sheet is difficult to read due to dirt or the like. In such cases, the identification index of such a cosmetic box may be read. Further, in the case of a bottled drug, an identification index such as a barcode attached to the bottle may be read.

(Evaluation of identification certainty)
Further, when the identification is performed by the AI model 50, it may be evaluated how reliable the identification is. For example, a value of "confidence" indicating how confidently the identification result is calculated according to how much the input captured image 22 matches the past captured image is calculated. Just do it. Then, if this "certainty" exceeds a predetermined threshold value (for example, 90%), the identification result may be combined with the captured image 22 and sent to the learning data storage unit 60 (high certainty). For example, even if it is identified by the AI model 50, drug imaging example data is created). Further, regardless of whether the "certainty" is high or low, the identification result in that case is displayed on the monitor 14 together with the value of the certainty. At this time, the monitor 14 displays the "data input button" as in the case of the above-mentioned "unidentifiable", and when the certainty is particularly low, the user may input the "correct data". In this way, the user only has to input "correct data" when the certainty is low, and the troublesome operation is reduced.

(Assistance for user data entry)
Further, instead of using the identification index or the identification by the AI model 50 as the identification result as it is, the identification index or the AI model 50 may be used as an aid for data input by the user. For example, the data is basically manually input by the user, and the data may be input by the user entering the data in the input field displayed on the monitor 14. On the other hand, when the identification index or the AI model 50 is used for identification, the identification result data is automatically inserted into the input field. The user may notify the drug identification unit 30 and the learning data storage unit 60 of the identification result by confirming the automatically inserted data, modifying the data as necessary, and confirming the data. In this way, all the input data is input after the user's confirmation, which increases the reliability of the data, while most of the data is automatically inserted, so that the user can operate it. It's not bothersome.

(Combination of AI model and identification index)
Further, the identification is not performed only by the AI model 50 or only the identification index, but the identification by the AI model 50 and the identification by the identification index may be combined to determine the identification result. In particular, for the determination of the quantity of "1 sheet" or "1 tablet", it is expected that the accuracy will be improved by combining the identification by the AI model 50 and the identification by the identification index.

For example, if the barcode of the PTP sheet is written only in one place on one sheet, if the identification is performed only by the identification index, the barcode is read to identify that the drug for "one sheet of PTP sheet" exists. Will be done. However, PTP sheets that are actually handled in pharmacies and patients' homes may be partially cut out and used while leaving a barcode. Moreover, even if the cutting is not performed, it is possible that only some tablets have already been taken out. In such a case, in the case of identification by the AI model 50, the number of tablets currently contained in the PTP sheet is based on the "shape of the cut out PTP sheet" and the "shape of the PTP sheet in which a part of the tablet region is torn". Can be identified if is left. Since the shape of the PTP sheet is almost the same even if the type of the packaged drug is different, the type of the drug is identified based on the identification index such as a barcode, and the quantity is determined from the shape of the PTP sheet. By identifying by the AI model 50, it is possible to perform highly accurate identification of both the type and the quantity. On the other hand, if the identification index written for each tablet such as the identification code can be detected by image analysis, it is possible to identify the quantity of the drug based on the identification index.

The specific correspondence between the shape and quantity of the drug package (for example, PTP sheet) can be stored in the learning data 62 as drug imaging example data in which the shape and quantity of the drug package are combined. be. For example, with respect to a PTP sheet in which only one of the 10 tablets of the PTP sheet has been taken out (only the area of one tablet is torn), the user who photographed the PTP sheet "remains in the PTP sheet shown in this photographed image". Information such as "9 tablets are used" may be input and transmitted to the learning data storage unit 60. When a large amount of information indicating the correspondence between the shape of the drug package and the quantity of the drug contained therein is accumulated in the learning data 62 in this way, the AI model 50 reflecting the information can be obtained from the shape of the drug package. It becomes possible to identify the quantity of the drug contained in the drug package. For example, if only one of the 10 PTP sheets containing the PTP sheet has been taken out and the drug imaging example data is accumulated, the AI model 50 can be used with respect to the PTP sheet having a shape similar to the accumulated drug imaging example data. It is possible to identify that "9 out of 10 sheets containing 10 tablets remain".

(About identification of brought-in medicine)
Furthermore, according to the well-educated AI model 50, even a drug to which an identification index is not attached can be identified based on the shape and color of the drug itself. Therefore, identification using the AI model 50 is very effective for identification of brought-in drugs. Bringing medicine is a medicine that a patient presents as a medicine that the patient usually takes at that time when he / she is examined, tested, treated, or hospitalized at a medical institution. For medical institutions, it is very important to understand the contents of the medicines brought by patients in order to prescribe medicines after medical examination, during treatment, and at the time of admission, and to properly perform surgery and examinations. That is, it is possible to avoid double administration of the prescribed drug and the brought-in drug, and to avoid dangerous combinations of drugs. In addition, it is possible to identify drugs that have been or should be discontinued before treatment, surgery, or examination. In addition, medical institutions can give appropriate advice to patients on how to take the drug, and can also give advice on how to take the drug with health foods and supplements. Even if the drug is not identified, it is possible to grasp the content of the drug that the patient is taking from the drug description or the information in the so-called drug notebook. However, this information does not give a complete picture of whether the patient is actually taking the drug. The most reliable way is to ask the patient to bring the drug they are actually taking and to confirm the drug they are taking. When confirming the drug to be brought, if the product name or identification code of the drug is described on the packaging of the drug as an identification index, the drug can be identified relatively easily without using the drug identification system. However, as a result of being dispensed in one package, the PTP sheet being cut off, or the tablet being taken out of the package and stored as a single substance, the identification index is not listed on the drug package in a readable form, or the drug package itself. May not exist. In this case, it is difficult to identify the brought medicine with the naked eye. The drug identification system of the present invention can identify a drug even in such a case, and is very effective for identifying a patient's brought-in drug and confirming the brought-in drug.

(Data input / output to external devices)
In the embodiment shown in FIG. 1, the imager 20 and the monitor 14 are components of the drug identification system 10, but these may be external devices of the drug identification system 10, and the drug identification system 10 is external. It suffices if the data input of the captured image 22 can be received from the above and the data of the identification result can be output to the outside.

(About data placement)
Further, in the embodiment shown in FIG. 1, the identification index table storage unit 40 and the AI model 50 are separate from the drug identification unit 30, but the data of the identification index table 42 and the data of the AI model 50 are the drug identification units. It may be incorporated in 30. On the other hand, the data in the identification index table 42 and the data in the AI model 50 may be arranged in a device distant from the drug identification unit 30 that can be accessed via the network.

(About shooting time)
Further, the photographing of the drug 12 does not need to be performed at the time of identification, and the data of the image taken in advance at a place different from the drug identification system 10 is taken in as the captured image 22 (input to the drug identification unit 30). Is also possible.

(About AI model creation method)
Further, the AI model 50 created based on the learning data 62 is not limited to the one created by supervised learning using the learning data 62 as teacher data. For example, the AI model obtained as a result of deep learning based on the learning data 62 may be used as the AI model 50 of the drug identification system 10.

10 Drug identification system 12 Drug 12a Tablet 12b Tablet PTP sheet 14 Monitor 20 Camera 22 Photographed image 30 Drug identification unit 32a, 32b Drug imaging example data 40 Identification index table Storage unit 42 Identification index table 50 AI model 52 Main AI model 56 Local AI model 60 Learning data storage unit 62 Learning data 70a Tablets 70b Powder 70c Capsules 71a, 71b, 71c Identification code 72a, 72b, 72c Product name 73a, 73c PTP sheet piece 73b Individual packaging 75a PTP sheet 76a, 76b, 76c Bar code 80 Drug identification system 82 Network 84 Main device 86 Local device 88a Drug DB
88b Prescription DB
88c inventory DB
88d medication management DB
90 Drug identification sensor 91 Board 92 Flexible cable 93 Computing element 94 Imaging element 95 Storage element 98 Connector S01-S09 Step

Claims (11)

In a drug identification system that identifies a drug taken by performing image recognition of a photographed image of the drug.
A learning data storage unit that stores as learning data how various drugs appear in captured images,
A drug identification unit that identifies the drug contained in the captured image,
With
The drug identification unit is an AI model created based on the learning data stored in the training data storage unit, and returns an captured image as input data and information on the corresponding drug as output data. A drug identification system characterized by identifying a drug contained in an input captured image by using. When a drug or a drug package for packaging a drug contains an identification index indicating drug identification information, the drug identification unit identifies the drug based on the drug identification information indicated by the identification index and identifies the drug. The drug imaging example data was created by combining the information of the drug and the information of the captured image at that time, and the drug imaging example data was transmitted to the learning data storage unit.
When the learning data storage unit receives the drug imaging example data from the drug identification unit, the learning data storage unit uses the drug information and the captured image information included in the drug imaging example data as an example of the appearance of the drug appearing in the captured image. The drug identification system according to claim 1, wherein the learning data is stored and updated. When it is determined by the AI model that the captured image contains the identification index, the drug identification unit identifies the drug based on the identification information of the drug corresponding to the identification index determined by the AI model. The drug identification system according to claim 2, wherein the drug imaging example data is created by combining the information of the identified drug and the information of the captured image at that time. Each is equipped with the drug identification unit, and has a plurality of local devices that can access the main device via a network.
When the drug imaging example data is created in the drug identification performed by the local device, the drug imaging example data is transmitted to the main device.
The drug identification system according to claim 2 or 3, wherein the main device updates learning data stored by itself based on drug imaging example data transmitted from the local device. The drug identification unit of the local device identifies the drug contained in the captured image by using the local AI model incorporated in the local device, and cannot identify the drug contained in the captured image even by using the local AI model. In this case, the claim is characterized in that the main device is accessed via a network and the drug is identified using the main AI model created based on the learning data stored in the main device. 4. The drug identification system according to 4. Claim 2 to claim that when the photographed drug is a ethical drug, the optical mark on the ethical drug packaging indicating the information about the ethical drug is used as the identification index. Item 5. The drug identification system according to any one of Item 5. If the drug taken is a drug dispensed based on prescription information entered at the discretion of the doctor
The drug identification according to any one of claims 1 to 6, wherein it is determined whether or not the information of one or more drugs identified by the drug identification unit matches the prescription information. system. When the photographed drug is a drug that is managed based on inventory information that fluctuates according to the information of the drug that has been received and the drug that has been issued at the pharmacy.
The drug identification according to any one of claims 1 to 6, wherein it is determined whether or not the information of one or more drugs identified by the drug identification unit matches the inventory information. system. If the drug taken is a drug that has been dispensed to the patient and should be taken according to the medication schedule instructed by the medical institution.
The drug identification according to any one of claims 1 to 6, wherein it is determined whether or not the information of one or more drugs identified by the drug identification unit matches the medication schedule. system. The claim is characterized in that the learning data storage unit and the drug identification unit are configured by separate devices, and the drug identification unit can access the learning data storage unit via a network. The drug identification system according to any one of claims 1 to 9. A processing unit including a storage element and a calculation element incorporating an AI model created based on the learning data obtained by the drug identification system according to any one of claims 1 to 10.
It has an image capturing element capable of capturing an image, and has
A drug identification sensor characterized in that a drug included in a photographed image is identified by performing image recognition processing on the photographed image captured by the image capturing element in the processing unit.

Images