KR102400018B1 - Method and apparatus for auto cooking - Google Patents

Abstract

The present disclosure relates to an artificial intelligence (AI) system that uses machine learning algorithms, such as deep learning, to simulate functions such as cognition and judgment of the human brain, and applications thereof. Disclosed are an automatic cooking apparatus and method for selectively irradiating light of different wavelength bands on food materials, acquiring information about the food material based on the reflected light, identifying the food ingredient, and controlling the cooking process of the food ingredient.

Description

Automatic cooking apparatus and method {Method and apparatus for auto cooking}

It relates to an automatic cooking apparatus and an automatic cooking method.

An artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike the existing rule-based smart system, the machine learns, judges, and becomes smarter by itself. As artificial intelligence systems are used, the recognition rate improves and users can understand user preferences more accurately.

Artificial intelligence technology consists of machine learning (deep learning) and element technologies using machine learning.

Machine learning is an algorithm technology that categorizes/learns the characteristics of input data by itself, and element technology uses machine learning algorithms such as deep learning to simulate functions such as cognition and judgment of the human brain. It consists of technical fields such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology for recognizing and applying/processing human language/text, and includes natural language processing, machine translation, dialogue system, question and answer, and speech recognition/synthesis. Visual understanding is a technology for recognizing and processing objects like human vision, and includes object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image improvement, and the like. Inferential prediction is a technology for logically reasoning and predicting by judging information, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendation. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data generation/classification) and knowledge management (data utilization). Motion control is a technology for controlling autonomous driving of a vehicle and movement of a robot, and includes motion control (navigation, collision, driving), manipulation control (action control), and the like.

Meanwhile, in the case of an oven for cooking food, a user directly inputs a cooking type, a cooking method, and setting information for cooking. However, it is sometimes not appropriate to use an oven according to a standardized recipe because it is complicated to set the oven according to various recipes, and even the same ingredients have different characteristics such as area and thickness. Accordingly, interest in a technology capable of minimizing the user's operation of the oven by applying artificial intelligence technology to a cooking device such as an oven and completing a dish in consideration of the characteristics of ingredients is increasing.

To provide an automatic cooking apparatus and method for selectively irradiating light of different wavelength bands on food materials, acquiring information about the food ingredients based on the reflected light, identifying the food ingredients, and controlling the cooking process of the food ingredients.

An automatic cooking apparatus according to a first aspect includes: a light irradiator for irradiating light of different wavelength bands; a photographing unit including an image sensor; a memory storing computer executable instructions; By executing the computer-executable command, the light irradiation unit and the photographing unit are controlled to obtain type information of the food material through vision recognition based on an image of the food material, and a wavelength band selected in response to the type information of the food material to obtain the characteristic information of the food material through spectral characteristic analysis based on the reflected light by irradiating the light of at least one processor; and a cooker operating according to the cooking process of the food material.

According to a second aspect, there is provided an automatic cooking method, comprising: acquiring type information of a food ingredient through vision recognition based on an image of the food ingredient; obtaining characteristic information of the food material through spectral characteristic analysis based on the reflected light by irradiating light of a wavelength band selected in response to the type information of the food material; and controlling the cooking process of the food material based on the type information of the food material and the characteristic information of the food material, wherein the type information of the food material and the characteristic information of the food material transmit light of different wavelength bands, respectively It is obtained by controlling a photographing unit including an irradiating light irradiator and an image sensor.

According to a third aspect, it is a computer-readable recording medium in which a program for executing the automatic cooking method in a computer is recorded.

1 is a diagram illustrating an automatic cooking apparatus according to an exemplary embodiment.
2 is a block diagram illustrating an automatic cooking apparatus according to an exemplary embodiment.
FIG. 3 is a diagram for explaining a process of learning and recognizing information about food ingredients by an automatic cooking apparatus in association with a server according to an exemplary embodiment;
4 is a view for explaining the configuration and operation of an automatic cooking apparatus according to an exemplary embodiment.
5 is a view for explaining the configuration and operation of an automatic cooking apparatus according to another embodiment.
6 is a flowchart illustrating an automatic cooking method according to an exemplary embodiment.
7 is a detailed flowchart illustrating a process of acquiring characteristic information of ingredients through spectral characteristic analysis in an automatic cooking method according to an exemplary embodiment.
8 is a detailed flowchart illustrating a process of controlling a cooking process in an automatic cooking method according to an exemplary embodiment.

Hereinafter, an exemplary embodiment will be described in detail with reference to the accompanying drawings. Of course, the following examples are not intended to limit or limit the scope of rights only for the purpose of specifying the technical content. What can be easily inferred by an expert in the technical field from the detailed description and examples is construed as belonging to the scope of the right.

On the other hand, in the present specification, when a component is "connected" with another component, this includes not only a case of 'directly connected' but also a case of 'connected with another component interposed therebetween'. In addition, when a component "includes" another component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

Also, terms including ordinal numbers such as 'first' or 'second' used in this specification may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

As used herein, the term “automatic cooking device” refers to a generic term for an electronic device having a cooking function. For example, as food and beverage equipment for cooking by applying heat to ingredients, an oven or a range may correspond to an automatic cooking device.

The present embodiments relate to an automatic cooking apparatus and an automatic cooking method, and detailed descriptions of matters widely known to those of ordinary skill in the art to which the following embodiments belong will be omitted.

1 is a diagram illustrating an automatic cooking apparatus 100 according to an exemplary embodiment.

Referring to FIG. 1 , an oven is illustrated as an example of an automatic cooking apparatus 100 according to an embodiment. The operation method of the automatic cooking apparatus 100 may be a convection method, a broil method, a high temperature steam method, a grill method, etc., but is not limited thereto, and may be a method in which various methods are combined. As shown in FIG. 1 , in the automatic cooking apparatus 100 , various types of heating devices may exist in the upper, lower, and side portions of the inner space, some of which may be removed or other heating devices may be added. can

Referring to FIG. 1 , the door of the automatic cooking apparatus 100 is opened so that the food material can be introduced into the inner space of the automatic cooking apparatus 100 , and the support is located at the bottom so that the food material can be placed in the inner space of the automatic cooking apparatus 100 . located at a height. In FIG. 1, the support on which the ingredients are to be placed is located at any one of the heights of the upper and lower two stages, but the present invention is not limited thereto. However, various types of supports such as 5-stage, 6-stage, and 7-stage may be provided. In the automatic cooking apparatus 100 , a manipulation unit for inputting a user operation may be provided outside the automatic cooking apparatus 100 so that the user can control the operation of the automatic cooking apparatus 100 . In the case of FIG. 1 , the automatic cooking apparatus 100 is provided with a dial-type operation unit, and the user can control the automatic cooking apparatus 100 by holding the handle of the dial and rotating it. However, the operation method of the automatic cooking apparatus 100 is not limited to the dial method illustrated in FIG. 1 , and may be various methods such as a touch screen method providing a user interface or a button method.

2 is a block diagram illustrating an automatic cooking apparatus 100 according to an exemplary embodiment.

Those of ordinary skill in the art related to the present embodiment can see that other general-purpose components other than the components shown in FIG. 2 may be further included.

The automatic cooking apparatus 100 may include a light irradiation unit 110 , a photographing unit 120 , a processor 130 , a memory 140 , and a cooker 150 .

The light irradiator 110 may irradiate light of different wavelength bands, respectively. The light irradiator 110 may sequentially irradiate light of different wavelength bands. For example, when photographing food ingredients located in the internal space of the automatic cooking apparatus 100, light of wavelength bands corresponding to red (Red, R), green (Green, G), and blue (Blue, B) colors are sequentially transmitted. It is possible to obtain an image of the food material by irradiating it. The automatic cooking apparatus 100 may mainly use light in the visible light region to photograph food ingredients located in the space inside the automatic cooking apparatus 100 , and may also mix and use light in the infrared region if necessary. As another example, the automatic cooking apparatus 100 may sequentially irradiate light of a wavelength band selected according to the type of food material. Since there is light of a wavelength band suitable for identifying the characteristics of ingredients according to the types of ingredients, the automatic cooking apparatus 100 determines the type information of ingredients located in the internal space of the automatic cooking apparatus 100 to display the information on the types of ingredients. Correspondingly, light of a selected wavelength band may be sequentially irradiated. The automatic cooking apparatus 100 may select and use light of a specific wavelength band in the near-infrared region in order to understand the characteristics of the food material located in the space inside the automatic cooking apparatus 100, and, if necessary, may emit light of a specific wavelength band of the visible light region. Light can also be used.

According to an example of the light emitting unit 110, the light emitting unit 110 is a plurality of light emitting elements (light emitting elements) each emitting light of a single wavelength band and a light emitting element selected from among the plurality of light emitting elements emitted from It may include a light guide that provides a path through which light travels.

According to another example of the light irradiation unit 110 , the light irradiation unit 110 includes a light emitting device emitting light of multiple wavelength bands, a band filter passing light of a selected wavelength band among the multiple wavelength bands, and light of the selected wavelength band. It may include a light guide providing this moving path.

The photographing unit 120 is a component that generates an image of an electrical signal from incident light, and may include at least one image sensor. In this case, the image sensor may be a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor that converts an optical signal into an electrical signal. The photographing unit 120 may photograph an image by using light of a single wavelength band emitted from the light irradiation unit 110 . The photographing unit 120 may not use a filter used to pass light of a single wavelength band by capturing an image using light of a single wavelength band. However, in some cases, the photographing unit 120 may include a predetermined bandpass filter for passing predetermined single wavelength bands.

The photographing unit 120 may further include a noise reduction filter for controlling light of a noise wavelength band incident to the image sensor. The photographing unit 120 may further include a lens and a lens driving unit for adjusting positions of the lenses.

The memory 140 may store a program for processing and control of the processor 130 , and may also store data input to or output from the automatic cooking apparatus 100 . Memory 140 may store computer executable instructions.

The processor 130 generally controls the overall operation of the automatic cooking apparatus 100 . The processor 130 may include at least one processor. The processor 130 may include a plurality of processors or may be a single processor in an integrated form, depending on functions and roles thereof.

The processor 130 controls the light irradiation unit 110 and the photographing unit 120 by executing the computer executable instructions stored in the memory 1100, and through vision recognition based on the image of the food material, type information of the food material. and irradiating light of a wavelength band selected in response to the type information of the food material to the food material, and through spectral characteristic analysis based on the reflected light, the characteristic information of the food material can be obtained. (PH), moisture content, etc.

Meanwhile, the processor 130 may further perform spectral characteristic analysis on the food material according to the type information of the food ingredient obtained through vision recognition to obtain detailed information of the food ingredient. The processor 130 may further perform spectral characteristic analysis on the type of a predetermined food material to obtain more precise type information of the food material or additional information of the food material.

The processor 130 selects a wavelength band in response to the type information of the food material, determines the amount of light according to the location of the food material and the external light intensity for the light of the selected wavelength band, irradiates the food material, and analyzes the spectral characteristics based on the reflected light At the time, calibration may be performed according to the position of the food material and the external light intensity, and characteristic information of the food material may be obtained. The processor 130 may select a wavelength band suitable for detecting characteristic information such as protein, fat, moisture, acidity, and moisture content for each food ingredient by using the database-formatted food ingredient information. The processor 130 may remove noise caused by external light for the selected wavelength band and perform spectral characteristic analysis.

The processor 130 may control the cooking process of the ingredients based on the type information of the ingredients and the characteristic information of the ingredients. The processor 130 may compare the type information of the ingredients and the characteristic information of the ingredients with the databased ingredient information before cooking, derive an optimal recipe using the ingredients, and control the cooking process according to the derived recipe. The processor 130 may control the cooking process of the ingredient by determining the cooking state of the ingredient based on the characteristic information of the ingredient, and determining at least one of a cooking method, a cooking time, and a cooking temperature based on the identified cooking state. can The cooking method may be a direct fire method, a convection method, a steam method, an RF cooking method, and the like. The cooking time may be a set time set for completion of cooking of the ingredients or a time elapsed from a cooking start time. The cooking temperature may be a set temperature set to complete cooking of the food or a current temperature in the cooking process.

When a predetermined time elapses or there is a change in the cooking state of the food ingredient when controlling the cooking process, the processor 130 may determine the current cooking stage based on the characteristic information of the ingredient in the cooking process, and based on the actual cooking state, You can set, change, and maintain the method, cooking time, and cooking temperature.

The processor 130 may selectively irradiate light of a wavelength band used to obtain predefined sensing information according to the progress of the cooking process of the food material to estimate the cooking state or the end of cooking time of the food ingredient. For example, the processor 130 obtains denaturation information on the surface of a food ingredient, in particular, denaturation information on a protein through spectral characteristic analysis, and optimizes the cooking temperature or cooking time based on the protein transformation according to the scheduling of the food ingredient. You can control the cooking process.

The processor 130 pre-defines the sensing information to be detected according to the progress of the cooking process of the food, and selectively irradiates light of a wavelength band used to obtain such sensing information, and analyzes the spectral characteristics based on the reflected light. You can check the doneness of the ingredients. For example, when the cooking process of the steak is sequentially performed according to the steps of surface drying, surface burning, and deep-cooking, the processor 130 detects the surface moisture content of the steak in the surface drying step, and protein denaturation in the surface burning step In the step of detecting information and cooking, it is possible to preset not only protein denaturation information but also sensing information required to detect information about changes in thickness or volume of food materials and wavelength bands in which sensing information can be obtained.

When the automatic cooking apparatus 100 can measure the volume of a food ingredient, the processor 130 determines the cooking state of the ingredient based on the characteristic information of the ingredient and the volume change information between the volume of the ingredient before cooking and the volume due to heating. Thus, it is possible to control the cooking process of the food material. To this end, the automatic cooking apparatus 100 measures the volume of the food material by applying an algorithm for estimating the volume to the captured image, or may further include a volume meter for measuring the volume of the food material.

When the automatic cooking apparatus 100 further includes a probe for obtaining internal temperature information and composition information of a food ingredient, the processor 130 determines the temperature information and composition information of the ingredient based on the characteristic information of the ingredient and the internal temperature information and configuration information of the ingredient. By understanding the cooking state, it is possible to control the cooking process of the ingredients. In this case, the processor 130 may recognize the position of the probe through the photographing unit 120 and inform the user to correct the position of the probe.

In order to estimate the type or characteristic information of a food ingredient in an image obtained by photographing the food material using at least one light of a single wavelength band of the visible light region and/or the near infrared region, a certain region in the image is the food material region or It is possible to learn a criterion for determining whether the characteristic information corresponds to an area where the characteristic information can be known. The processor 130 determines what data to use in order to determine whether it corresponds to the food ingredient region or the region in which characteristic information can be known, and how to determine whether it corresponds to the food ingredient region or the region in which characteristic information can be known using the data. standards can be learned. The processor 130 acquires data to be used for learning, and applies the acquired data to a data recognition model to be described later, thereby learning a criterion for determining which region in the image corresponds to a food ingredient region or a region in which characteristic information is known. can do.

The processor 130 may determine which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known, based on information on various types of ingredients.

The processor 130 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or using an existing general-purpose processor (eg, CPU or application processor) and a graphics-only processor (eg, GPU). It may be manufactured and mounted on the automatic cooking apparatus 100 .

The processor 130 may acquire data necessary to learn a criterion for determining which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known. For example, the processor 130 may acquire image data, for example, an image or a moving picture. The processor 130 may acquire data directly input from the automatic cooking apparatus 100 , selected data, set data, and the like.

The processor 130 may acquire data input from a user, data photographed or pre-stored in the automatic cooking apparatus 100 , or data received from an external device, but is not limited thereto.

The processor 130 may remove noise or process data such as images and moving pictures in a predetermined form so as to select meaningful data.

The processor 130 may have a criterion for data selection for each data type such as an image or a moving picture, and may select data necessary for learning using such a criterion. The processor 130 may select data required for learning for learning which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known.

The processor 130 may learn a criterion for determining which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known, based on the learning data. In addition, the processor 130 may learn a criterion for which learning data to use in order to determine which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known.

The processor 130 may learn how to determine which region in the image corresponds to a food ingredient region or a region in which characteristic information is known. For example, the processor 130 may learn how the first data recognition model determines which region in the image corresponds to the food ingredient region. In addition, the processor 130 may learn how to determine which region in the image corresponds to a region in which the characteristic information of the ingredient is known by the second data recognition model.

In addition, the processor 130 may train a data recognition model used to determine which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known, using the learning data.

The data recognition model may be a model based on a neural network. For example, a model such as a deep neural network (DNN) may be used as the data recognition model, but is not limited thereto.

The processor 130 may train the data recognition model through supervised learning, unsupervised learning, reinforcement learning using feedback, or the like.

The processor 130 may input the evaluation data to the data recognition model and, when the recognition result output from the evaluation data does not satisfy a predetermined criterion, may allow it to learn again.

The processor 130 may determine which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known, using the learned data recognition model.

The processor 130 may acquire various types of data to determine which region in the image corresponds to a food ingredient region or a region in which characteristic information is known. For example, the processor 130 may acquire image data such as an image or a moving picture. For example, the processor 130 may acquire data directly inputted or selected from the automatic cooking apparatus 100 , or information sensed using various sensors in the automatic cooking apparatus 100 .

The processor 130 may pre-process the acquired data or information so that the acquired data or information can be used to determine which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known. For example, the processor 130 may remove noise or process image data such as an image or a moving picture in a predetermined form so as to select meaningful data.

The processor 130 may select data required to determine which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known.

The processor 130 may apply the selected data to the data recognition model to determine which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known.

The processor 130 determines which region in the image obtained by photographing the food material using at least one light of a single wavelength band of the visible light region and/or the near-infrared region corresponds to a food material region or a region in which characteristic information can be known, It is possible to estimate the type or characteristic information of the food material.

The processor 130 compares the estimated type or characteristic information of the food material with the database material information, re-estimates the current state or cooking state of the food material, and based on the estimated current state or cooking state of the food material, etc. You can control the cooking process.

The cooker 150 may operate according to the cooking process of the food material controlled by the processor 130 . The cooker 150 may be a heating device for cooking food or a steamer used in a cooking process of food. The cooker 150 may be a heating means of various types according to a cooking method. The cooker 150 may further include a partial heating means or an additional heating means together with the heating means mainly used in the automatic cooking apparatus 100 .

FIG. 3 is a diagram for explaining a process in which the automatic cooking apparatus 100 according to an exemplary embodiment interworks with the server 200 to learn and recognize information about food ingredients.

Referring to FIG. 3 , the server 200 may collect various information related to food ingredients and store the food ingredient information in a database. The server 200 may learn a criterion for determining which area in the image corresponds to a food material area or an area in which characteristic information can be known by checking various image data of the food material. The automatic cooking apparatus 100 may determine which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known by using the data recognition model learned from the server 200 .

The server 200 determines which data to use in order to determine which region in the image corresponds to the food ingredient region or the region in which characteristic information can be known, and which region in the image can know the food ingredient region or characteristic information using the data. You can learn the criteria for how to judge whether it falls within the domain. The server 200 acquires data to be used for learning, and applies the acquired data to a data recognition model to learn a criterion for determining which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known. can

The automatic cooking apparatus 100 may receive the data recognition model learned by the server 200 and determine which region in the image corresponds to a food ingredient region or a region in which characteristic information can be known. Alternatively, the automatic cooking apparatus 100 transmits an image of a food ingredient to the server 200, and the server 200 applies the data recognition model to a region in the image where the ingredient region or characteristic information can be known. You can request to determine whether it is applicable, and receive the result.

The automatic cooking apparatus 100 determines which region in an image obtained by photographing a food material using at least one light of a single wavelength band of a visible light region and/or a near-infrared region corresponds to a food material region or a region in which characteristic information can be known and the type or characteristic information of the ingredients can be estimated. The automatic cooking apparatus 100 compares the estimated type or characteristic information of the food material with the database material information, re-estimates the current state or cooking state of the food material, and the estimated current state or cooking state of the food material. Based on the cooking process can be controlled.

4 is a view for explaining the configuration and operation of the automatic cooking apparatus 100 according to an exemplary embodiment.

Referring to FIG. 4 , it can be seen that the support 170 on which the food material can be placed is installed in the internal space 160 of the automatic cooking apparatus 100 , and the food material is positioned on the support 170 . The automatic cooking apparatus 100 may place the food material on the support 170 of the inner space 160 through the door that opens and close, and seal the inner space 160 so that the food material can be cooked. The support 170 may have different installation positions depending on the type of food. The automatic cooking apparatus 100 may adjust the height of the support unit 170 according to the type information of the ingredients.

Referring to FIG. 4 , it can be seen that the automatic cooking apparatus 100 irradiates light to the food material on the support 170 of the inner space 160 . To this end, the automatic cooking apparatus 100 may include a light irradiation unit 110 , and the light irradiation unit 110 may be implemented in various ways in consideration of the purpose and performance of the automatic cooking apparatus 100 .

As shown in FIG. 4 , the light irradiation unit 110 emits light from a plurality of light emitting devices 111 each emitting light of a single wavelength band and light emitted from a light emitting device selected from among the plurality of light emitting devices 111 . It may include a light guide 113 that provides a moving path. The plurality of light emitting devices 111 may be in the form of a light emitting device array, and each of the light emitting devices may be a light emitting diode (LED) emitting light of a single wavelength band. 4, the light emitting unit 110 includes a light emitting device (not shown) emitting light of multiple wavelength bands, a band filter (not shown) passing light of a selected wavelength band among multiple wavelength bands, and a selected A light guide 113 that provides a path through which light of a wavelength band travels may be included.

Referring to FIG. 4 , the light emitted from the plurality of light emitting devices 111 in the form of an array is irradiated to the food material in the internal space 160 of the automatic cooking apparatus 100 through the light guide 113 . It can be seen that the irradiation unit 110 is installed. The plurality of light emitting elements 111 in the form of an array may be mounted in the housing 190 forming the inner space 160 of the automatic cooking apparatus 100 . The housing 190 is a frame in which the components of the automatic cooking apparatus 100 can be mounted, and may serve as a muffle to prevent heat generated in the automatic cooking apparatus 100 from being emitted to the outside. The light guide 113 is a medium capable of transmitting light, such as a fiber optic cable, and may provide a path through which the light emitted from the plurality of light emitting devices 111 moves. One end of the light guide 113 may be in contact with a plurality of light emitting devices 111 from which light is emitted. The other end of the light guide 113 is connected to the left side of the inner space 160 of the automatic cooking apparatus 100 as shown in FIG. 4 so that the light passing through the light guide 113 can be irradiated to the food material. Each may be located on the upper end of the right side portion, but is not limited thereto. In order to minimize the light saturation, the light irradiator 110 may irradiate light with an incident angle of about 45 degrees to the food material, but the incident angle to the food material is not limited thereto. As shown in FIG. 4 , the light guide 113 is disposed in the housing 190 of the automatic cooking apparatus 100 and a first heat-resistant window 115 is disposed at the distal end of the light guide from which light is irradiated from the light guide 113 . can be located The light irradiation unit 110 may irradiate light to the food material at an angle α.

In the housing 190 of the automatic cooking apparatus 100 , the second heat-resistant window 125 may be positioned at the front end of the photographing unit 120 to which the photographing unit 120 is mounted and light is incident.

The cooling fan 180 may cool the light irradiation unit 110 and the photographing unit 120 , and may be disposed at an appropriate place in the housing 190 in consideration of the size, function, operation method, etc. of the automatic cooking apparatus 100 . can For example, the cooling fan 180 may be disposed in a suitable place to cool both the light emitting elements 111 and the photographing unit 120 of the light irradiation unit 110, and the light emitting elements 111 and the photographing unit ( 120) may be arranged in plurality according to the location.

5 is a view for explaining the configuration and operation of the automatic cooking apparatus 100 according to another embodiment.

A description of the content overlapping with the part described in FIG. 4 will be omitted.

Referring to FIG. 5 , as shown in FIG. 4 , it can be seen that the automatic cooking apparatus 100 irradiates light to the food material on the support 170 of the inner space 160 . Unlike FIG. 4 , the light irradiator 110 ) is different in that there is only one. As shown in FIG. 5 , the light irradiator 110 may be located on any one of the side surfaces of the inner space 160 of the automatic cooking apparatus 100 , but is not limited thereto. In order to irradiate light from a higher place, the light irradiator 110 is located at the top of the side of the inner space 160 of the automatic cooking apparatus 100 or at the edge of the upper portion of the inner space 160 of the automatic cooking apparatus 100 . ) can be located.

The light irradiation unit 110 may irradiate light to the food material at an angle β. Unlike the light irradiation unit located on both sides of the inner space 160 of the automatic cooking apparatus 100 in FIG. 4 , in FIG. 5 , since the light irradiation unit is located on one side of the inner space 160 of the automatic cooking apparatus 100 , , the light irradiation unit 110 of the automatic cooking apparatus 100 may irradiate light to the food material at an irradiation angle β that is wider than or equal to the irradiation angle α of FIG. 4 .

6 is a flowchart illustrating an automatic cooking method according to an exemplary embodiment.

In operation 610 , the automatic cooking apparatus 100 may acquire type information of the food ingredient through vision recognition based on an image of the food ingredient. The type information of the food material may be obtained by controlling a light irradiator that irradiates light of different wavelength bands, respectively, and a photographing unit including an image sensor. The automatic cooking apparatus 100 may further perform spectral characteristic analysis on the food material according to the type information of the food material obtained through vision recognition to obtain detailed information of the food material. The automatic cooking apparatus 100 may further perform spectral characteristic analysis on the type of a predetermined food material to obtain more precise type information of the food material or additional information of the food material.

In operation 620 , the automatic cooking apparatus 100 may acquire the characteristic information of the food material through spectral characteristic analysis based on the reflected light by irradiating light of a wavelength band selected in response to the type information of the food material. The characteristic information of the food material may be obtained by controlling a light irradiator for irradiating light of different wavelength bands, respectively, and a photographing unit including an image sensor.

7 is a detailed flowchart illustrating a process of acquiring characteristic information of ingredients through spectral characteristic analysis in an automatic cooking method according to an exemplary embodiment.

In operation 710 , the automatic cooking apparatus 100 may select a wavelength band corresponding to the type information of the food material.

In operation 720, the automatic cooking apparatus 100 may acquire the position of the food material and the external light intensity. For example, the automatic cooking apparatus 100 may recognize the location of the food material by analyzing an image of the food material or sensing the height of a support on which the food material is placed. When the automatic cooking apparatus 100 takes a spectrum image of a single wavelength band for each of the light emitting devices 111 emitting light of a single wavelength band, saturation according to adjusting the light intensity of a specific light emitting device By grasping the viewpoint, the intensity of external light in a specific wavelength band can be grasped. Meanwhile, the automatic cooking apparatus 100 may measure the external light intensity using a separate external light sensor for measuring the external light intensity.

In operation 730, the automatic cooking apparatus 100 determines the amount of light according to the location of the food material and the external light intensity with respect to the light of the selected wavelength band, and irradiates the light to the food material.

In operation 740 , the automatic cooking apparatus 100 may perform calibration according to the location of the food material and the external light intensity when analyzing the spectral characteristics based on the reflected light. For example, the automatic cooking apparatus 100 may change the light intensity of the irradiated light emitting device 111 according to the intensity of external light, and may determine a calibration weight value according to the light intensity of the irradiated light emitting device 111 . The automatic cooking apparatus 100 may perform calibration by removing a value equal to the external light intensity from the photographed image and then multiplying it by a calibration weight value.

In operation 750 , the automatic cooking apparatus 100 may acquire characteristic information of the ingredients.

Referring back to FIG. 6 , in operation 630 , the automatic cooking apparatus 100 may control the cooking process of the food material based on the type information of the food ingredient and the characteristic information of the ingredient.

8 is a detailed flowchart illustrating a process of controlling a cooking process in an automatic cooking method according to an exemplary embodiment.

In operation 810 , the automatic cooking apparatus 100 may determine a cooking process by determining the current state of the automatic cooking apparatus 100 . The automatic cooking apparatus 100 may determine whether the automatic cooking apparatus 100 is cooking or before cooking, and, if it is before cooking, may perform a cooking setting operation, and if it is cooking, may perform a cooking process to complete the cooking. .

In step 820 , if it is determined that the cooking time is before cooking, the automatic cooking apparatus 100 compares the type information of the ingredients and the characteristic information of the ingredients with the databased ingredient information before cooking to derive an optimal recipe using the ingredients. .

In operation 830, the automatic cooking apparatus 100 may control the cooking process according to the derived recipe. For example, the automatic cooking apparatus 100 determines a cooking method, a cooking time, a cooking temperature, etc. according to recipe-related information among the databased ingredient information, and operates according to the determined cooking method, cooking time, and cooking temperature. can Even without a user's input, the automatic cooking apparatus 100 recognizes the food ingredient by itself, grasps the state of the ingredient, and starts cooking so that the cooking process of the ingredient is performed according to the recipe most suitable for the current state of the ingredient.

In step 840, when it is determined that cooking is in progress, the automatic cooking apparatus 100 selectively irradiates light of a wavelength band used to obtain predefined sensing information according to the progress of the cooking process of the food material to determine the cooking state or the cooking completion time of the food ingredient. can be estimated The automatic cooking apparatus 100 detects a cooking state of a food ingredient based on the characteristic information of the ingredient, and determines at least one of a cooking method, a cooking time, and a cooking temperature based on the identified cooking state, thereby performing a cooking process of the ingredient. can be controlled The automatic cooking apparatus 100 operates a heating mechanism capable of heating a specific part or part when there is a part or part that requires further partial heating or additional heating based on the cooking state of each part or each part of the food material. can For example, when a specific part of a food ingredient has a lower cooked level compared to other parts, the specific part of the ingredient may be partially heated using microwave waves, light waves, high frequency waves, or the like. In addition, when it is desired to cook the food material faster as a whole, the food material may be cooked by further using an additional heating means together with the main heating means.

On the other hand, when the automatic cooking apparatus 100 determines that it is necessary to correct the initially set cooking method, cooking time, cooking temperature, etc. according to the optimal recipe based on the current cooking state of the ingredients, the set value of the item to be corrected You can add new ones, change them, or keep them.

When the automatic cooking apparatus 100 is capable of measuring the volume of the food material, the automatic cooking apparatus 100 may more accurately determine the amount of the food material based on the characteristic information of the food ingredient and the volume change information between the volume of the ingredient before cooking and the volume due to heating. By understanding the cooking state, it is possible to control the cooking process of the ingredients.

When the automatic cooking apparatus 100 further acquires the temperature information and the composition information inside the food material by using a probe that acquires the temperature information and the composition information inside the food material, the automatic cooking apparatus 100 may Based on the internal temperature information and configuration information, it is possible to more accurately determine the cooking state of the food material, thereby controlling the cooking process of the food material. In this case, the automatic cooking apparatus 100 may recognize the position of the probe through the photographing unit 120 and inform the user to correct the position of the probe. The automatic cooking apparatus 100 may acquire temperature information and configuration information inside the food material at the corrected position of the probe. The automatic cooking apparatus 100 more accurately grasps and learns the cooking state of the current food material based on the temperature information and composition information of the food material obtained using the probe and the characteristic information of the food material obtained through spectral characteristic analysis, The current cooking process may be controlled or may be reflected in future cooking process control.

In operation 850, the automatic cooking apparatus 100 may determine whether cooking of the food material is completed. When cooking is not completed, the automatic cooking apparatus 100 may perform the process of step 840 again.

In operation 860, when the cooking of the food material is completed, the automatic cooking apparatus 100 may notify that the cooking is completed. In this case, the automatic cooking apparatus 100 may inform the user of information about the cooked dish and how to eat it.

Meanwhile, the automatic cooking method described above can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates such a program using a computer-readable storage medium. Such computer-readable storage media include read-only memory (ROM), random-access memory (RAM), flash memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, and DVDs. -Storing ROMs, DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, magnetic tapes, floppy disks, magneto-optical data devices, optical data storage devices, hard disks, solid-state disks (SSDs), and may store instructions or software, related data, data files, and data structures, and are stored on a processor or computer so that the processor or computer may execute the instructions. It can be any device capable of providing instructions or software, associated data, data files, and data structures.

So far, examples have been mainly looked at. Those of ordinary skill in the art to which the disclosed embodiments pertain will understand that the disclosed embodiments may be implemented in modified forms without departing from essential characteristics. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the invention is indicated in the claims rather than the description of the above-described embodiments, and all differences within an equivalent range should be construed as being included in the scope of the invention.

Claims (20)

a light irradiator for irradiating light of different wavelength bands, respectively;
a photographing unit including an image sensor;
a memory storing computer executable instructions;
By executing the computer-executable instructions, the light irradiation unit and the photographing unit are controlled, the type information of the food material is obtained through vision recognition based on an image of the food material, and the information of the food material is selected from the light of the different wavelength bands. The food material is irradiated with light of a wavelength band selected in response to the type information, and the characteristic information of the food material is obtained through spectral characteristic analysis based on the reflected light, and based on the type information of the food material and the characteristic information of the food material, at least one processor for controlling the cooking process of the food; and
a cooker operating according to the cooking process of the food;
Automatic cooking device comprising a. The method of claim 1,
The at least one processor,
Select a wavelength band corresponding to the type information of the food material, determine the amount of light according to the location of the food material and the external light intensity with respect to the light of the selected wavelength band, and irradiate the food material, and analyze the spectral characteristics based on the reflected light An automatic cooking apparatus for performing calibration according to the location and external light intensity of the food material, and acquiring characteristic information of the food material. The method of claim 1,
the at least one processor,
An automatic cooking apparatus that compares the type information of the ingredients and the characteristic information of the ingredients with the ingredient information stored in a database before cooking, derives a recipe using the ingredients, and controls a cooking process according to the derived recipe. The method of claim 1,
the at least one processor,
An automatic cooking apparatus for estimating a cooking state or an end of cooking time of the food material by selectively irradiating light in a wavelength band used to obtain predefined sensing information according to the progress of the cooking process of the food material. The method of claim 1,
The light irradiation unit,
a plurality of light emitting elements each emitting light of a single wavelength band; and
a light guide providing a path through which light emitted from a light emitting device selected from among the plurality of light emitting devices moves;
Including, automatic cooking device. The method of claim 1,
The light irradiation unit,
a light emitting device emitting light of multiple wavelength bands;
a band filter for passing light of a selected wavelength band among the multiple wavelength bands; and
a light guide providing a path through which light of the selected wavelength band travels;
Including, automatic cooking device. The method of claim 1,
the at least one processor,
An automatic cooking apparatus for controlling a cooking process of the food material by grasping a cooking state of the food material based on the characteristic information of the food material and volume change information between the volume of the food material before cooking and the volume according to heating. The method of claim 1,
Further comprising a probe for obtaining temperature information and composition information inside the food material,
the at least one processor,
An automatic cooking apparatus for controlling a cooking process of the food material by grasping a cooking state of the food material based on the characteristic information of the food material and internal temperature information and configuration information of the food material. 9. The method of claim 8,
the at least one processor,
An automatic cooking device that recognizes the position of the probe through the photographing unit and informs the user to correct the position of the probe. The method of claim 1,
a first heat-resistant window disposed at a distal end of the light guide through which a light guide is disposed in the housing of the automatic cooking device to irradiate light from the light guide;
a second heat-resistant window positioned at the front end of the photographing unit to which the photographing unit is mounted in the housing to receive light; and
a cooling fan for cooling the light irradiation unit and the photographing unit;
Further comprising, an automatic cooking device. obtaining type information of the food material through vision recognition based on an image of the food material;
obtaining characteristic information of the food material through spectral characteristic analysis based on the reflected light by irradiating light of a wavelength band selected corresponding to the type information of the food material among lights of different wavelength bands to the food material; and
controlling a cooking process of the food material based on the type information of the food material and the characteristic information of the food material;
including,
The type information of the food material and the characteristic information of the food material are obtained by controlling a light irradiator for irradiating light of different wavelength bands, respectively, and a photographing unit including an image sensor. 12. The method of claim 11,
The step of obtaining the type information of the food material includes:
According to the type information of the food material obtained through the vision recognition, further performing spectral characteristic analysis on the food material to obtain detailed information of the food ingredient, the automatic cooking method. 12. The method of claim 11,
The step of obtaining the characteristic information of the food material through the spectral characteristic analysis,
selecting a wavelength band corresponding to the type information of the food material;
determining the amount of light according to the position of the food material and the external light intensity with respect to the light of the selected wavelength band and irradiating the food material;
performing calibration according to the position of the food material and external light intensity when analyzing the spectral characteristics based on the reflected light; and
obtaining characteristic information of the food material;
Including, automatic cooking method. 12. The method of claim 11,
The step of controlling the cooking process of the food material,
deriving a recipe using the ingredients by comparing the type information of the ingredients and the characteristic information of the ingredients with the ingredient information stored in a database before cooking; and
controlling a cooking process according to the derived recipe;
Including, automatic cooking method. 12. The method of claim 11,
The step of controlling the cooking process of the food material,
and estimating a cooking state or a cooking completion time of the food material by selectively irradiating light in a wavelength band used to obtain predefined sensing information according to the progress of the cooking process of the food material. 12. The method of claim 11,
Further comprising the step of acquiring volume change information between the volume of the food before cooking and the volume according to heating,
The step of controlling the cooking process of the food material,
An automatic cooking method for controlling a cooking process of the food material by grasping a cooking state of the food material based on the characteristic information of the food material and the volume change information. 12. The method of claim 11,
Further comprising the step of obtaining temperature information and composition information inside the food material by using a probe for obtaining the temperature information and composition information inside the food material,
The step of controlling the cooking process of the food material,
An automatic cooking method for controlling a cooking process of the food material by grasping a cooking state of the food material based on the characteristic information of the food material and the internal temperature information and configuration information of the food material. 18. The method of claim 17,
The step of obtaining the temperature information and the composition information inside the food material,
recognizing the position of the probe and notifying the user to correct the position of the probe; and
acquiring temperature information and composition information inside the food material at the modified position of the probe;
Including, automatic cooking method. 12. The method of claim 11,
The step of controlling the cooking process of the food material,
Controlling the cooking process of the ingredient by determining the cooking state of the ingredient based on the characteristic information of the ingredient, and determining at least one of a cooking method, a cooking time, and a cooking temperature based on the identified cooking state; Automatic cooking method. A computer-readable recording medium in which a program for executing the method of any one of claims 11 to 19 in a computer is recorded.

Images