CN112037440A - Intelligent settlement system and settlement method based on machine learning - Google Patents
Intelligent settlement system and settlement method based on machine learning Download PDFInfo
- Publication number
- CN112037440A CN112037440A CN202010762293.2A CN202010762293A CN112037440A CN 112037440 A CN112037440 A CN 112037440A CN 202010762293 A CN202010762293 A CN 202010762293A CN 112037440 A CN112037440 A CN 112037440A
- Authority
- CN
- China
- Prior art keywords
- intelligent
- settlement
- dinner plate
- menu
- machine learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/0036—Checkout procedures
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/12—Cash registers electronically operated
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an intelligent settlement system based on machine learning, which comprises an intelligent menu planning platform, an intelligent product output machine and an AI visual settlement table; the intelligent menu planning platform is connected with the intelligent product output machine and the AI visual settlement table; the intelligent product output machine is used for recording dishes and output quantity; the intelligent menu plan platform is used for generating a menu plan according to the intelligent product output machine; the AI visual settlement table is used for settlement of food. According to the invention, from the arrangement of a menu plan, the big data analysis is carried out through the data collected by the product output machine and the AI visual settlement table, so that reasonable menu recommendation is intelligently generated for the user, manual menu making is also avoided, the diversity of the menu is more comprehensive, and manpower and material resources are saved.
Description
Technical Field
The invention belongs to the technical field of machine vision recognition, and particularly relates to an intelligent settlement system and a settlement method based on machine learning.
Background
With the continuous development of economy in China and the gradual improvement of the living standard of people, the consumption concept of people is changed, the fast pace is already the main state of life of people at present, fast food which is convenient and fast to fetch immediately becomes a busy dining mode which is more favored by young people and office workers.
However, fast food items are various in style, the dining time interval is concentrated, the instantaneous flow of personnel is large, and the settlement speed and the accuracy rate are greatly stressed by adopting the traditional manual cash-collecting mode.
And the menu arrangement of daily meals is very painful, the diversity is considered, the sales rate is kept, a large amount of waste cannot be caused, and the cost is considered.
The existing solution in the market at present is to embed an RFID chip in tableware, mark the chip by an RFID reader-writer before use, put the chip into a fixed area at a corresponding cash-receiving and settlement desk after a consumer takes the meal, and the RFID reader-writer in the settlement desk can read the mark of the tableware, thereby achieving the purposes of identification and collection. However, the tableware cost of the method is high, the chip fails to work and is leaked, and the tableware needs to be frequently marked when the dishes are changed, which brings troubles to workers.
Disclosure of Invention
In view of the above, the present invention is directed to a machine learning-based intelligent settlement system, so as to solve the above-mentioned problems in the background art.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
an intelligent settlement system based on machine learning comprises an intelligent menu planning platform, an intelligent product output machine and an AI visual settlement table; the intelligent menu planning platform is connected with the intelligent product output machine and the AI visual settlement table;
the intelligent product output machine is used for recording dishes and output quantity;
the intelligent menu plan platform is used for generating a menu plan according to the intelligent product output machine;
the AI visual settlement table is used for settlement of food.
Furthermore, intelligence goes out article machine and includes the dining groove put table of product machine body and body upper end, goes out article machine body side and passes through the connecting rod and connect the camera, stretches into the camera directly over the dining groove put table for shoot the dining ware in the dining groove, the fixed touch display screen in intermediate position of connecting rod for show that the camera shoots the content and operate the product, the lower part of dining groove put table is equipped with intelligent electronic scale, is used for weighing the weight of dining groove and the contained food.
Further, the AI visual settlement table comprises a settlement table body, an intelligent industrial personal computer, a sound device and a cooling fan, wherein the intelligent industrial personal computer, the sound device and the cooling fan are embedded in the settlement table body, the four corners below the settlement table body are provided with universal wheels with adjustable height, a table making table is placed above the universal wheels, the left side of the table making table is printed with an identification area, a frosted sticking film for preventing reflection is stuck on the identification area, the right side of the frosted sticking film is printed with a waiting area, one side of the table facing a diner is fixedly provided with a vertical display screen for displaying the identified dinning, the amount of money and the settlement state, the uppermost end of the display screen shell is provided with a lampshade, the lampshade is internally provided with a binocular camera and double rows of LED lamp tubes, the camera and the lamp tubes are extended into the position right above the table identification area for shooting and supplementing the dinning in the identification area, the display screen is also provided with a thermal imaging thermometer for measuring the temperature of a human body, for monitoring whether a dinner plate is placed in the identification area.
The invention also provides a settlement method of the intelligent settlement system based on machine learning, which comprises the following steps:
(1) information entry: the intelligent menu plan platform stores preset information required by generating a menu plan to a server in advance, wherein the preset information comprises basic data, food material information, item information and tableware information;
(2) and (3) generating a menu: setting required conditions, and generating a reasonable menu plan by an intelligent menu plan platform through an algorithm according to the set conditions and daily acquired data;
(3) intelligent settlement: the diner selects the contained meal, the tray is placed in the identification area at an AI visual settlement table, the settlement table collects and identifies the image information of the identification area, and the identification result is displayed;
(4) and selecting a payment mode and carrying out body temperature measurement.
Further, the step (3) specifically includes the following steps:
(31) acquiring a dinner plate image, separating a dinner plate from a background, and identifying the color of the dinner plate;
(32) acquiring the outline edge of the dinner plate and identifying the shape of the dinner plate;
(33) and associating each dinner plate type with preset price information, and calculating to obtain the amount of money to be paid.
Further, in the step (32), an improved Canny algorithm is adopted to carry out edge detection to obtain the edge of the dinner plate contour, a selective fuzzy algorithm with a function of reserving the edge is used to replace a Gauss filter to carry out smoothing filtering, and only the gray difference between the central pixel point and the gray difference between the central.
Further, in the step (32), a dinner plate shape recognition algorithm based on geometric features is adopted to recognize the dinner plate shape, and specifically:
after the edge of the dinner plate is obtained, the geometrical characteristics of the dinner plate are extracted, and the following formula is the circularity, wherein area is the area, and perimeter is the perimeter:
Metric＝4π×area/perimeter
the following eleven-point method finds the discrete curvature formula, p1... pi is the point on the contour, ki is the approximate curvature:
ri1＝pi-pi-5
ri2＝pi+5-pi
ki＝sign((xi-xi-5)(yi+5-yi)-(yi-yi-5)(xi+5-xi))×ri1ri2 /||ri1||||ri2||
and then, detecting contour parameters by using Hough transformation, and traversing and comparing the geometrical shape characteristics of the dinner plate obtained by the Hough transformation with a template in a database to identify the dinner plate.
Further, the method comprises the step of identifying the dinner plate under the condition of the occlusion by using a method of fitting the complete contour.
Further, the method also comprises the step of correcting errors caused by the distance between the tableware and the collecting equipment by using binocular stereo vision.
Further, the AI visual settlement table further comprises an image recognition training unit for recognizing the meal through AI machine learning, so as to count the sales of the meal, and to push a nutrition map and an intake suggestion to the eater, and specifically comprises the following steps:
s1, collecting sample data of the dish;
s2, detecting the dish image collected by the camera by using the trained model;
and S3, combining the dish detection result with the tableware detection result, and performing mutual verification to obtain a final result.
Compared with the prior art, the intelligent settlement system based on machine learning has the following advantages:
(1) according to the invention, from the arrangement of a menu plan, the big data analysis is carried out through the data collected by the product output machine and the AI visual settlement table, so that reasonable menu recommendation is intelligently generated for the user, manual menu making is also avoided, the diversity of the menu is more comprehensive, and manpower and material resources are saved;
(2) the invention collects money through the AI visual settlement table, greatly improves the dining experience of diners and the settlement speed of cashiers, and can detect the body temperature of diners through temperature measurement, thereby realizing artificial intelligence.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:
FIG. 1 is a schematic front view of an AI visual checkout station according to an embodiment of the invention;
FIG. 2 is a schematic top view of an AI visual settlement table according to an embodiment of the invention;
fig. 3 is a schematic side view of an AI visual settlement table according to an embodiment of the invention.
Description of reference numerals:
1-a table; 2-a display screen; 3-a lampshade; 4-binocular camera; 5-lamp tube.
Detailed Description
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used only for convenience in describing the present invention and for simplicity in description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present invention. Furthermore, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first," "second," etc. may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless otherwise specified.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art through specific situations.
The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
The invention provides an intelligent settlement system for self-selected fast food, which has the characteristics of convenience, rapidness, accuracy and the like. The invention relates to an intelligent settlement system for self-selection fast food, which comprises an intelligent menu plan platform, an intelligent product output machine and an AI visual settlement table; the intelligent menu plan platform is in butt joint with the intelligent product output machine and is further connected with the AI visual settlement table.
In order to enable the intelligent menu planning platform to generate menus, in the invention, the intelligent menu planning platform needs to store preset information required by generating a menu plan to a server in advance; the preset information comprises basic data, food material information and item information;
in order to enable the intelligent product discharging machine to record which dishes are and the product discharging amount, the intelligent product discharging machine comprises a product discharging machine body and a meal slot placing table embedded at the upper end of the product discharging machine body, the side face of the product discharging machine body is connected with a camera through a connecting rod, and the camera is inserted into the position right above the meal slot placing table and used for shooting the meals in the meal slot. A touch display screen is fixed in the middle of the connecting rod through screws and used for displaying shooting contents of the camera and operating products. The lower part of the dining groove placing table is inlaid with an intelligent electronic scale for weighing the weight of the dining groove and the contained food.
In order to enable the AI visual settlement table to settle accounts conveniently and quickly, the AI visual settlement table comprises a settlement table body, and an intelligent industrial personal computer, a sound device and a cooling fan which are embedded in the settlement table body. But settlement platform body below four corners is equipped with height-adjusting's universal wheel, and the user can adjust the bearing height of universal wheel through the screw hole on the adjusting bolt cooperation runing rest to reach the purpose of overall adjustment desktop height, conveniently be applicable to special height environment such as primary school. Stainless steel table platform has been placed to the top, and the desktop of table platform uses glass to make, has printed "identification area" in the left side of glass desktop, pastes the dull polish pad pasting of anti-light-reflection in identification area, and the right side is printed "waiting area" typeface, and the screw is fixed on the body for the table platform. The table top is fixedly provided with a vertical display screen through screws on one side facing diners for displaying the dinning items to be identified and the amount of money and the settlement state by customers, the display screen is wrapped by a metal shell, a lampshade is fixed at the top end of the outer shell through screws and glue, a binocular camera and double rows of LED lamp tubes are embedded in the lampshade, the camera and the lamp tubes are inserted right above the table top identification area for shooting and supplementing light to the dinning items in the identification area. The fixable sliding rail is mounted on one side, connected with the guest display auxiliary screen structure, of the table top, and can be lifted and adjusted after screws are loosened so as to adjust the heights of the binocular camera, the guest display auxiliary screen and the face recognition camera, and only the screws need to be screwed after the height of the guest display auxiliary screen and the face recognition camera are adjusted to a proper height.
In order to use various consumption modes, in the stainless steel table, a smart card reader and a code scanning box are embedded in the left side of an identification area for card-swiping settlement and two-dimensional code payment.
In order to carry out face payment, a camera is embedded in the logo position in the center of the upper end of the display screen shell and used for shooting face payment.
A thermal imaging thermometer is embedded in the parallel position of the face payment camera, body temperature measurement can be carried out while face payment is carried out, once the temperature exceeds 37 ℃, the system can warn, and personnel data with abnormal body temperature are uploaded to a background system to form an early warning report.
In order to check the arrangement state of the dinner plate, an infrared sensor is embedded in the lower end of the display screen shell and used for monitoring whether the dinner plate is arranged in the identification area or not.
In order to enable a cashier to operate, in the invention, a touch display screen is fixed on one side surface of the stainless steel desk close to the card reader and the code scanning box through screws, and the cashier can check meals and money in the identification area through the display screen and can perform ordering operation.
The invention also provides an implementation method of the intelligent settlement system for identifying tableware, which comprises the following steps:
step 1: information entry: the intelligent menu plan platform needs to store preset information required by generating a menu plan to a server in advance; the preset information comprises basic data, food material information, item information and tableware information;
And 3, the cook makes food according to the generated menu plan, and after one food is made, the food needs to be weighed and subjected to image recognition on a product output machine, the food and the weight of the product are recorded, and then the food can be sold.
And 4, selecting the contained food by diners and putting the food into the tray. And then to an AI visual settlement station to place the tray in the identification area. The infrared sensor detects that an object enters the recognition area, then the camera at the top is started, image information of the recognition area is collected and recognized, recognized results are displayed on the customer display auxiliary screen and the cashier touch screen respectively, and corresponding voice prompt is carried out.
The diner can select various payment modes, the diner can directly put the diner card in the card reading area by selecting card swiping payment, and the card reader in the table can automatically read the diner card and deduct the fee; if code scanning payment is selected, the two-dimensional code of the WeChat, Payment treasure or electronic meal card on the mobile phone is aligned to the code scanning area, and a code scanning box in the table can automatically read the two-dimensional code and deduct the fee; if people's face payment is selected, the people need to stand right in front of the secondary screen of the customer display, a camera at the LOGO position right above the direct-faced customer display can acquire the face information and deduct the fee. The settlement desk displays the corresponding result after settlement on the customer display auxiliary screen and the cashier touch screen, and carries out corresponding voice prompt. The body temperature is measured while the human face is paid, once the temperature exceeds 37 ℃, the system can warn and upload the personnel data with abnormal body temperature to the background system to form an early warning report.
In step 4 of the invention, the first step of the dinner plate identification is image segmentation, the dinner plate and the background are separated for the next identification, and the algorithm carries out image segmentation by using a dynamic threshold value method.
The principle of thresholding segmentation is that a pixel point set is divided according to the gray value of an image, and the pixel points are classified by setting a threshold.
When detecting the dinner plate, because the shape problem of dinner plate itself, splendid attire dish scheduling problem in the dinner plate use fixed threshold value method to cause the image segmentation failure easily, lead to the dinner plate to omit, for solving this problem, this algorithm adopts a dynamic threshold value method based on morphology to detect.
The principle of the dynamic threshold method is that under the global optimal condition, a global threshold is determined first, then the value of the pixel points around the position of the pixel point to be detected is used for adjusting the global threshold, and finally the obtained threshold is the dynamic threshold.
The dynamic threshold method has the advantages that the threshold of each pixel point is not fixed and is determined by the global threshold and the values of the surrounding neighborhood pixels, and the threshold can be automatically adjusted in different illumination areas under the condition of unbalanced brightness of the whole image, so that the robustness of the algorithm is improved.
The formula of the dynamic threshold method is as follows, where U1 is a global threshold, U2 is a feature quantity of surrounding pixels with a to-be-detected pixel point as a center, and a is an adjustment rate, then the dynamic threshold is as follows:
U＝(1-a)U1+aU2
the dynamic threshold method combines the advantages of a statistical domain segmentation algorithm and a classical template algorithm, can give consideration to the whole and the details, and can improve the accuracy of edge detection;
the invention supports the identification of dinner plates with various colors, and the dinner plates can be preliminarily classified by distinguishing the colors, thereby being beneficial to further identification.
One of the most commonly used color models in image recognition is an RGB color model, wherein R represents red, G represents green, and B represents blue, and the three colors are mixed in different proportions to express various colors; another common color model is the HSV color model, where H represents hue, S represents saturation, and V represents brightness, which is more intuitive to the human eye than the RGB color model.
The image collected by the camera is RGB color, the RGB color model is easily affected by factors such as illumination and shadow, and the HSV color model can be measured by using saturation and brightness components, so that the RGB color model is not easily affected by the factors and is more stable, and the RGB color is converted into HSV color to be processed in the next step.
The formula for converting RGB color into HSV color is as follows, where r, g, b are red, green, blue in RGB color space, h, s, v are hue, saturation, brightness in HSV color space:
k1＝max{r，g，b}，k2＝min{r，g，b}
(k1-k2)/k1,k1≠0
undefined,s＝0
and converting the RGB image into an HSV image, carrying out channel separation on the obtained HSV image to obtain three single-channel images, wherein the numerical value of an H channel representing the tone can be used for distinguishing different colors, the S, V channel can assist in judging the shade and the brightness of the color, and the color of the dinner plate can be identified by setting different thresholds and utilizing the dynamic threshold algorithm.
And (4) carrying out edge detection on the image obtained by the processing by using a Canny algorithm to obtain the edge of the dinner plate outline.
The Canny algorithm is based on the principle that a certain Gauss filter is selected for smooth filtering of an image to remove image noise, the gradient strength and the direction of each pixel point in the image are calculated, then a non-maximum suppression method is adopted to refine an image gradient amplitude matrix and search for pixel points which may be edges in the image, and finally a double-threshold detection method is used to search for edge points through double-threshold recursion to suppress isolated weak edges and realize edge detection.
The algorithm is improved from the traditional Canny algorithm, a selective fuzzy algorithm with a function of reserving edges is used for replacing a Gauss filter to carry out smooth filtering, the principle of the selective fuzzy algorithm is that all pixel points are not allowed to participate in smooth calculation, a threshold value is set, only the gray difference between the pixel points and the central pixel point is allowed to participate in the smooth calculation, and therefore the pixel with the larger difference between the pixel points and the central pixel value is considered to be effective, not noise, and not to participate in the smooth calculation, so that effective information is reserved.
After the dish edges are obtained, the geometrical features of the dish are extracted.
The circularity formula is as follows, where area is the area and perimeter is the perimeter:
Metric＝4π×area/perimeter
the following eleven-point method finds the discrete curvature formula, p1... pi is the point on the contour, ki is the approximate curvature:
ri1＝pi-pi-5
ri2＝pi+5-pi
ki＝sign((xi-xi-5)(yi+5-yi)-(yi-yi-5)(xi+5-xi))×ri1ri2 /||ri1||||ri2||
and then detecting contour parameters by using Hough transformation, wherein the principle of the Hough transformation is to transform points on the contour to a group of parameter spaces, and find a corresponding solution of a maximum value according to an accumulated value of the points of the parameter spaces, so that the parameters of the geometric shape of the contour can be obtained.
The Hough transform formula is as follows, wherein (a, b) is an arbitrary point on the contour, which is taken as a reference point, phi is the tangential direction, r is the offset vector from the tangential direction to the position of the reference point, and alpha is the included angle between r and the x axis:
a＝x+r(φ)cos(α(φ))
b＝x+r(φ)sin(α(φ))
after the geometrical shape characteristics of the dinner plate are obtained through Hough transformation, traversal comparison is carried out on the geometrical shape characteristics and the template in the database, and the dinner plate can be identified.
In the actual use process, the placing positions of the dinner plates cannot be controlled, the dinner plates are overlapped and shielded possibly, and for the situation, the dinner plates can be identified after being processed by a method for fitting and completing the outline.
Firstly, the dinner plate contour is extracted by the method, the obtained incomplete contour is obtained, and then the DLS algorithm is used for fitting the incomplete dinner plate contour.
The principle of the DLS algorithm is that a matching function of known data is found through the square sum minimum value of errors between actual data and obtained data, and an unknown value is obtained through a measured value.
The following is the DLS algorithm formula:
xDLS＝argminx||△A||22subject to b∈Range(A+△A)
in order to reduce errors, the algorithm carries out segmentation iterative fitting on the obtained contour, and accumulates results obtained by the segmentation fitting, so that interference influence is reduced, and a correct contour is obtained.
After the complete and correct outline is obtained, the dinner plate can be identified by using the dinner plate identification algorithm.
The invention also comprises the steps of preprocessing the collected image, carrying out rotation gray scale, Gaussian filtering smoothing, dynamic threshold value binaryzation, extracting edges, searching the center point of the tableware and extracting the tableware area.
Because the height of the tableware from the camera can influence the size of the tableware, binocular stereoscopic vision is added to correct errors caused by the distance.
In the monocular view a, the distance between the points is not visible, and in addition to the view B, the camera A, B can determine the distance between the three points.
Binocular detection principle:
by calculating the parallax of the two images, the distance measurement is directly performed on the front scene (the range where the images are shot) without judging what type of obstacle appears in front. Therefore, for any type of obstacles, necessary early warning or braking can be carried out according to the change of the distance information. The principle of a binocular camera is similar to that of the human eye. Human eyes can perceive the distance of an object because the images of the same object presented by the two eyes are different, which is also called as parallax. The farther the object distance is, the smaller the parallax error is; conversely, the greater the parallax. The magnitude of parallax corresponds to the distance between the object and the eyes, which is also the reason why the 3D movie enables the person to have a stereoscopic layered perception.
Firstly, a series of operations such as calibration of a binocular camera and the like comprise the following steps:
1. correcting the left camera and the right camera respectively to obtain their internal reference and distortion coefficient
2. Finding the coordinates of the inner corners of the left and right calibration graphs, adding the three-dimensional world coordinates, and adding the above parameters and distortion coefficients of the left and right cameras, to obtain a dual-purpose calibration parameter matrix R, T, E, F (rotation matrix, translation matrix, eigen matrix, basis matrix)
3. According to the above calibration result, correcting the image, and constraining polar line, wherein the two corrected images are in the same plane and parallel to each other
4. And performing pixel matching on the two corrected images to obtain a disparity map, and calculating the depth of each image to obtain a depth map.
Point lO and point rO are the imaging focal points of the left and right cameras, respectively, i.e., the origins of the left and right camera coordinate systems; spatial arbitrary point p (X)W,YW,ZW) The point projected on both the left and right images is pl(xl，yl) And pr (xr，yr) Finding p (X) by stereo matching algorithmW,YW,ZW) And point pl(xl，yl) And pr(xr， yr) The relationship between them.
By the nature of the projection principle, there are:
then
Where f is the focal length of the left and right cameras and T representsIs the distance of two parallel binocular cameras. Here, point p (X)W,YW,ZW) With olThe left camera imaging coordinate system is a coordinate system; the imaging coordinate system of the left camera is a coordinate system; the relationship between the projection points of the left and right images and the spatial midpoint is expressed by expression (2-2).
And then the distance of the tableware is calculated by a binocular distance measurement formula:
Disparity＝d＝xl-xr
and finally, calculating the distance Z according to a formula:
zoom factor-measured actual distance/standard distance-derived
The actual size of the tableware is C ═ C1P, thereby achieving the effect that the corrected distance influences the tableware identification result.
The invention solves the problem of counting the price per single bill of a diner by identifying the tableware technology, thereby enabling the price of the food to be more flexibly set and managed; and then carrying out image recognition training through AI machine learning to recognize the food so as to achieve the purposes of counting the food sales, pushing a nutrition map to a eater, ingesting suggestions and the like.
The machine learning method comprises the following steps:
s1, collecting sample data of the dish;
s2, detecting the dish image collected by the camera by using the trained model;
s3, combining the dish detection result with the tableware detection result, and performing mutual verification to obtain a final result;
step S1, the collecting a dish sample includes:
acquiring a dish picture and labeling dish data through a camera;
if the number of samples is less, data enhancement is carried out, such as Gaussian noise and salt and pepper noise addition, turnover rotation, random brightness adjustment, radiation transformation and the like;
the SSD network training dish model used in step S2 includes:
defining parameters of the SSD;
constructing a whole network;
matching prior frames;
defining a loss function;
optimizing a training process, comprising:
the learning rate is increased, and the Batch Size is enlarged, so that the variance is reduced;
the method includes the following steps that according to the Warm up, a plurality of epochs are trained firstly with small learning rate, and therefore unstable initialization of network parameters due to large learning rate is prevented;
a cosine function is adopted in the learning rate attenuation strategy, so that the generalization capability is improved;
using a Label smoothing Label Smooth to soften the original one-hot type Label, so that overfitting can be reduced when loss is calculated;
adjusting and optimizing the network structure, comprising:
ResNet-B, the first convolution layer in 4 stages carries out down-sampling operation and is moved to the third convolution layer, the convolution kernel is larger, and the loss of characteristic information can be reduced;
ResNet-C, replacing one convolution of 7 × 7 with two convolutions of 3 × 3;
ResNet-D, using 1 × 1 convolution layer with stride equal to 1, and adding a p-average pool to make down-sampling, reducing information loss;
the detecting of the dish image using the model at step S3 includes:
1. inputting a dish picture to obtain a prediction frame;
2. sorting all the prediction frames, and discarding invalid prediction frames below a confidence threshold;
3. performing non-maximum value suppression NMS on the prediction frame, taking out a result with the highest confidence coefficient from the overlapped frame, calculating the result and the IOU of the rest amount, and eliminating redundant overlapped frames to obtain an accurate and effective prediction frame;
the mutual verification of the dish use detection result and the tableware detection result in the step S3 includes:
1. calculating the number, position and size of the two results, and when the contact ratio reaches a similarity threshold value, the result is correct;
2. when the dish detection result is more than the tableware detection result, alarming;
the intelligent menu plan, the product output machine and the AI visual settlement table are combined, so that intelligent recommendation and data support are provided for the restaurant, and the cost and the waste are greatly reduced. Compared with the traditional RFID scheme, the current hottest artificial intelligence technology is adopted, and the images of the tableware and the food are collected through the camera, so that the quick settlement can be realized without processing or replacing the tableware; the final scheme is used for settlement through recognizing tableware, and the food recognition is only used for providing food sales, pushing nutrition maps to dinners and taking suggestions, so that the risk of miscount after the food is recognized wrongly is avoided, meanwhile, the data of the food can be relatively accurately reserved, and a theoretical basis is provided for data analysis of a background.
For the AI visual settlement table, the invention judges whether the dinner plate is placed in the identification area through the infrared sensor, and judges whether the dinner plate is blocked more stably and quickly by the shooting identification compared with other similar products.
The system is used for dealing with the new crown epidemic situation, and in order to ensure the safety of diner personnel and facilitate the investigation, the settlement is added, and simultaneously the body temperature measurement is carried out, and the alarm can be carried out after abnormality occurs, and the data of the abnormal diner personnel is uploaded to a background to alarm the manager.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. The utility model provides an intelligence settlement system based on machine learning which characterized in that: the system comprises an intelligent menu planning platform, an intelligent product output machine and an AI visual settlement table; the intelligent menu planning platform is connected with the intelligent product output machine and the AI visual settlement table;
the intelligent product output machine is used for recording dishes and output quantity;
the intelligent menu plan platform is used for generating a menu plan according to the intelligent product output machine;
the AI visual settlement table is used for settlement of food.
2. The intelligent settlement system based on machine learning of claim 1, wherein: the intelligent product discharging machine comprises a product discharging machine body and a meal groove placing table at the upper end of the product discharging machine body, the side face of the product discharging machine body is connected with a camera through a connecting rod, the camera is inserted into the position right above the meal groove placing table and used for shooting meal products in the meal groove, a touch display screen is fixed in the middle of the connecting rod and used for displaying shooting contents of the camera and operating the products, and an intelligent electronic scale is arranged at the lower part of the meal groove placing table and used for weighing the weight of the meal groove and the weight of the contained meal products.
3. The intelligent settlement system based on machine learning of claim 1, wherein: the AI visual settlement table comprises a settlement table body, an intelligent industrial personal computer, a sound device and a cooling fan, wherein the intelligent industrial personal computer, the sound device and the cooling fan are embedded in the settlement table body, the four corners below the settlement table body are provided with universal wheels with adjustable height, a table making table is placed above the universal wheels, the left side of the table making table is printed with an identification area, an anti-reflection frosted adhesive film is adhered to the identification area, the right side of the table making table is printed with a waiting area, one side of the table facing diners is fixedly provided with a vertical display screen for displaying the identified diners, the amount of money and the settlement state in a customer display mode, the uppermost end of the display screen shell is provided with a lampshade, a binocular camera and double rows of LED lamp tubes are arranged in the lampshade, the camera and the lamp tubes are inserted into the position right above the identification area of the table making table for shooting and supplementing light to the diners in the identification area, a thermal imager is also arranged in the display screen, for monitoring whether a dinner plate is placed in the identification area.
4. A settlement method of an intelligent settlement system based on machine learning is characterized in that: the method comprises the following steps:
(1) information entry: the intelligent menu plan platform stores preset information required by generating a menu plan to a server in advance, wherein the preset information comprises basic data, food material information, item information and tableware information;
(2) and (3) generating a menu: setting required conditions, and generating a reasonable menu plan by an intelligent menu plan platform through an algorithm according to the set conditions and daily acquired data;
(3) intelligent settlement: the diner selects the contained meal, the tray is placed in the identification area at an AI visual settlement table, the settlement table collects and identifies the image information of the identification area, and the identification result is displayed;
(4) and selecting a payment mode and carrying out body temperature measurement.
5. The intelligent settlement method based on machine learning of claim 4, wherein: the step (3) specifically comprises the following steps:
(31) acquiring a dinner plate image, separating a dinner plate from a background, and identifying the color of the dinner plate;
(32) acquiring the outline edge of the dinner plate and identifying the shape of the dinner plate;
(33) and associating each dinner plate type with preset price information, and calculating to obtain the amount of money to be paid.
6. The intelligent settlement method based on machine learning of claim 5, wherein: in the step (32), an improved Canny algorithm is adopted to carry out edge detection to obtain the edge of the dinner plate outline, a selective fuzzy algorithm with a function of reserving the edge is used for replacing a Gauss filter to carry out smoothing filtering, and only the gray difference between the central pixel point and the gray difference is allowed to participate in smoothing calculation, so that the pixel with the overlarge difference between.
7. The intelligent settlement method based on machine learning of claim 6, wherein: in the step (32), a dinner plate shape recognition algorithm based on geometric features is adopted to recognize the dinner plate shape, and the method specifically comprises the following steps:
after the edge of the dinner plate is obtained, the geometrical characteristics of the dinner plate are extracted, and the following formula is the circularity, wherein area is the area, and perimeter is the perimeter:
Metric＝4π×area/perimeter
the following eleven-point method finds the discrete curvature formula, p1... pi is the point on the contour, ki is the approximate curvature:
ri1＝pi-pi-5
ri2＝pi+5-pi
ki＝sign((xi-xi-5)(yi+5-yi)-(yi-yi-5)(xi+5-xi))×ri1ri2/||ri1||||ri2||
and then, detecting contour parameters by using Hough transformation, and traversing and comparing the geometrical shape characteristics of the dinner plate obtained by the Hough transformation with a template in a database to identify the dinner plate.
8. The intelligent settlement method based on machine learning of claim 7, wherein: the method also comprises the step of identifying the dinner plate under the condition of the occlusion by using a method of fitting the complete contour.
9. The intelligent settlement method based on machine learning of claim 4, wherein: the method also comprises the step of correcting errors caused by the distance between the tableware and the collecting equipment by using binocular stereo vision.
10. The intelligent settlement method based on machine learning of claim 4, wherein: the AI visual settlement table further comprises image recognition training through AI machine learning, is used for recognizing food, counts the food sales, and pushes nutrition maps and intake suggestions to the eaters, and specifically comprises the following steps:
s1, collecting sample data of the dish;
s2, detecting the dish image collected by the camera by using the trained model;
and S3, combining the dish detection result with the tableware detection result, and performing mutual verification to obtain a final result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010762293.2A CN112037440A (en) | 2020-07-31 | 2020-07-31 | Intelligent settlement system and settlement method based on machine learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010762293.2A CN112037440A (en) | 2020-07-31 | 2020-07-31 | Intelligent settlement system and settlement method based on machine learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112037440A true CN112037440A (en) | 2020-12-04 |
Family
ID=73582137
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010762293.2A Pending CN112037440A (en) | 2020-07-31 | 2020-07-31 | Intelligent settlement system and settlement method based on machine learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112037440A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113077596A (en) * | 2021-03-24 | 2021-07-06 | 盐城市权绪商贸有限公司 | Intelligent control system for assisting self-service supermarket commodity cash register |
| CN113537119A (en) * | 2021-07-28 | 2021-10-22 | 国网河南省电力公司电力科学研究院 | Detection method of transmission line connecting parts based on improved Yolov4-tiny |
| CN113536014A (en) * | 2021-06-30 | 2021-10-22 | 杭州电子科技大学 | A dish information retrieval method integrating container information |
| CN114966363A (en) * | 2022-04-15 | 2022-08-30 | 南通深南电路有限公司 | A plate inspection mechanism and plate inspection system |
| CN116309559A (en) * | 2023-05-17 | 2023-06-23 | 山东鲁玻玻璃科技有限公司 | Intelligent identification method for production flaws of medium borosilicate glass |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106056802A (en) * | 2016-06-06 | 2016-10-26 | 杭州汇萃智能科技有限公司 | Tableware-color-based pricing method of dish automatic identification system |
| CN107992871A (en) * | 2017-12-21 | 2018-05-04 | 陕西伟景机器人科技有限公司 | The automatic accounting method in dining room based on image recognition |
| CN109740579A (en) * | 2019-03-01 | 2019-05-10 | 上海柏珍信息科技有限公司 | A kind of fast food cash register system based on dish intelligent recognition |
| CN109830072A (en) * | 2019-02-26 | 2019-05-31 | 魔珐(上海)信息科技有限公司 | The valuation of view-based access control model identification and cash device, control system and method |
| CN111326233A (en) * | 2020-02-10 | 2020-06-23 | 天津联信电子科技有限公司 | Conditional intelligent menu plan generation method, system, equipment and storage medium |
| CN212647599U (en) * | 2020-07-31 | 2021-03-02 | 天津联信电子科技有限公司 | Intelligent settlement system |
-
2020
- 2020-07-31 CN CN202010762293.2A patent/CN112037440A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106056802A (en) * | 2016-06-06 | 2016-10-26 | 杭州汇萃智能科技有限公司 | Tableware-color-based pricing method of dish automatic identification system |
| CN107992871A (en) * | 2017-12-21 | 2018-05-04 | 陕西伟景机器人科技有限公司 | The automatic accounting method in dining room based on image recognition |
| CN109830072A (en) * | 2019-02-26 | 2019-05-31 | 魔珐(上海)信息科技有限公司 | The valuation of view-based access control model identification and cash device, control system and method |
| CN109740579A (en) * | 2019-03-01 | 2019-05-10 | 上海柏珍信息科技有限公司 | A kind of fast food cash register system based on dish intelligent recognition |
| CN111326233A (en) * | 2020-02-10 | 2020-06-23 | 天津联信电子科技有限公司 | Conditional intelligent menu plan generation method, system, equipment and storage medium |
| CN212647599U (en) * | 2020-07-31 | 2021-03-02 | 天津联信电子科技有限公司 | Intelligent settlement system |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113077596A (en) * | 2021-03-24 | 2021-07-06 | 盐城市权绪商贸有限公司 | Intelligent control system for assisting self-service supermarket commodity cash register |
| CN113536014A (en) * | 2021-06-30 | 2021-10-22 | 杭州电子科技大学 | A dish information retrieval method integrating container information |
| CN113536014B (en) * | 2021-06-30 | 2023-09-01 | 青岛中科英泰商用系统股份有限公司 | Dish information retrieval method integrating container information |
| CN113537119A (en) * | 2021-07-28 | 2021-10-22 | 国网河南省电力公司电力科学研究院 | Detection method of transmission line connecting parts based on improved Yolov4-tiny |
| CN114966363A (en) * | 2022-04-15 | 2022-08-30 | 南通深南电路有限公司 | A plate inspection mechanism and plate inspection system |
| CN116309559A (en) * | 2023-05-17 | 2023-06-23 | 山东鲁玻玻璃科技有限公司 | Intelligent identification method for production flaws of medium borosilicate glass |
| CN116309559B (en) * | 2023-05-17 | 2023-08-04 | 山东鲁玻玻璃科技有限公司 | Intelligent identification method for production flaws of medium borosilicate glass |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112037440A (en) | Intelligent settlement system and settlement method based on machine learning | |
| CN107992871B (en) | Restaurant automatic checkout method based on image recognition | |
| CN113498530B (en) | Object size marking system and method based on local visual information | |
| CN107169405B (en) | Method and device for living body identification based on binocular camera | |
| CN104463167B (en) | A kind of dining room automatic settlement method and system | |
| CN107767590A (en) | Automatic identification commercialization bar code electronic scale and Automatic identification method | |
| CN111080493B (en) | Dish information identification method and device and dish self-service settlement system | |
| CN103914691B (en) | Target group's analysis system, method and height recognition methods based on recognition of face | |
| KR101744123B1 (en) | Apparatus and method for detecting medicinal products | |
| JP2001216571A (en) | Method and device for adjusting charge | |
| CN106503644A (en) | Glasses attribute detection method based on edge projection and color characteristic | |
| CN107038425A (en) | The settlement system of intelligent restaurant based on machine vision | |
| US11562338B2 (en) | Automated point of sale systems and methods | |
| CN104978585A (en) | Automatic pricing method | |
| CN110097540A (en) | The visible detection method and device of polygon workpeace | |
| CN113062421A (en) | Intelligent closestool for health detection and health detection method thereof | |
| CN109146834A (en) | Car damage identification method and device, computer readable storage medium, terminal | |
| US11367266B2 (en) | Image recognition system, image recognition method, and storage medium | |
| CN109215260A (en) | A kind of cafeteria's artificial intelligence settlement system | |
| RU2724797C1 (en) | Cash register system and method for identification of courses on tray | |
| CN108537280A (en) | A kind of bulk commodity charge system based on machine vision | |
| CN111272673A (en) | Non-pure-color cloth color difference detection system and detection method | |
| CN212647599U (en) | Intelligent settlement system | |
| CN110533451A (en) | Intelligent vision pricing system | |
| CN112115902A (en) | Dish identification method based on single-stage target detection algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |