JP2024061292A - Ready-mixed concrete quality prediction system and ready-mixed concrete prediction method - Google Patents

Abstract

To provide a ready-mixed concrete quality prediction system and a ready-mixed concrete quality prediction method that can predict with high accuracy, quality of ready-mixed concrete in an early stage after the start of kneading and mixing.SOLUTION: A ready-mixed concrete quality prediction system comprises: a storage unit that stores a learned model; an input unit that receives input of input data for prediction; and an output unit that outputs an output result. Training data is data in which input data for learning based on a learning data group including image extraction information and time-series information extracted from a plurality of pieces of image data acquired by an imaging unit photographing ready-mixed concrete being kneaded and mixed and different in photographing time, and output data for learning including quality prediction information are associated with each other. The plurality of pieces of image data of ready-mixed concrete to be predicted is input as the input data for prediction. The quality prediction information of the ready-mixed concrete to be predicted is output as the output result.SELECTED DRAWING: Figure 2

Description

The present invention relates to a system and method for predicting the quality of ready-mix concrete.

The required properties of ready-mix concrete vary depending on the purpose of use and the environment of the site where it is used. For this reason, in the conventional manufacturing process for ready-mix concrete, the mixed state was checked visually based on intuition and experience to ensure that the ready-mix concrete produced met the required properties before it was shipped.

One of the important characteristics related to the quality of ready-mix concrete is the slump value. The slump value is a value used as an index showing the consistency of concrete before hardening, and generally, when ready-mix concrete is shipped or unloaded, it is evaluated and inspected to see if the slump value of the ready-mix concrete is within the required numerical range.

The evaluation and inspection results of the slump value of ready-mixed concrete are the standard for judging whether the manufactured ready-mixed concrete can be shipped and whether it can be used at the unloading site. For this reason, it is expected that the slump value of ready-mixed concrete can be predicted with high accuracy, as much as possible, before it is shipped from the manufacturing plant, independent of the worker's senses and experience.

In recent years, therefore, in order to predict the slump value of ready-mixed concrete with higher accuracy, a method has been proposed for predicting the slump value of ready-mixed concrete using a trained model generated by machine learning that applies training data that associates image data of the ready-mixed concrete being mixed in the mixer with the slump value (see Patent Document 1 below).

JP 2020-144132 A

The prediction method described in Patent Document 1 can predict the slump value of fresh concrete with a sufficiently high degree of accuracy. However, this method predicts the slump value based on image data obtained by photographing the fresh concrete in a state where the various ingredients have been stirred and thoroughly mixed, approximately 1 to 5 minutes after mixing has begun. The main reason for this is that for a while after mixing has begun, the ingredients are not sufficiently mixed, and the state of the fresh concrete changes from moment to moment, making it difficult to stably predict the slump value.

It takes about 1 to 5 minutes from the start of mixing for the ingredients to be sufficiently mixed, so if the prediction result indicates that the ready-mixed concrete will not achieve the desired slump value, several minutes of the mixing process may be wasted. In some cases, this type of response can result in tens of minutes or even hours of wasted time, which can pose a major risk to the stabilization of quality in the production of ready-mixed concrete.

The prediction method described in Patent Document 1 can predict the slump value of fresh concrete with a sufficiently high degree of accuracy by evaluating each image one by one. However, this method uses prediction results obtained from each image, and it is more desirable to be able to make predictions that are close to the sense of a skilled technician, since the predictions are made from video in the evaluation of an actual skilled technician.

In order to avoid these risks as mentioned above and to further improve production efficiency, it is hoped that it will be possible to predict quality-related indicators based on the fluidity of fresh concrete, such as the slump value, or other indicators related to the quality of fresh concrete with high accuracy early after mixing begins, without having to wait until the materials are fully mixed.

In view of the above problems, the present invention aims to provide a ready-mixed concrete quality prediction system and a ready-mixed concrete quality prediction method that can predict the quality of ready-mixed concrete with high accuracy early after mixing begins.

The quality prediction system for ready-mixed concrete of the present invention comprises:
A storage unit that stores a trained model generated by machine learning based on teacher data;
an input unit that accepts input of prediction input data;
An output unit that outputs an output result derived by the trained model,
The teacher data is data in which learning input data based on a learning data group including image extraction information, which is information related to the state of the fresh concrete during mixing, and time-series information, which is information related to state changes over time, extracted from a plurality of image data captured at different times by an imaging unit that captures the fresh concrete being mixed, is associated with learning output data including quality prediction information, which is information related to the quality of the fresh concrete after mixing,
As the prediction input data, a plurality of image data of the ready-mixed concrete to be predicted is input,
The quality prediction information of the ready-mix concrete to be predicted is output as the output result.

The image data applied as the learning input data and prediction input data of the quality prediction system may be data acquired by a camera as is, or may be image data of one frame cut out from video data acquired by shooting a video with a video camera. The "multiple image data captured at different times" may be multiple image data acquired by taking photographs at different times, multiple image data acquired by extracting frame images at different times from the acquired video data, or multiple image data acquired by taking snapshots at different times while playing the video data. Depending on the processing power of the hardware constituting the system, the "multiple image data captured at different times" may naturally be each image data of each frame that constitutes the video data.

The trained model used in the quality prediction system of the present invention is created by machine learning, and the learning method used in the machine learning is described in detail in the section "Form for implementing the invention."

Image extraction information is information extracted based on multiple image data, such as the amplitude of the surface of the fresh concrete and the depth to which the fresh concrete sinks due to the movement of the mixer blades.

Time-series information is information based on multiple image data taken at different times, such as the amount of change in the amplitude of the surface of the fresh concrete over time during the mixing process, or the amount of change in the depth to which the fresh concrete sinks due to the movement of the mixer blades.

As mentioned above, conventional quality prediction methods could not make stable predictions until the changes in the state of fresh concrete over time during the mixing process had become small. Therefore, the inventors focused on associating the changes in the state of fresh concrete over time with its quality, and came up with the idea for the above quality prediction system.

Time series information is thought to contribute to predicting the quality of fresh concrete as well as understanding the mixing state of fresh concrete. In other words, time series information can also be used to determine at what time image data should be acquired in order to extract image extraction information.

As described above, the quality prediction system configured as above can predict the quality of ready-mixed concrete with higher accuracy by using image extraction information and time series information. Furthermore, the time series information can be used to predict how the state of the ready-mixed concrete will change as the mixing process progresses. Therefore, the quality prediction system configured as above can predict the state when the ready-mixed concrete is sufficiently mixed, and can predict the quality of the ready-mixed concrete, even when the ready-mixed concrete is not sufficiently mixed.

In the above quality prediction system,
The learning input data and the prediction input data may be data based on the image extraction information extracted from a plurality of image data including image data of the state in which the blades of a mixer that mixes fresh concrete are scooping up the fresh concrete, and the learning data group including the time series information.

In the above quality prediction system,
The learning input data and the prediction input data may be data based on the image extraction information extracted from a plurality of image data including image data of the mixer blades submerged in fresh concrete and image data of the mixer blades scooping up fresh concrete, and the learning data group including the time series information.

The above configuration makes it easier to obtain feature quantities based on time-series information used to predict the state of fresh concrete, such as the amplitude of the surface of the fresh concrete and the depth to which the fresh concrete sinks due to the movement of the mixer blades. Therefore, the quality prediction system of the above configuration improves the prediction accuracy based on time-series information, further improving the quality prediction accuracy and enabling quality prediction at an earlier stage.

In the above quality prediction system,
The learning input data and the prediction input data may be data based on the learning data group including the image extraction information extracted from image data acquired by an imaging unit that photographs the fresh concrete being mixed during the period from the start of mixing the fresh concrete to the time the mixed fresh concrete is discharged from the mixer, and the time series information.

In the above quality prediction system,
The learning input data and the prediction input data may be data based on the learning data group that further includes mix-related information or production-related information for ready-mix concrete.

The environmental parameters inside the container in which the ready-mix concrete is mixed, the mix-related information for ready-mix concrete, and the manufacturing-related information are described in detail in the section "Form for carrying out the invention," along with image extraction information, etc.

The quality prediction system configured as described above can predict indicators related to the quality of ready-mix concrete based on further information in addition to image extraction information and time series information, thereby further improving prediction accuracy.

The method for predicting the quality of ready-mixed concrete of the present invention comprises the steps of:
a storage unit in which a trained model generated by machine learning based on teacher data in which training input data based on a training data group including image extraction information, which is information related to the state of the fresh concrete during mixing, and time-series information, which is information related to state changes over time, extracted from a plurality of image data captured at different times by an imaging unit that captures the fresh concrete being mixed, and training output data including quality prediction information, which is information related to the quality of the fresh concrete after being mixed, are associated, is stored;
A step (A) of inputting a plurality of image data of a prediction target fresh concrete being mixed as prediction input data into the storage unit, and applying the plurality of image data to the trained model;
and (B) acquiring the quality prediction information of the fresh concrete to be predicted, which is derived by the trained model.

The present invention provides a ready-mix concrete quality prediction system and a ready-mix concrete quality prediction method that can predict the quality of ready-mix concrete with high accuracy early after mixing begins.

FIG. 1 is a schematic overall configuration diagram of one embodiment of a quality prediction system for ready-mix concrete. FIG. 2 is a block diagram illustrating a configuration of a main server according to an embodiment. 2 is a diagram illustrating a schematic view of a part of a mixer. 1 is a graph plotting the results of a verification experiment. 1 is a diagram illustrating seven pieces of image data acquired under specified conditions. 1 is a diagram illustrating seven pieces of image data acquired under specified conditions. 3 is a diagram illustrating 30 pieces of image data acquired under specified conditions. 1 is a graph plotting the results of a verification experiment.

The ready-mix concrete quality prediction system of the present invention will be described below with reference to the drawings. Note that each of the drawings is a schematic illustration, and the numbers on the drawings do not necessarily correspond to the actual numbers.

The ready-mix concrete quality prediction system of the present invention is a system that predicts the quality of ready-mix concrete using a trained model generated by machine learning based on training data that associates training input data including image extraction information extracted from multiple image data captured at different times (described below) and time series information with training output data including quality prediction information for ready-mix concrete.

First, we will explain the details of the image extraction information and time series information, and the relationship between this information and the quality of ready-mix concrete. Note that the learning input data may also include mix-related information and manufacturing-related information, which will also be explained here.

[Image extraction information]
The image extraction information is information extracted from image data of the fresh concrete being mixed in the mixer, and is used to grasp the viscosity, water content, etc. of the fresh concrete, and is also an element for predicting the state of the fresh concrete after a predetermined time has passed. Therefore, quality prediction information related to the predicted value of the quality of the fresh concrete can be associated with the image extraction information.

Therefore, by applying a trained model for predicting the quality of ready-mix concrete generated by machine learning using learning input data including image extraction information and learning output data including quality prediction information, it is possible to predict the quality of the ready-mix concrete to be produced.

Specific examples of image extraction information include the amplitude of the surface of the fresh concrete and the depth to which the fresh concrete sinks due to the mixer blades. The image extraction information applied to the trained model may be one of the above image extraction information or a combination of two or more types.

[Time series information]
The time series information is image extraction information based on multiple image data acquired by photographing the ready-mixed concrete in the mixer at different times, and information on the amount of change over time related to the image extraction information, and when combined with the image extraction information, it becomes an element for predicting the state of ready-mixed concrete after a predetermined time has passed. Therefore, quality prediction information related to the predicted value of the quality of ready-mixed concrete can be associated with the time series information. Furthermore, the time series information may include information on the change in the state of ready-mixed concrete from the start of mixing to the time when sufficient mixing is performed. Therefore, by combining the image extraction information and the time series information, it is possible to associate image extraction information at an early stage after the start of mixing with image extraction information of ready-mixed concrete in a sufficiently mixed state.

Therefore, by applying a trained model for predicting the quality of ready-mix concrete generated by machine learning using learning input data including image extraction information and time series information, and learning output data including quality prediction information, it is possible to predict the quality of the ready-mix concrete to be produced.

Specific examples of time series information include the amount of change in the amplitude of the surface of fresh concrete over time, and the amount of change in the depth to which fresh concrete sinks due to the movement of the mixer blades. The time series information applied to the trained model may be one of the above time series information, or a combination of two or more types. The time series information applied to the trained model may be information about features extracted from multiple image data captured at different times, and different from the image extraction information applied to the trained model.

[Formulation related information]
Information related to the mix proportions of ready-mixed concrete is information that contributes to the speed at which the hydration reaction progresses in the ready-mixed concrete being produced, and is a factor in predicting what changes will occur in the fluidity, etc. of the ready-mixed concrete and how much slump loss will occur over time.

Therefore, by applying a trained model for predicting the quality of ready-mix concrete generated by machine learning using training input data including mix-related information and training output data including quality prediction information, it is possible to predict the quality of the ready-mix concrete to be produced with greater accuracy.

Specific examples of the mix-related information include the water-cement ratio (W/C), cement type, admixture type, admixture amount, and unit water volume. The mix-related information applied to the trained model may be one of the above mix-related information, or a combination of two or more of them.

[Manufacturing related information]
The information related to the production of ready-mixed concrete is an element that can confirm the hardness and fluidity of the ready-mixed concrete during mixing, and is an element for predicting the quality of ready-mixed concrete. In addition, by combining it with the above-mentioned mix proportion information, it can be used as an element for grasping the progress of the hydration reaction, etc.

Therefore, a quality prediction system that applies a trained model generated by training input data including manufacturing-related information and output data including quality prediction information can predict the quality of ready-mix concrete with higher accuracy.

Specific examples of the manufacturing-related information include mixer type, mixer model, mixing time, mixing sound, power load value, ready-mixed concrete temperature during mixing, and temperature and humidity inside the mixer. The manufacturing-related information applied to the trained model may be one of the above manufacturing-related information, or a combination of two or more types.

At least one of the above mix-related information and production-related information may be acquired from an existing information management system in the ready-mix concrete factory. Specifically, the mix-related information may be acquired from a system that manages mix-related information of specified materials that are normally used when shipping ready-mix concrete in an existing ready-mix concrete factory. Furthermore, production-related information such as mixing time, mixing sound, power load value, temperature of ready-mix concrete during mixing, and temperature and humidity inside the mixer may be acquired from a management system that manages the situation when mixing ready-mix concrete. Note that the above information management system is one example of a method for acquiring mix-related information and production-related information, and does not limit the method for acquiring each piece of information.

From the above, a quality prediction system to which a trained model generated by performing machine learning on training input data including image extraction information and time series information, and training output data including quality prediction information is applied, can predict the quality of ready-mix concrete with higher accuracy and earlier from the start of mixing than a quality prediction system to which a trained model generated by performing machine learning on training input data not including time series information is applied.

Furthermore, a quality prediction system that applies a trained model generated by performing machine learning using training input data including mix-related information and manufacturing-related information, and training output data including quality prediction information, can predict the quality of ready-mix concrete with higher accuracy.

The number of training data samples used to train the trained model is preferably 100 or more, more preferably 1,000 or more, even more preferably 10,000 or more, even more preferably 50,000 or more, and particularly preferably 100,000 or more, from the viewpoint of extracting features necessary to derive slump information from the training input data and further improving prediction accuracy. Furthermore, the number of training times for each trained model is preferably 1,000 or more, more preferably 8,000 or more, and particularly preferably 10,000 or more, from the viewpoint of improving prediction accuracy, but is not particularly limited. For example, only one sample with high quality as training data may be adopted as training data.

Next, we will explain the machine learning method for creating the trained model used in the quality prediction system of the present invention. Examples of the machine learning method for creating the trained model used in the quality prediction system of the present invention include neural networks, linear regression, decision trees, support vector regression, ensemble methods, support vector machines, discriminant analysis, naive Bayes methods, nearest neighbor methods, etc. These methods may be used alone or in combination of two or more.

Among these methods, machine learning using a neural network is preferably selected from the viewpoint of being able to predict quality with higher accuracy. From the viewpoint of being able to predict quality with higher accuracy, it is preferable for the neural network to be a hierarchical neural network having one or more intermediate layers between the input layer and the output layer.

Examples of neural networks include convolutional neural networks (CNN) such as 3D Convolutional Neural Networks (3DCNN), deep neural networks (DNN), recurrent neural networks (RNN), and long short-term memory (LSTM) neural networks (recurrent neural networks improved using LSTM).

Among these, a three-dimensional convolutional neural network (a neural network having a convolutional layer, a pooling layer, or the like as an intermediate layer) is more suitable, as it has excellent performance in the field of image recognition and is related to the time axis. A three-dimensional convolutional neural network can detect features (including features that change over time) from multiple image data acquired at different times, and create a prediction model capable of performing classification or regression using the features. In a convolutional neural network, the number of layers consisting of a combination of convolutional layers and pooling layers is preferably two or more, more preferably three or more, from the viewpoint of being able to perform predictions with higher accuracy.

In addition, as a tool for performing machine learning, for example, "TensorFlow (registered trademark)", a software library developed by Google, or "IBM Watson (registered trademark)", a system developed by IBM, can be used.

Next, we will explain the specific configuration of the quality prediction system 1.

The configuration of this embodiment of the ready-mix concrete quality prediction system 1 of the present invention will be described. FIG. 1 is a schematic overall configuration diagram of one embodiment of the ready-mix concrete quality prediction system 1. As shown in FIG. 1, the quality prediction system 1 is composed of a main server 10, a data server 20 connected to the main server 10 via a network, an operation terminal 30, and multiple reference terminals 40.

The number of data servers 20, operation terminals 30, and reference terminals 40 connected to the main server 10 via a network is arbitrary, and each is connected to perform data communication via wire or wirelessly. Also, although not shown in FIG. 1, an imaging unit 60 that captures images of the state of mixing the ready mixed concrete, the subject of quality prediction, is connected to the main server 10 via a network (see FIG. 2).

In this embodiment, the main server 10 starts a prediction operation when the worker operates the operation terminal 30 to start the operation. Then, when prediction input data d1 including image data to be applied to the trained model M1 is input from the imaging unit 60 to the main server 10, a predetermined process is performed within the main server 10, and quality prediction information, which is information related to the predicted value of the quality of ready-mix concrete, is output from the main server 10.

The output data d3 including the quality prediction information output from the main server 10 is transmitted to each reference terminal 40. Then, each reference terminal 40 displays the predicted value of the quality of ready-mix concrete obtained from the output data d3 output from the main server 10 on each display device 50.

In this embodiment, each reference terminal 40 is a desktop PC separate from the display device 50 as shown in FIG. 1, but it may be a terminal in which the display device 50 is integrally configured, such as a notebook PC, a smartphone, or a tablet. Also, each reference terminal 40 may also function as an operation terminal 30.

Figure 2 is a block diagram showing a schematic configuration of the main server 10 in this embodiment. As shown in Figure 2, the main server 10 in this embodiment includes a data input unit 11, a data output unit 12, and a calculation processing unit 13.

Figure 3 is a diagram that shows a schematic of a portion of the mixer 2. As shown in Figure 3, the imaging unit 60 is fixed at a position where the blades 2a of the mixer 2 are captured within the area 60a to be photographed. For ease of explanation, in Figure 3, the area 60a to be photographed by the imaging unit 60 is shown diagrammatically by a dashed line.

In this embodiment, the imaging unit 60 continuously captures images from the start of mixing to the end of mixing, and acquires video data (multiple image data in a time series) with a frame rate of 60 fps (frames per second). Therefore, the video data includes image data acquired by capturing images of the blades 2a of the mixer 2 submerged in the fresh concrete and the blades 2a of the mixer 2 scooping up the fresh concrete. Note that the imaging unit 60 does not have to be a camera that acquires video data as described above, but may be a camera that captures images at a predetermined time interval and acquires multiple image data.

Depending on the rotation speed of the mixer 2, etc., the image extraction information extracted from image data captured while the mixer 2 blades 2a are submerged in the fresh concrete may not contribute much to quality prediction due to the variations in the surface of the fresh concrete. In such cases, the image data acquired by the imaging unit 60 does not need to include image data obtained by capturing an image of the mixer 2 blades 2a submerged in the fresh concrete.

The calculation processing unit 13 and the memory unit 13a are composed of a calculation processing unit such as a CPU or an MPU. The memory unit 13a is composed of a memory such as a flash memory or a hard disk. The memory unit 13a stores a trained model M1 generated by machine learning based on teacher data.

The teacher data is data that associates learning input data, which includes image extraction information extracted from multiple image data captured by the imaging unit 60 and time series information that is information related to changes in the state of the image extraction information over time, with learning output data that includes information related to the quality of ready-mix concrete.

The image extraction information is temporary information about the ready-mix concrete extracted from the image data acquired by the imaging unit 60, and is information related to the state of the ready-mix concrete at the time of image capture.

The information related to the quality of fresh concrete is the quality prediction information of fresh concrete contained in the output data. In this embodiment, the quality prediction information of fresh concrete is a slump value, but other information related to the quality of fresh concrete may be used. The fresh concrete quality prediction system 1 can make sufficient predictions, particularly if the index is related to the consistency or fluidity of fresh concrete, because it is associated with the change in the state of fresh concrete over time during mixing. Furthermore, based on the above association, the fresh concrete quality prediction system 1 can predict other qualities besides the indexes related to the consistency or fluidity of fresh concrete.

The quality prediction information included in the learning output data may be data acquired during pre-shipment quality inspection at a ready-mix concrete manufacturing plant, or may be data acquired during pre-pouring quality inspection at a pouring site. The data to be applied is selected, for example, depending on the point in time at which the quality of ready-mix concrete is to be predicted.

The data input unit 11 in this embodiment accepts input of image data transmitted from the imaging unit 60 as prediction input data d1. The data input unit 11 in this embodiment also accepts prediction input data d1 transmitted from the data server 20 or the operation terminal 30, which includes ready-mix concrete mix-related information and manufacturing-related information for prediction to be applied to the trained model M1.

When the data input unit 11 receives input of prediction input data d1, it generates calculation input data d2 to be input to the calculation processing unit 13 and outputs it to the calculation processing unit 13.

When the data output unit 12 receives the output data d3 output from the calculation processing unit 13, it generates transmission data d4 that can be received by the reference terminals 40 based on the output data d3, as shown in FIG. 2, and transmits the transmission data d4 to each reference terminal 40.

When the calculation processing unit 13 receives the calculation input data d2 output from the data input unit 11, it reads out the trained model M1 recorded in the memory unit 13a and applies the calculation input data d2 to the trained model M1.

The calculation processing unit 13 executes calculation processing using the trained model M1, and outputs output data d3 including quality prediction information derived based on the calculation input data d2 to the data output unit 12.

[Verification Experiment 1]
Here, we will explain the verification experiment that confirmed the accuracy with which the quality prediction system 1, which applies the trained model M1 trained based on the above information, can predict the slump value.

Example 1
In Example 1, the slump value of the produced concrete immediately after mixing is predicted by the above-mentioned ready-mixed concrete quality prediction system 1. Specifically, in Example 1, cement, fine aggregate (mountain sand), coarse aggregate A (crushed stone No. 5), and coarse aggregate B (crushed stone No. 6) are charged into a twin-shaft mixer and dry mixed, and then water is charged and mixed to produce ready-mix concretes 1 to 100, and the slump value of the concrete is predicted. The unit amounts of each material are adjusted to 360 to 600 kg/m ³ for cement, 150 to 165 kg/m ³ for fine aggregate (mountain sand), kg/m ³ for coarse aggregate A (crushed stone No. 5), and kg/m ³ for coarse aggregate B (crushed stone No. 6).

(Comparative Example 1)
Comparative Example 1 is similar to Example 1, except that learning input data not including time series information was applied and a trained model trained by applying a two-dimensional convolutional neural network (2D-CNN) was used.

The timing for inputting prediction input data to predict the quality of fresh concrete can be the period from the start of mixing the fresh concrete to the time the mixed fresh concrete is discharged from the mixer. The start of mixing refers to the state when the ingredients for the fresh concrete other than water are added to the mixer and the mixer starts rotating.

In particular, the accuracy rate can be improved by setting the prediction timing to the period from when mixing and water injection begin until the mixed fresh concrete begins to be discharged from the mixer. Discharge from the mixer begins when the opening of the discharge section located on the lower side of the mixer opens and the mixed fresh concrete begins to be discharged from there.

Even before the start of water pouring, image data can be obtained by photographing the state in which the mixer 2 blades 2a are submerged in the fresh concrete and the state in which the mixer 2 blades 2a are scooping up the fresh concrete. Then, when water pouring starts, the hydration reaction between the cement and water starts and the effects of the admixtures are expressed, resulting in the properties of the fresh concrete, so by predicting after the start of water pouring, the accuracy rate can be improved. Also, when discharge starts, the liquid level of the fresh concrete in the mixer drops as the volume decreases due to discharge, and the state of the mixer blades and inner walls reflected in the data can change, so from the perspective of data consistency, the accuracy rate can be improved by setting the period up to before discharge starts.

In both Example 1 and Comparative Example 1, the predicted timing was 120 seconds after the start of mixing. Water was injected 60 seconds after the start of mixing.

More specifically, in both Example 1 and Comparative Example 1, the images used for learning were cut out in a 256 x 256 pixel range with the center position of the specified image data cut out in a 256 x 256 pixel range as the origin ((x, y) = (0, 0)), and the range was 256 x 256 pixels centered on the coordinates (x, y) = (-10, -10), (0, -10), (+10, -10), (-10, 0), (+10, 0), (-10, +10), (0, +10), (+10, +10). Note that, although omitted to avoid cumbersome description, the units of the x and y coordinates above are pixels.

The number of output data for each was set to 1,043.

The trained model M1 was generated using the Tensor Flow (registered trademark) learning library, and the number of training runs was 200,000.

The accuracy rate was confirmed by considering the predicted value to be within the tolerance range of the actual slump value as the correct answer. The tolerance range was confirmed as 0.5 cm, 1.0 cm, 1.5 cm, 2.0 cm, 2.5 cm, and 3.0 cm.

(result)
4 is a graph plotting the results of verification experiment 1. According to FIG. 4, it is confirmed that, under any condition of the allowable error range, Example 1 has a higher accuracy rate than Comparative Example 1. In other words, when predictions are made at the same timing using a trained model trained by applying a two-dimensional convolutional neural network (2D-CNN), the accuracy rate is low in Comparative Example 1, but it can be confirmed that Example 1, which has been trained with time-series information, has a higher accuracy rate. In other words, the prediction accuracy is improved in Example 1, which includes time-series information in the learning data, compared to Comparative Example 1, which does not include time-series information.

[Verification Experiment 2]
Furthermore, we conducted a verification experiment to confirm the relationship between the conditions for obtaining the image data used as learning input data and prediction accuracy, which is described below.

Example 2
The same materials, mixing time, and video data as those in Example 1 were used, and the mixing image data cut out from the video data was used as the learning input data. For the learning, images of the timing when the mixer blades were not switched and the fresh concrete was moving, and seven consecutive images at 1/30 second intervals were used as the learning input data. Figure 5 is a drawing illustrating the seven pieces of image data obtained under the above conditions.

Example 3
When the same materials, mixing time, and video data as those in Example 1 were used, the mixing image data cut out from the video data was used as the learning input data. For learning, a series of movements regarding the switching of the mixer blades was captured, and images of the timing at which the movement was performed multiple times, a continuous time, and seven consecutive images at intervals of 1/30 seconds were used as learning data. Figure 6 is a diagram illustrating seven pieces of image data acquired under the above conditions. Note that the timing at which a series of movements regarding the switching of the mixer blades was captured and the movement was performed multiple times may be one or more times.

Example 4
The same materials, mixing time, and video data as those in Example 1 were used, and the mixing image data cut out from the video data was used as the learning input data. For learning, there was an image of a timing when there was no switching of the mixer blades and there was movement of the fresh concrete, and the image captured a series of movements of the switching of the mixer blades and the timing when the movement was performed multiple times, and 30 consecutive images at 1/30 second intervals were used as the learning data. Fig. 7 is a drawing illustrating the 30 pieces of image data acquired under the above conditions.

(Comparative Example 2)
Mixing image data extracted from video data was used as learning input data when the same materials, mixing time, and video data were used as in Example 1. Three consecutive images with a continuous time interval of 1/30 seconds, in which there was no movement of fresh concrete, were used as learning data ( FIG. 8 ).

(result)
Fig. 8 is a graph plotting the results of verification experiment 2. Fig. 8 confirms that, under any of the tolerance range conditions, Examples 2, 3, and 4 have higher accuracy rates than Comparative Example 2. In other words, the prediction accuracy is improved in Examples 2, 3, and 4, in which images of consecutive times in which there is no movement of fresh concrete are included in the time-series information, compared to Comparative Example 2, in which such images are not included in the learning data.

In particular, Example 2, which includes images of a timing when there is no mixer blade switching and fresh concrete is moving, had the best accuracy rate, and it was confirmed that Example 3, which captures a series of mixer blade switching movements and includes images of the timing when this movement occurs multiple times, and Example 4, which captures a series of mixer blade switching movements and includes images of the timing when there is fresh concrete moving and the timing when there is no mixer blade switching, and includes images of the timing when this movement occurs multiple times, all had roughly the same accuracy rate.

[Verification Experiment 3]
Finally, we conducted a verification experiment to confirm the difference in prediction accuracy for each elapsed time from the start of mixing depending on whether or not time-series information is included, and we explain this below.

Example 5
When the same materials, mixing times, and videos as in Example 1 were used, mixing images cut out from the videos were used as learning data, and the accuracy rate for each mixing time was confirmed.

(Comparative Example 3)
Comparative Example 3 is similar to Example 1, except that learning input data not including time series information was applied and a trained model trained by applying a two-dimensional convolutional neural network (2D-CNN) was used.

For both Example 5 and Comparative Example 3, a comparison was made at a specified time. The allowable error was 1.5 cm, and the evaluation was based on the accuracy rate. Water was poured 60 seconds after the start of mixing.

(result)
The results are shown in Table 1 below.

As can be seen from Table 1 above, it was confirmed that Example 5, which includes time series information in the learning data, is able to make predictions at a relatively early timing and with relatively high accuracy compared to Comparative Example 3, which does not include time series information in the learning data.

As described above, by applying a trained model trained based on image extraction information and time series information, the quality prediction system 1 can predict the quality of ready-mixed concrete with higher accuracy than conventional predictions based on image extraction information. In addition, as described above, the state of the mixed ready-mixed concrete can be predicted immediately after mixing begins, making it possible to predict the quality of ready-mixed concrete early after mixing begins.

In the quality prediction system 1 of this embodiment, as shown in FIG. 3, the imaging unit 60 is arranged at a position where the blades 2a of the mixer 2 are included in the area 60a to be photographed, but the imaging unit 60 may be arranged at a position where the mixer 2 is not included in the area 60a to be photographed. Even if the mixer 2 is not included, the mixed fresh concrete is always flowing, and changes in fluidity occur as the mixing progresses.

In other words, the quality prediction system 1 can be configured to predict the quality of ready-mixed concrete based on image extraction information extracted from image data capturing the rippling movement of ready-mixed concrete that occurs within the area 60a that is the subject of imaging by the imaging unit 60 due to the operation of the mixer 2 outside the area 60a, and on time-series information.

In addition, in this embodiment, the learning input data and prediction input data include formulation-related information and manufacturing-related information in order to improve prediction accuracy, but it is up to the user to decide whether or not to include this information in the learning input data and prediction input data.

Note that the configuration of the quality prediction system 1 described above is merely an example, and the present invention is not limited to the configurations shown in the figures.

1: Quality prediction system 2: Mixer 2a: Blade 10: Main server 11: Data input unit 12: Data output unit 13: Calculation processing unit 13a: Memory unit 20: Data server 30: Operation terminal 40: Reference terminal 50: Display device 60: Camera 60a: Area M1: Trained model

Claims (6)

A storage unit that stores a trained model generated by machine learning based on teacher data;
an input unit that accepts input of prediction input data;
An output unit that outputs an output result derived by the trained model,
The teacher data is data in which learning input data based on a learning data group including image extraction information, which is information related to the state of the fresh concrete during mixing, and time-series information, which is information related to state changes over time, extracted from a plurality of image data captured at different times by an imaging unit that captures the fresh concrete being mixed, is associated with learning output data including quality prediction information, which is information related to the quality of the fresh concrete after mixing;
As the prediction input data, a plurality of image data of the ready-mixed concrete to be predicted is input,
A ready-mixed concrete quality prediction system, characterized in that the quality prediction information of the ready-mixed concrete to be predicted is output as the output result. The ready-mixed concrete quality prediction system according to claim 1, characterized in that the learning input data and the prediction input data are data based on the image extraction information extracted from a plurality of image data including image data of the blades of a mixer that mixes ready-mixed concrete scooping up the ready-mixed concrete, and the learning data group including the time series information. The ready-mixed concrete quality prediction system according to claim 2, characterized in that the learning input data and the prediction input data are data based on the image extraction information extracted from a plurality of image data including image data of the mixer blades submerged in the ready-mixed concrete and image data of the mixer blades scooping up the ready-mixed concrete, and the learning data group including the time series information. The ready-mixed concrete quality prediction system according to any one of claims 1 to 3, characterized in that the learning input data and the prediction input data are data based on the learning data group including the image extraction information extracted from image data acquired by an imaging unit that captures the ready-mixed concrete being mixed during the period from when the mixed ready-mixed concrete is discharged from the mixer, and the time series information. The ready-mixed concrete quality prediction system according to any one of claims 1 to 3, characterized in that the learning input data and the prediction input data are data based on the learning data group that further includes ready-mixed concrete mix-related information or manufacturing-related information. a storage unit in which a trained model generated by machine learning based on teacher data in which training input data based on a training data group including image extraction information, which is information related to the state of the fresh concrete during mixing, and time-series information, which is information related to state changes over time, extracted from a plurality of image data captured at different times by an imaging unit that captures the fresh concrete being mixed, and training output data including quality prediction information, which is information related to the quality of the fresh concrete after being mixed, are associated, is stored;
A step (A) of inputting a plurality of image data of a prediction target fresh concrete being mixed as prediction input data into the storage unit, and applying the plurality of image data to the trained model;
and (B) acquiring the quality prediction information of the ready-mixed concrete to be predicted, which is derived by the trained model.