Detailed Description
The following description of the embodiments of the present invention will be made apparent and fully in view of the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the present invention, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, unless explicitly stated or limited otherwise, the terms "mounted," "connected," "coupled," and "connected" are to be construed broadly, and may be, for example, fixedly connected, mechanically connected, electrically connected, directly connected, indirectly connected via an intermediate medium, or in communication with each other between two elements, and may be a wireless connection or a wired connection. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
The embodiment of the invention provides a core column analysis method combined with an attention mechanism, as shown in fig. 1, comprising the following steps:
S101, acquiring a core box picture of a geological exploration site, and generating a core box main body picture by using an enrichment region identification algorithm and a semantic segmentation algorithm based on the core box picture of the geological exploration site.
Specifically, through mobile shooting equipment (mobile phone, camera, etc.), shooting pictures as much as possible in orthographic mode to reduce image distortion, two-dimensional images including a core box main body, a core exploration description board corresponding to the core box, a field environment, etc. can be obtained, the image format needs to contain three primary color channels of RGB (one color standard), and the images are clear and complete, uniform in color tone and moderate in contrast.
Further, a core box diagram data set is constructed according to the core box diagram of the geological exploration site, and the data is amplified under the condition that the size of the data set is not changed by using methods of horizontal/vertical overturning, rotation, scaling, cutting, shearing, translation, contrast, color dithering, hue saturation modification, brightness modification, contrast modification and the like, as shown in fig. 2.
Further, the core box diagram data set is manually marked by labelme (image marking tool), wherein the marking method comprises the steps of selecting four vertexes to frame a main body part of the core box, marking to obtain a json (JavaScript Object Notation, a lightweight data exchange format) file, wherein the file contains vertex information, label information and pictures, and converting the json file into an original diagram and a label diagram by writing a corresponding program, wherein the original diagram is shown in fig. 3, and the label diagram is shown in fig. 4.
S102, dividing the width of the main body diagram of the core box equally, and generating standard segmentation information.
Specifically, as shown in fig. 5, based on the assumption that the width of each core box is approximately equal, the width between the upper edge and the lower edge obtained by intelligent identification of the core box is equally divided into N parts, where N is the number of core columns, and a boundary line (i.e., a standard straight line) of each core box is obtained, and a core box area is located between two adjacent standard straight lines.
Furthermore, the standard straight line can divide each core box to a certain extent, but the standard straight line is only a theoretical straight line generated from the upper boundary and the lower boundary based on the assumption that the width of each core box is equal, so that the prediction of the dividing straight line of the picture with irregular core placement is not accurate enough, and in order to obtain higher accuracy, the straight line must be detected from the original image, and the actual straight line is obtained and then compared with the standard straight line.
S103, based on the core box main body diagram, generating piecewise fitting straight line information by using a slope optimized straight line detection fitting algorithm.
Specifically, when the human eyes observe the image, some missing line segments can be automatically complemented, as shown in fig. 6, although the edge of the core box is blocked by a large amount of rock and soil, the human eyes can still linearly associate the whole edge through the non-blocked part of the edge, based on the principle, a linear detection fitting algorithm based on slope optimization is provided, local line segments of the edge of all the core boxes can be detected first, and then the line segments are combined through the property that the local line segments are collinear.
S104, fusing the standard segment information and the segment fitting straight line information to generate a core column diagram.
S105, inputting the core column diagram into an image neural network model to generate lithology classification information.
Specifically, the core column diagram is segmented by utilizing a sliding window mode, and the segmented core column diagram is input into an image neural network model to generate lithology classification information.
Further, through the earlier-stage rock core layering historical data set, the rock lithology is identified by training through the VGG16 classification network according to different rock core classification heights and the rock core classification model data set manufactured by combining the rock core column images.
Further, the obtained core column diagram is cut according to a mode that the length is 20cm and the step length is 10cm by using a sliding window mode, the core column diagram is sent into a trained VGG16 network, core types in the diagram are identified, and finally, an exploration report is output according to a corresponding automatic identification result (namely lithology classification information).
According to the core column analysis method combining the attention mechanism, a core box main body diagram is generated based on a geological exploration site core box picture by utilizing an enrichment region identification algorithm and a semantic segmentation algorithm, the core box main body part in the picture containing the core box can be separated more accurately, complex operations of geological professionals on exploration site pictures through professional software and equipment are avoided, a traditional computer vision algorithm (enrichment region identification algorithm) and a depth learning algorithm (semantic segmentation algorithm) are combined, the problem that the traditional computer vision algorithm cannot adapt to a complex environment, core box region identification is poor in a field scene, processing efficiency after semantic segmentation results is accelerated, box region identification accuracy is improved, standard segmentation information and segmentation fit straight line information are fused, a core column diagram is generated, the problem that a spliced column is cut according to priori knowledge is solved, finally, the pile diagram is input into an image neural network model, lithology classification information is generated, a column report is automatically output according to regulations, the initial stage section marking is convenient for the exploration personnel, and the importance of a core column section is significant to construction site construction.
As an optional embodiment of the present invention, as shown in fig. 7, S101, that is, generating, based on a core box picture of a geological exploration site, an optimal core box circumscribed rectangle and an outline vertex coordinate by using an enrichment region identification algorithm and a semantic segmentation algorithm, includes:
s1011, processing the core box picture of the geological exploration site by using an enrichment region identification algorithm to generate a core box contour region.
S1012, acquiring a core box data set, training the semantic segmentation neural network model based on the core box data set, and generating a core box identification model.
The core box identification model comprises a backbone network, a backbone network and a head network, wherein three classical neural network algorithms are selected for comparison experiments, namely FCN (Fully Convolutional Networks, full convolution network), segNet (image semantic segmentation depth network), unet (medical image semantic segmentation network) and improved MT-Unet (medical image segmentation network).
Further, MT-Unet uses a transducer (a sequence model based on an attention mechanism) structure to replace 2-dimensional convolution operation in an original Unet structure, a displacement multi-head attention mechanism (M-MSA, move multi-head self attention) is used to extract edge information in a cyclic displacement mode, the effect of the displacement operation is good only when resolution is large, deep errors are large, so that when the size is 1/16 of the original size, the layer uses a common transducer structure, residual blocks of a bottleneck structure are introduced in a network jump connection part, the jump connection enables the network to autonomously learn parameters in a training process, whether the layer network is ignored or not is determined through back propagation, the bottleneck structure firstly uses a1×1 convolution check input to process, channel feature dimensions are changed into 64, then 3×3 convolution operation is carried out under the condition that channel dimensions are unchanged, finally a1×1 convolution kernel is used to expand the channel dimensions of a feature map to 256, compared with a common residual block, the number of the structure is shortened, the number of parameters is required by the structure is reduced, the core model Unet is well recognized, and the problem of the bottleneck model is well solved.
Further, the core box dataset was trained and tested on FCN, segNet, unet and modified MT-Unet, respectively, with the model training conditions of display card using Tesla100, optimizer using SGD, momentum (impulse unit) =0.9, weight_decay) =0.0001, initial learning rate set to 0.001, learning rate variation using cosine annealing strategy, minimum learning rate set to 1e-5, maximum iteration number set to 100, batch number set to 16, loss function using BCE price Loss, BCE price Loss being a fused Loss function consisting of BCE Loss (Binary Cross Entropy) and price Loss, where BCE Loss is the cross entropy of the two classification problems, price Loss describes the similarity of two contours (predictor and true value), BCE Loss function values are fused together by weighting, training and test results are shown in Table 1 below:
table 1:
Based on the above table, the FCN series has poor effect with UNet++, segNet and UNet are very similar, MT-Unet has the best effect, segNet has the most parameters, the parameters of the decoder and the encoder are equivalent to almost twice the parameters of the rest model due to the highly symmetrical structure, MT-Unet has the best effect under the condition that the parameter quantity is not obviously increased, the improved effectiveness of MT-Unet is proved, the prediction results of each model are shown in figure 8, the edges of the prediction effects of UNet and UNet++ are smoother, the prediction effects of MT-Unet and SegNet are finer, and the edges are rough.
Further, the labeled core box data set is input into the MT-Unet neural network model for training, and a trained MT-Unet neural network model, namely a core box identification model, is obtained, wherein MT-Unet is trained through 100 iterations, the loss value of the training result is 0.0546, and the identification of the core box can be well completed.
S1013, inputting a core box picture of a geological exploration site into a core box identification model to generate a mask map.
The method comprises the steps of carrying out model output on a geological exploration site core box picture obtained through mobile shooting equipment by using a core box identification model, receiving input of any size by MT-Unet, for example, dividing an image into small blocks with the same size by using a patch module, namely, the size of the image block is 342 multiplied by 192 multiplied by 3, extracting image features by using two M-transformers in an encoder stage, converting feature dimensions from 342 multiplied by 192 multiplied by 3 into 86 multiplied by 48 multiplied by 6, 22 multiplied by 12 and 6 multiplied by 3 multiplied by 24, adding a 2-fold feature dimension of a patch (image block) along with the reduction of the image size, embedding a residual module with a bottleneck structure in a jump connection part in a decoder stage, decoding the patch by using the M-transformers symmetrical to an encoder, carrying out 2-time up sampling on the feature map by using a sub-pixel convolution mode instead of the traditional bilinear interpolation, carrying out pixel level prediction on the image, and finally carrying out pixel level prediction on the image by using a full connection layer, so that the probability value of the box is converted into a box region with a value of 0, and the box region is not converted into a two-dimensional mask value of the box region with the pixel value of 255.
And S1014, determining the optimal core box circumscribed rectangle and the outline vertex coordinates based on the mask map and the core box outline region.
Specifically, a mask map is subjected to binarization processing, a contour detection post-processing algorithm is provided, an optimal external contour rectangle is selected by combining a core box contour region and the external contour rectangle in a multi-mode combination and optimization mode, the optimal core box external rectangle and contour vertex coordinates are obtained, and the contour vertex coordinates are returned to the front end.
S1015, generating a core box main body diagram by utilizing perspective transformation based on the contour vertex coordinates.
Specifically, different original core box images (namely core box images on geological exploration sites) have different shooting angles, irregular core box areas in the images are required to be converted into standard rectangular areas by using perspective conversion operation, the rectangular sizes take average values of the length and the width of the core boxes in the original images, and the contour vertex coordinates are combined with the input original core box images to obtain core box area images with regular space positions through perspective conversion, so that image distortion caused by the problem of shooting angles of mobile equipment can be eliminated.
In the above alternative embodiment, compared with the classical semantic segmentation neural network model such as FCN, segNet, deeplab, etc., the MT-Unet model obtains extremely excellent recognition effect under the condition that the parameters are significantly smaller than other models, reduces service deployment cost, saves a large amount of cost, and has stronger practical application value.
As an optional embodiment of the present invention, as shown in fig. 9, S1011, that is, processing a core box image of a geological exploration site by using an enrichment region identification algorithm, generates a core box contour region, includes:
S10111, identifying a core box picture of the geological exploration site by using an Ojin binarization algorithm, and generating a core exploration description plate.
The method comprises the steps of uploading a geological exploration site core box picture to a server through a front-end webpage, and establishing core box regional characteristic engineering, wherein a white core exploration description board (generally rectangular) for identifying the property of rock and soil is arranged on the side of a core box to be identified, and the lower part of the core exploration description board is the main body of the core box.
Further, the core box picture on the geological exploration site can automatically identify the pixel threshold value through binarization processing of the Ojin method, and the core exploration description board is identified.
S10112, removing the upper area of the lower edge of the core exploration description plate, and generating a two-dimensional denoising gray picture.
S10113, carrying out morphological corrosion expansion treatment on the two-dimensional denoising gray level picture to generate a minimum circumscribed rectangle of the core box.
Specifically, as shown in fig. 10-12, the outline of the middle area is relatively rich in the binary gray level map of the core box after interference elimination above the lower edge of the whiteboard (i.e. the core exploration description board), whether the core box area is the core box area can be judged through the enrichment degree of the outline, but the outline of the core at the moment is relatively sparse, so that part of noise in the picture is removed through morphological blurring operation, then corrosion operation is carried out, a disordered irrelevant area in the background of the picture environment is removed, then expansion operation is carried out, so that the enrichment area is more obvious, and finally screening of the boundary of the enrichment area is carried out in two directions, namely, the boundary reaching the enrichment area is determined when the white area through which a straight line in a certain horizontal direction passes is enough, the boundary is marked as the boundary, and the same in the vertical direction, so that the minimum circumscribed rectangle of the core box is generated.
S10114, comparing the minimum circumscribed rectangle of the core box with an enrichment region identification threshold value, and determining a contour region of the core box based on a comparison result.
Specifically, as the minimum circumscribed rectangle of the core box is sensitive to local small interference around, for example, a bad effect can be generated when a short enrichment region appears, and then the robustness of an enrichment region identification threshold value algorithm is improved aiming at the noise influence of the short enrichment region, and the boundary algorithm is limited by utilizing the identification threshold value to determine the contour region of the core box.
Further, the step of determining the contour area of the core box by utilizing the identification threshold limiting boundary algorithm comprises the steps of determining that the boundary is correct only when a plurality of continuous judgments reach the enrichment area identification threshold, otherwise, the boundary is not considered, the enrichment area identification threshold is set as thresh, the continuously satisfied number identification threshold is set as thresh num, when the boundary is judged, the number of points with 255 pixel values is calculated from the transverse direction and the longitudinal direction and is set as n, statistics is carried out in a sliding window mode, if the number of n of the continuous thresh num exceeds thresh, sliding of the window is stopped, the starting position of the window is set as the boundary, the boundary in each direction needs to be determined in the horizontal direction, and in the vertical direction, as the upper boundary is already determined in the process of identifying the whiteboard, only the lower boundary needs to be confirmed, the position vertex coordinate information of the contour area of the core box can be obtained, and further the position vertex coordinate information of the contour area of the core box is generated, wherein the code of the identification threshold limiting boundary algorithm is as follows:
Input is binary image1, upper edge miny, threshold p is identified, threshold number p num Output is identified, and four vertex coordinates are identified;
1.x1=0,x1then=0,x2=image1.shape[1]-1,x2then=x2,y1=image1.shape[0]-1,y1then=y1;
2. The// initializations x1, x2, y1, x1then, x2then, y1then 1, x2, y1 respectively represent the two abscissas and the ordinate of the lower edge of the rectangle, with the upper edge miny. x1then, x2then, y1then are used to count, as x1then records the number of consecutive abscissas with pixels 255 exceeding the recognition threshold;
3.x1Num=0,x2Num=0,y2Num=0;
4. Calculating the number of 255 pixels under a certain transverse coordinate, and recording the number as a plurality of groups y;
5. Calculating the number of 255 pixels under a certain longitudinal coordinate, and recording the number as a plurality of groups x;
6.while x1<image1.shape[1]do
if y[x1then]>p then
x1Num=x1Num+1;
else
x1=x1then,x1Num=0;
end
if x1Num>pnum then
break;
end
x1then++;
end
the calculation modes of x2 and y1 are the same as the above;
18. Returning 4 vertex coordinates, coordinates of (x 1, y 1) (x 2, y 1) (x 1, miny) (x 2, miny), respectively
As an alternative embodiment of the present invention, as shown in fig. 13, S1014, that is, determining the optimal bounding rectangle of the core box and the coordinates of the contour vertex based on the mask map and the contour region of the core box, includes:
s10141, performing binarization processing on the mask map, and performing contour detection on the mask map after the binarization processing to generate a plurality of contour rectangles.
The mask map is output by using the core box identification model, however, the core box is not completely irregular, the core box area is quadrilateral, the irregular shape of the mask map cannot embody the characteristics of the quadrilateral, in addition, the irregular pattern is unfavorable for a user to adjust the core box area, so that the information of the core box area is simplified for obtaining four vertexes of the quadrilateral, and the mask map is processed by using a contour detection post-processing algorithm.
Further, the mask map is subjected to binarization processing, and then the binary map is subjected to contour detection, and a plurality of recognition areas may be generated during semantic segmentation of a part of the picture, so that a plurality of contours may be generated during contour detection, and the largest connected domain needs to be found.
S10142, analyzing the plurality of contour rectangles based on the contour region of the core box, and removing the contour rectangles outside the core box.
Specifically, according to the contour area of the core box, the mask map is analyzed, so that a plurality of contours located outside the core box can be eliminated.
S10143, sequencing the remaining contour rectangles according to the contour circumferences, and selecting the contour rectangle with the largest contour circumference as a target area based on the sequencing result, wherein the remaining contour rectangles are other contour rectangles except the core box in the contour rectangles.
Specifically, the remaining contour rectangles are ordered according to the contour circumferences, the contour with the largest circumference is taken as a target area, and the vertex is obtained through polygon fitting.
S10144, performing polygon fitting on the target area, and determining the number of vertexes of the target area.
S10145, comparing the number of vertexes of the target area with the number of quadrilateral vertexes, and when the number of vertexes of the target area does not accord with the number of quadrilateral vertexes, performing minimum rectangle fitting on the target area to generate an optimal core box circumscribed rectangle and profile vertex coordinates corresponding to the optimal core box circumscribed rectangle.
Specifically, the number of polygon vertexes (namely, the number of vertexes of a target area) is judged, if the number is 4 vertexes, the fitting of a quadrangle is described, otherwise, the minimum rectangular fitting is carried out on the polygon, the IOU (Intersection over Union, a standard for measuring the accuracy of detecting the corresponding object in a specific data set) calculation is carried out on the recognition area and the target area, generally, the result obtained by MT-Unet semantic segmentation is more accurate than the enrichment recognition method, if the IOU is more than 90%, the coordinate value recognized by MT-Unet and the information such as the IOU are returned to a front-end page, and meanwhile, the next step of upright post is carried out, otherwise, the recognition error is returned, and manual intervention processing is needed.
As an optional embodiment of the present invention, as shown in fig. 14, S103, that is, generating piecewise fitting line information by using a slope-optimized line detection fitting algorithm based on a core box main body diagram, includes:
S1031, generating a local horizontal boundary line segment set by using a straight line segment detection algorithm based on the core box main body diagram.
The invention adopts EDLines algorithm (EDGE DRAWING LINES, edge drawing straight line algorithm) to detect the local horizontal boundary line segment in the main body diagram of the core box, specifically, the invention comprises the steps of traversing the image by a filter kernel to play a role of noise suppression, calculating pixel gradient, screening the gradient which is more likely to be the point of a part of the line segment, connecting the anchor points to generate an initial line segment, generating the initial line segment, screening, removing the point which does not meet the condition, smoothing the line segment by using a least square method, and after EDLines introducing the anchor, calculating the circumscribed rectangle by connecting the anchor points, thereby greatly improving the detection speed by using EDLines to detect the straight line segment.
S1032, carrying out connected domain analysis on the local horizontal boundary line segment set to generate a plurality of straight lines.
Specifically, firstly, calculating the gradient delta and the direction theta of all points in an image, wherein the gradient delta is 20, the gradient theta is 3, then, taking the point with small gradient direction change and adjacent points as a connected domain, judging whether the point needs to be disconnected according to the rectangle degree of each domain to form a plurality of domains with larger rectangle degrees according to rules, and finally, improving and screening all the generated domains, and reserving the domains meeting the conditions, namely, the final straight line detection result, so that the detection speed is high, parameter adjustment is not needed, and the accuracy of straight line detection is improved by using an error control method.
The method comprises the steps of obtaining a local horizontal boundary line segment set, obtaining the position of the boundary from the boundary line segment set, combining adjacent local edge lines by using connected domain analysis, namely integrating the connected domains, returning the attribute of each connected region including the label number, the label matrix, the connected region circumscribed rectangle and the mass center, searching the connected region with the diagonal length larger than a certain threshold value, obtaining the position of the circumscribed rectangle, grouping the edge line segments, grouping the connected regions belonging to one boundary into a group by each rectangular region, respectively carrying out linear detection on each group, classifying the obtained rectangles according to the vertical position, classifying the characteristic values of the rectangles on the same horizontal position into the same class, obtaining the circumscribed rectangle of the connected region belonging to each horizontal line at the moment, classifying the points according to the region, namely, respectively fitting the straight lines on each horizontal line.
S1033, comparing the plurality of straight lines with preset conditions, and longitudinally sequencing the straight lines meeting the preset conditions to generate piecewise fitting straight line information.
Specifically, as shown in fig. 15, after the straight line detection and the connected domain extraction, a plurality of straight lines are obtained, wherein the plurality of straight lines comprise a plurality of straight lines with excessive deviation and a plurality of straight lines with too close distances, the straight lines with the deviation angle from the middle straight line exceeding a certain threshold (namely a preset condition) are judged to be deviated straight lines, the straight lines with the interval between the two straight lines being smaller than a certain threshold are judged to be overlapped, the deviated straight lines and the redundant overlapped straight lines are removed, and the extracted edge straight lines of the core box are obtained by sorting all the straight lines from top to bottom according to the ordinate, namely piecewise fitting straight line information.
Further, the straight line detection algorithm pseudo code is as follows:
Input, core box region image, maximum slope max_angle, minimum length min_len
Output true straight line set image lines
Grey= cvtColor (image);// converting the picture to a grey scale;
Sobel=sobel (grey)// use Sobel operator to perform a horizontal sharpening operation on the picture;
Lines= CREATEEDLINES (). Detect (sobel)// using EDLines line segment detection algorithm to get a set of selected line segments;
Selected_lines=filter_lines (lines, max_angle, min_len)// filtering line segments in the collection, requiring a slope approximately level, culling from selected_lines if the slope is greater than max_angle or the line length is less than min_len for each line segment;
Image lines = lines2binary (selected_lines, image. Shape)// drawing the image individually in a pair of artwork size matrices, noted image lines;
coupled_components=detected_coupled_components (image lines)// operating on image lines, expanding the straight line and integrating the connected regions in the figure so that the junctions are separated into closely straight lines;
Classification of incoming rectangles according to vertical position, classifying the feature values of the rectangles on the same horizontal position into the same class, thus finding the connected domain of the same horizontal line;
Line=fit_ lines (classify), respectively performing straight line fitting on the connected domains on each group of horizontal lines, sampling points in the connected domains, and fitting a plurality of line segments into a straight line according to the sampling points;
Lines=modification_lines (lines);// straight lines with the removed portions not meeting the conditions are mainly two cases, the first is that the angle deviation is too large and the second is that the interval is too small;
10. return straight line set
In the above alternative embodiment, the slope optimization-based straight line detection fitting algorithm obtains the core box diagram to perform standard segmentation information through feature engineering and priori knowledge, so that the core box segmentation fitting straight line information can be better obtained, and the core in the core box diagram is dense and covers.
As an optional embodiment of the present invention, as shown in fig. 16, S104, that is, fusing standard segment information and segment fitting straight line information to generate a core column diagram, further includes:
S1041, calculating an intercept difference and a straight line included angle between standard segmentation information and segmentation fitting straight line information.
Specifically, the two groups of straight lines, namely the standard straight line and the real straight line, have respective characteristics, the standard straight line can approximately determine the positions of all the boundaries, but the accuracy is relatively low, the accuracy of the real straight line is relatively high, but due to the reasons of blurring and shielding of the boundaries, some boundary straight lines cannot be detected, therefore, the straight lines are required to be combined to find accurate boundaries, the missing boundaries are complemented, the judgment standard for judging whether the two straight lines represent the same straight line is whether the distance between the two straight lines is smaller than a certain threshold value, and the specific algorithm flow is as follows:
s1042, comparing the intercept difference with a first preset threshold value, and comparing the included angle of the straight line with a second preset threshold value.
S1043, when the intercept difference accords with a first preset threshold value and the included line angle accords with a second preset threshold value, generating current line information.
Specifically, equations of two straight lines are obtained firstly, intercept differences of two adjacent straight lines and included angles of the straight lines are calculated to serve as judging basis, threshold values are set, if one of the two indexes exceeds the threshold value, the straight lines are regarded as unqualified, 5 lines of core are arranged in most of pictures in a data set, in order to unify the number of the straight lines, 1197 pictures with the number of 5 lines of all the core are collected in the experiment, the number of the straight lines meeting the requirement of each picture is counted, two threshold values are set in the experiment, namely, the intercept differences are not more than 20, the included angles are not more than 0.2, the intercept differences are not more than 30, the included angles are not more than 0.25, the experimental results are shown in the following table 2, the method can be used for automatically detecting the straight lines well, and when the qualified straight lines are smaller than 4, the algorithm codes can automatically adjust the pictures to the straight lines through manual adjustment.
Table 2:
further, through the obtained final straight line information, the edge position coordinates of each core segment can be obtained, when the number of the identified line segments is smaller than 4, the relevant information is returned to the front end and marked as an abnormal identification condition, the abnormal identification condition is required to be manually processed, and when the number of the line segments is identified as 4, namely the core box block area is 5, the follow-up operation can be carried out.
S1044, determining edge position coordinates of each core segment based on the current straight line information, and generating a core column diagram based on the edge position coordinates of each core segment.
With the structure, as shown in fig. 17, based on the edge position coordinates of each core segment, the irregular core segment region in the image is transformed into a standard rectangular region by using perspective transformation operation in combination with the core box picture of the geological exploration site, the rectangular region has a length of 1m and a width of 20cm, and image distortion caused by the problem of shooting angles of mobile equipment can be eliminated by perspective transformation, so that the splicing effect is improved.
In the above optional embodiment, based on the traditional line segment detection method, operations such as straight line fusion, corner detection, affine transformation and the like are added, and standard segmentation information and segmentation fitting straight line information are fused to form a set of complete core box region segmentation and splicing flow, so that the core box diagram can be better cut and spliced for the rock columns, the core segments are spliced by utilizing perspective transformation to obtain the core columns, and the problem of non-continuity and non-fluency in the segmentation of the spliced columns according to priori knowledge is solved.
As shown in fig. 18, a core column analysis method combined with the attention mechanism is described below by a specific embodiment.
Example 1:
As shown in fig. 19, the embodiment is applied to a multi-level core column analysis device, where the core column analysis device includes a front end (vue+ ElementUI), an application layer, a service layer, and an algorithm layer, and a data storage layer (including a relational database management system MySQL and an object storage service database OBS), the front end adopts a vue+ ElementUI (Vue is a library for building an interactive Web interface, elementUI is a user interface frame based on Vue), the algorithm layer and the service layer belong to a java back end, the service layer provides python (a cross-platform computer programming language) service, and analysis of the core column is implemented through interaction between the front end and the java back end, and specific analysis steps are as follows:
Acquiring a core box picture of a geological exploration site through mobile shooting equipment, and manufacturing a core box picture data set;
Calculating a picture gray value through a core shooting image, identifying a core exploration description plate corresponding to a core box, and removing environmental noise;
acquiring a minimum circumscribed rectangle of the core box through an enrichment degree identification optimization algorithm, and further screening a core box region;
Training a classical semantic segmentation model and an improved Unet neural network respectively, selecting an optimal algorithm in combination with exploration requirements, and carrying out morphological recognition on a contour region of the core box;
Performing binarization processing on the mask map, providing a contour detection post-processing algorithm, combining an enrichment degree region with an external contour rectangle, selecting an optimal external contour rectangle through multi-mode combination optimization, acquiring contour vertex coordinates, and returning vertex information to the front end;
Obtaining a cut core box main body diagram by obtaining contour vertex coordinates and using morphological affine transformation and perspective transformation;
Obtaining core box body diagram standard segmentation information through feature engineering and priori knowledge, and providing a slope optimization-based straight line detection fitting algorithm to obtain core box segmentation fitting straight line information;
the standard segmentation information and the segmentation fitting straight line information are fused by proposing a straight line fusion algorithm, and a rock pillar map is obtained by splicing the upright posts of the core segment through perspective transformation;
And identifying lithology of a rock pillar map (namely a rock core pillar map) in a sliding window mode through the trained image classification neural network model, acquiring lithology classification information, and outputting an exploration report.
The embodiment of the invention also discloses a core column analysis system combined with an attention mechanism, as shown in fig. 20, comprising:
The acquisition module 201 is configured to acquire a core box picture on the geological exploration site, generate a core box main body picture based on the core box picture on the geological exploration site by using an enrichment region identification algorithm and a semantic segmentation algorithm, and refer to the related description of S101 in the above method embodiment for details.
The bisection module 202 is configured to divide the width of the main body diagram of the core box equally to generate standard segment information, and for details, see the description related to S102 in the above method embodiment.
The detection module 203 is configured to generate piecewise fit line information by using a slope-optimized line detection fitting algorithm based on the core box main body diagram, and for details, see the relevant description of S103 in the above method embodiment.
And the fusion module 204 is used for fusing the standard segment information and the segment fitting straight line information to generate a core column diagram, and the detailed description of the step S104 in the embodiment of the method is referred to.
The generation module 205 is configured to input the core pillar map into the image neural network model, and generate lithology classification information, for details, see the description related to S105 in the above method embodiment.
The core column analysis system combining the attention mechanism provided by the invention is based on a geological exploration site core box picture, generates a core box main body diagram by utilizing an enrichment region identification algorithm and a semantic segmentation algorithm, can more accurately separate a core box main body part from the picture containing the core box, avoids complex operations of geological professionals on the exploration site shooting picture through professional software and equipment, combines a traditional computer vision algorithm (enrichment region identification algorithm) and a deep learning algorithm (semantic segmentation algorithm), solves the problems that the traditional computer vision algorithm cannot adapt to complex environments, the core box region is poorly identified in a field scene, accelerates the post-processing efficiency of semantic segmentation results, improves the recognition accuracy of the box region, fuses standard segmentation information and segmentation fitting straight line information, generates a core column diagram, solves the problem of discontinuous and unsmooth segmentation of a column according to priori knowledge, finally inputs the core column diagram into an image neural network model, generates lithology classification information, automatically outputs a column report according to a specified format, brings convenience to the initial labeling of lithology section regions, and has significance for the exploration of a power station and has important significance for construction and site construction.
As an optional implementation manner of the invention, the acquisition module 201 comprises a processing submodule, a training submodule, a first generation submodule and a determination submodule, wherein the processing submodule is used for processing a core box picture of a geological exploration site by using an enrichment degree region identification algorithm to generate a core box outline region, the training submodule is used for acquiring a core box data set, training a semantic segmentation neural network model based on the core box data set to generate a core box identification model, the first generation submodule is used for inputting the core box picture of the geological exploration site into the core box identification model to generate a mask map, the determination submodule is used for determining an optimal core box circumscribed rectangle and outline vertex coordinates based on the mask map and the core box outline region, and the transformation submodule is used for generating a core box main body map by using perspective transformation based on the outline vertex coordinates.
The processing submodule comprises a generating unit, a removing unit, a processing unit and a first comparing unit, wherein the generating unit is used for identifying a core box picture of a geological exploration site by using an Ojin binarization algorithm to generate a core exploration description plate, the removing unit is used for removing an upper area of the lower edge of the core exploration description plate to generate a two-dimensional denoising gray level picture, the processing unit is used for carrying out morphological corrosion expansion processing on the two-dimensional denoising gray level picture to generate a minimum circumscribed rectangle of the core box, and the first comparing unit is used for comparing the minimum circumscribed rectangle of the core box with an enrichment area identification threshold value and determining a contour area of the core box based on a comparison result.
The determination submodule comprises a detection unit, an analysis unit, a sequencing unit, a matching unit and a second comparison unit, wherein the detection unit is used for carrying out binarization processing on a mask graph, carrying out contour detection on the mask graph subjected to binarization processing to generate a plurality of contour rectangles, the analysis unit is used for carrying out analysis on the plurality of contour rectangles based on a core box contour region to remove the contour rectangles outside the core box, the sequencing unit is used for sequencing the rest contour rectangles according to contour circumferences and selecting the contour rectangle with the largest contour circumference as a target region based on the sequencing result, the rest contour rectangles are other contour rectangles except the core box in the plurality of contour rectangles, the matching unit is used for carrying out polygon matching on the target region to determine the vertex number of the target region, the second comparison unit is used for comparing the vertex number of the target region with the quadrilateral vertex number, and carrying out minimum rectangular matching on the target region when the vertex number of the target region does not accord with the quadrilateral vertex number to generate an optimal core box circumscribed rectangle and contour vertex coordinates corresponding to the optimal core box circumscribed rectangle.
The detection module 203 comprises a second generation sub-module, an analysis sub-module and a first comparison sub-module, wherein the second generation sub-module is used for generating a local horizontal boundary line segment set based on a core box main body diagram by using a straight line segment detection algorithm, the analysis sub-module is used for conducting connected domain analysis on the local horizontal boundary line segment set to generate a plurality of straight lines, and the first comparison sub-module is used for respectively comparing the plurality of straight lines with preset conditions and longitudinally sequencing the straight lines meeting the preset conditions to generate piecewise fitting straight line information.
As an optional implementation manner of the present invention, the fusion module 204 includes a calculation submodule, a second comparison submodule, a third generation submodule and a fourth generation submodule, wherein the calculation submodule is used for calculating an intercept difference and a straight line included angle between standard segment information and segment fitting straight line information, the second comparison submodule is used for comparing the intercept difference with a first preset threshold value and comparing the straight line included angle with a second preset threshold value, the third generation submodule is used for generating current straight line information when the intercept difference accords with the first preset threshold value and the straight line included angle accords with the second preset threshold value, and the fourth generation submodule is used for determining edge position coordinates of each core segment based on the current straight line information and generating a core column graph based on the edge position coordinates of each core segment.
In addition, an electronic device is provided in an embodiment of the present invention, as shown in fig. 21, where the electronic device may include a processor 110 and a memory 120, where the processor 110 and the memory 120 may be connected by a bus or other manner, and in fig. 21, the connection is exemplified by a bus. In addition, the electronic device further includes at least one interface 130, where the at least one interface 130 may be a communication interface or other interfaces, and the embodiment is not limited thereto.
The processor 110 may be a central processing unit (Central Processing Unit, CPU). The Processor 110 may also be other general purpose processors, digital Signal Processors (DSP), application SPECIFIC INTEGRATED Circuits (ASIC), field-Programmable gate arrays (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 120 is used as a non-transitory computer readable storage medium, and can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to a core column analysis method in combination with an attention mechanism in an embodiment of the present invention. The processor 110 executes various functional applications and data processing of the processor by running non-transitory software programs, instructions, and modules stored in the memory 120, i.e., to implement a core column analysis method that incorporates an attention mechanism in the above-described method embodiments.
The memory 120 may include a storage program area that may store an operating system, application programs required for at least one function, and a storage data area that may store data created by the processor 110, etc. In addition, memory 120 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 120 may optionally include memory located remotely from processor 110, which may be connected to processor 110 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
In addition, at least one interface 130 is used for communication of the electronic device with external devices, such as with a server or the like. Optionally, at least one interface 130 may also be used to connect peripheral input, output devices, such as a keyboard, display screen, etc.
The one or more modules are stored in the memory 120 and when executed by the processor 110, perform a core column analysis method incorporating an attention mechanism as in the embodiment of fig. 1.
The specific details of the electronic device may be understood correspondingly with respect to the corresponding related descriptions and effects in the embodiment shown in fig. 1, which are not repeated herein.
It will be appreciated by those skilled in the art that implementing all or part of the above-described embodiment method may be implemented by a computer program to instruct related hardware, where the program may be stored in a computer readable storage medium, and the program may include the above-described embodiment method when executed. The storage medium may be a magnetic disk, an optical disc, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a Flash Memory (Flash Memory), a hard disk (HARD DISK DRIVE, HDD), or a Solid state disk (Solid-state-STATE DRIVE, SSD), and the like, and the storage medium may further include a combination of the above types of memories.
It is apparent that the above examples are given by way of illustration only and are not limiting of the embodiments. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. While still being apparent from variations or modifications that may be made by those skilled in the art are within the scope of the invention.