CN113936164A

CN113936164A - Recognition method, network and training method of hidden coding composed of directional primitives

Info

Publication number: CN113936164A
Application number: CN202111027485.XA
Authority: CN
Inventors: 杜旺; 陆正宏
Original assignee: Shanghai Huayou Information Technology Co ltd
Current assignee: Shanghai Huayou Information Technology Co ltd
Priority date: 2021-09-02
Filing date: 2021-09-02
Publication date: 2022-01-14

Abstract

The invention discloses an identification method, a network and a training method of hidden codes composed of directional primitives.A hidden pattern is formed on a commodity page through the directional primitives, and the hidden pattern is identified by utilizing the identification network to obtain a corresponding code so as to obtain value-added resources associated with commodities; by training the recognition network, the recognition network has excellent recognition speed and recognition accuracy. Compared with two-dimensional codes and bar codes, the hidden pattern adopted by the invention realizes a more beautiful coding mode in commodity application, and retains the original artistic value; for scenes needing simple coding, the method has higher detection precision and more attractive expression form; after the recognition network is trained, the recognition network can detect and obtain codes within 3s, the accuracy of actually detecting the codes is more than 98%, and better use experience is achieved.

Description

Recognition method, network and training method of hidden codes composed of directional primitives

Technical Field

The invention relates to the technical field of image coding, in particular to a method for identifying hidden codes consisting of directional primitives, a network and a training method.

Background

Currently, printed products such as books and product packages on the market are widely used in the forms of two-dimensional codes and bar codes to store additional information related to commodities such as sales promotion links and commodity numbers of the products. For another example, the children's book capable of being used with the touch and talk pen uses the invisible pattern printed by the special infrared sensitive pigment to code the corresponding audio resource, thereby realizing the reading experience of ' reading where the touch and talk ' is realized.

However, in some special scenarios, these methods also have certain disadvantages. For example, in order to realize interactive reading, such as code scanning to view associated content, adding a large number of two-dimensional codes in the content page of the children's picture book will seriously affect the beauty and layout of the content page. The hidden codes of the audio reading material matched with the point reading pen are printed by using special pigment and can only be recognized by the point reading pen and can not be seen by naked eyes, but the codes are matched with the appointed point reading pen hardware for use, and meanwhile, the point reading pen adopts a scheme that the hidden OID codes are printed by using special coating capable of reflecting infrared light in a very high ppi mode, so that the requirements on the coating and printing equipment are high, and the hidden codes cannot be printed on the surfaces of rough materials such as cloth and the like, so that the audio reading material does not have the possibility of further popularization and application. The invention provides a method for identifying hidden codes consisting of directional primitives, a network and a training method, which are used for solving the problems.

Disclosure of Invention

The invention provides an identification method, a network and a training method of hidden codes composed of directional primitives, which have the characteristic of attractive appearance and effectively reduce the process cost.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a method for identifying hidden codes composed of directional primitives comprises the following steps:

s1, randomly generating a primitive: randomly generating a group of primitives, arranging the primitives according to a corresponding sequence to form a hidden pattern, and laying the hidden pattern on a commodity page;

s2, code scanning identification: using app and calling a camera to scan a commodity page, and scanning the commodity page by the app and identifying a hidden pattern;

s3, pattern decoding: decoding the identified hidden pattern by using an identification algorithm built in the app to obtain an identifiable code corresponding to the hidden pattern, and calling a corresponding function in the app according to the identifiable code;

s4, outputting the code: and displaying the function corresponding to the hidden pattern on the display page of the app, and performing subsequent operation.

Further, in step S1, the hidden pattern includes encoding primitives and positioning primitives, where the encoding primitives correspond to the recognizable codes one by one, and the positioning primitives are used for positioning during code scanning recognition; the color of the hidden pattern is similar to that of the commodity page.

Further, in step S3, the decoding procedure of the identified network is as follows:

s31, primitive identification and detection: the method comprises the steps that a recognition network recognizes a hidden pattern obtained by a camera, and detects the type, position, size, rotation angle and confidence coefficient of a graphic element in the hidden pattern;

s32, primitive confidence filtering: filtering the primitives according to the confidence coefficient, and deleting the primitives with the confidence coefficient smaller than 0.5;

s33, primitive hierarchy filtering: filtering the primitive according to the category, the position and the size, and deleting the primitive detection result which does not meet the preset hierarchical structure limiting condition;

s34, encoding mapping: mapping the primitive to a final code according to a preset hierarchical structure and a mapping relation between the primitive and the code;

s35, outputting the recognizable code: and the identification network outputs the decoded identifiable code to a corresponding module of the app and calls a corresponding function.

Furthermore, the coding primitives are patterns with directivity and correspond to different identifiable codes according to different directions and orientations; the positioning graphic element is a rectangular pattern and comprises a rectangular solid line outer frame, an internal filling color, a preset pattern located in the center and a blank area reserved for the encoding graphic element.

Furthermore, the coding primitives are arranged into two rows, and each row is provided with five columns; and the two rows of coding primitives are positioned inside the rectangular frame of the positioning primitive, and the two rows of coding primitives are respectively arranged at the top and the bottom of the positioning primitive.

A hidden coding identification network composed of directional primitives comprises a trunk convolution network, a fusion network and a prediction network, wherein the trunk convolution network, the fusion network and the prediction network are connected in series from front to back;

the backbone convolutional network is used for extracting multi-scale features in the picture and outputting feature maps of multiple scales;

the fusion network is used for fusing a plurality of scale features, receives a plurality of scale feature graphs input by the backbone convolution network, and outputs a fusion feature graph fused with the plurality of scale features;

the prediction network is used for predicting the class confidence and the position information of the target to be detected, receiving the fusion characteristic diagram output by the fusion network and outputting a prediction result.

A training method of a hidden coding recognition network composed of directional primitives comprises the following steps:

and S01, generating a sample set: randomly generating a training set and a testing set in batches by using a python script;

s02, data enhancement: performing data enhancement on the sample to obtain an enhanced sample image and an enhanced sample label;

s03, training the model: training the recognition network by using a training set, and storing network parameters after multiple iterations until the model converges;

s04, test model: and loading the network parameters into the identification network, taking the test set as input, and comparing the difference between the predicted result and the real result so as to judge whether the model has robustness for the strange input sample.

Further, in step S01, the python script will correspondingly scale, rotate and arrange the primitives on a random background according to the randomly generated codes to generate sample images, where a single sample includes the sample images and the sample labels, and a huge number of samples constitute a sample set.

Further, in step S02, the data enhancement operation includes random cropping, random brightness adjustment, random color perturbation, random saturation adjustment, random sharpening, and random blurring, which are randomly combined.

Further, in step S03, the training process of identifying the network includes the following steps:

s031, for given input sample, input the sample picture into the network, finish the forward propagation computational process, the network outputs and obtains the label predicted finally;

s032, calculating an error according to the real label of the input sample and the prediction label obtained in the step S031;

s033, reversely propagating the error calculated in the step S032 from the output end to the input end of the network, and adjusting parameters of each layer of the network by using a gradient descent optimization strategy, thereby completing a round of iterative process;

and S034, dividing the training set into a plurality of small batches of samples, repeating the processes of S031, S032 and S033 aiming at each batch of samples, and performing loop iteration until the error obtained by calculation in the step S032 is basically unchanged, the model is converged and the training process is finished.

The invention has the following beneficial effects:

compared with two-dimensional codes and bar codes, the hidden pattern adopted by the invention realizes a more beautiful coding mode in commodity application, and retains the original artistic value; for scenes needing simple coding, the method has higher detection precision and more attractive expression form; after the recognition network is trained, the recognition network can detect and obtain codes within 3s, the accuracy of actually detecting the codes is more than 98%, and better use experience is achieved.

Drawings

FIG. 1 is a schematic flow chart of a hidden primitive recognition method according to the present invention;

FIG. 2 is a schematic decoding flow of the identification network of the present invention;

FIG. 3 is a schematic diagram of the identification network structure of the present invention;

FIG. 4 is a schematic diagram of a training process of the recognition network of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the specification, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In the description of this patent, it is to be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are used in the orientations and positional relationships indicated in the drawings for the convenience of describing the patent and for the simplicity of description, and are not intended to indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and are not to be considered limiting of the patent.

As shown in fig. 1, a method for recognizing a hidden code composed of directional primitives includes the following steps:

Further, in step S1, the hidden pattern includes coding primitives and positioning primitives, where the coding primitives perform a role of coding mapping, the coding primitives correspond to identifiable codes in a one-to-one manner, and the positioning primitives are used for positioning during code scanning identification; the color of the hidden pattern is similar to that of the commodity page.

In the first embodiment of the present invention, the coding primitive is a pattern with directivity, and different identifiable codes are corresponding to different directions of the pattern; the positioning graphic element is a rectangular pattern and comprises a rectangular solid line outer frame, an internal filling color, a preset pattern located in the center and a blank area reserved for the encoding graphic element.

Preferably, the coding primitives set 8, the 8 coding primitives in different directions represent 8 codes, the upright coding primitive is taken as a reference, the corresponding code is 0, the subsequent coding primitives rotate by 45 degrees clockwise relative to the previous coding primitive in sequence, and the corresponding code is increased by one; 8 coding primitives in different directions realize the mapping of 0-7 in the octal counting method.

Preferably, the coding primitives are any pattern capable of identifying directionality, such as various individual patterns or combined patterns.

Preferably, the coding primitives are arranged in two rows, and each row is provided with five columns, which totally comprise 10 primitives; the positioning primitives are rectangular frames, and two lines of coding primitives are respectively arranged at the top and the bottom of the frame.

As shown in fig. 2, further, in step S3, the decoding process of the identified network is as follows:

s32, primitive confidence filtering: filtering the primitives according to the confidence coefficient, and deleting the primitives with the confidence coefficient smaller than 0.5; the smaller the confidence coefficient is, the more unreliable the detection result is, and the larger the confidence coefficient is, the more reliable the detection result is;

s33, primitive hierarchy filtering: filtering the primitive according to the category, the position and the size, and deleting the primitive detection result which does not meet the preset hierarchical structure limiting condition; the hierarchy constraints include: whether the primitives of each category exist or not, the number of the primitives of each category, and the relative position, relative size and inclusion relation among the primitives;

the specific way of performing the hierarchical structure filtering on the hidden pattern of the first embodiment is as follows: whether 10 coding primitives exist and are located in the range of one positioning primitive is detected, and the 10 coding primitives approximately meet the preset relative position relationship of two rows and five columns.

Preferably, in step S31, the input picture is identified and detected by using an identification network, the category confidence and the location information obtained after the identification network detection are filtered by the following algorithms in steps S31 and S32, and the filtered primitives are mapped to obtain codes and output the codes.

As shown in fig. 3, a hidden coded recognition network composed of directional primitives includes a trunk convolutional network, a fusion network, and a prediction network, where the trunk convolutional network, the fusion network, and the prediction network are connected in series from front to back;

the backbone convolutional network is used for extracting a plurality of scale features in the picture and outputting a feature map of a plurality of scales;

the backbone convolution network is formed by connecting a plurality of basic convolution modules in series;

The preferred embodiment of the identification network of the present invention: and inputting a color image with the shape of 512 × 3, and outputting the color image as the category confidence coefficient and the position information of the target to be detected.

The main convolution network obtains a feature map with the size of 256 × 256 and the number of channels of 32 after convolution operation is carried out through a basic convolution module, and then the feature map is subjected to convolution operation successively through the basic convolution module; selecting feature maps which form a low scale, a medium scale and a high scale from the front part, the middle part and the rear part of the backbone convolution network respectively, wherein the low scale is 64 × 48, the medium scale is 32 × 120, and the high scale is 16 × 352;

inputting low, medium and high three-scale feature maps formed by a backbone convolution network into a fusion network, unifying the number of feature channels of each scale through the fusion network, outputting a top-layer feature map after channel reforming, performing bit-wise addition operation on the top-layer feature map and a feature map of an adjacent lower layer after the top-layer feature map is sampled, outputting the result, and repeating the same operation with the lower layer until the lowest layer is reached to obtain a high-scale fusion feature map of 16 × 128, a medium-high-scale fusion feature map of 32 × 158 and a medium-high-low-scale fusion feature map of 64 × 128;

inputting the three scales of fusion feature maps into a prediction network, further extracting features through multi-round convolution operation respectively, and predicting category confidence and position information of the target to be detected through convolution branches respectively; and filtering the information identified by the identified network through the finally obtained category confidence coefficient and position information through subsequent algorithms of the steps S31 and S32.

As shown in fig. 4, a training method for a hidden coded recognition network composed of directional primitives includes the following steps:

Preferably, in step S01, the number of training set samples is 100000, and the number of test set samples is 10000

and S034, dividing the training set into a plurality of small batches of samples, repeating the processes of S031, S032 and S033 for each batch of samples, and performing loop iteration until the error calculated in the step S032 is basically unchanged, wherein the model can be considered to be converged, and the training process is finished.

Further, the trained recognition network can realize a detection speed of more than 20 frames per second for 640 × 640 resolution image input; in the recognition accuracy, under the condition that the illumination and the shooting angle are normal, after the angle is manually adjusted, the code can be detected within 3s, and the accuracy of the actually detected code is more than 98%.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein, and any reference signs in the claims are not intended to be construed as limiting the claim concerned.

Claims

1. the identification method of the hidden coding that a kind of directional graphic element is formed, is characterized in that, comprises the following steps:

S1, Randomly generate graphic elements: randomly generate a set of graphic elements, arrange them in corresponding order to form hidden patterns, and lay them on the product page;

S2, scan code recognition: use the app and call the camera to scan the product page, the app scans the product page and recognizes the hidden pattern;

S3, pattern decoding: the built-in recognition algorithm in the app decodes the recognized hidden pattern, obtains a recognizable code corresponding to the hidden pattern, and calls the corresponding function in the app according to the recognizable code;

S4, output encoding: display the function corresponding to the hidden pattern on the display page of the app, and perform subsequent operations.

2. the identification method of the hidden coding that a kind of directional graphic element is formed according to claim 1, it is characterized in that: in step S1, described hidden pattern comprises coding graphic element and positioning graphic element, and described coding graphic element and The identifiable codes are in one-to-one correspondence, and the positioning graphic element is used for positioning when scanning the code for identification; the color of the hidden pattern is similar to the color of the product page.

3. the identification method of the hidden coding that a kind of directional graphic element according to claim 1 is formed is characterized in that: in step S3, the decoding process of described identification network is as follows:

S31, primitive identification and detection: the identification network identifies the hidden pattern obtained by the camera, and detects the category, position, size, rotation angle and confidence of the primitive in the hidden pattern;

S32, primitive confidence filter: filter primitives according to the confidence, and delete the primitives whose confidence is less than 0.5;

S33, filtering of primitive hierarchical structure: filtering primitives according to category, location and size, and deleting detection results of primitives that do not meet preset hierarchical structure constraints;

S34, encoding mapping: map the primitive to the final encoding according to the preset hierarchical structure and the mapping relationship between the primitive and the encoding;

S35, output the identifiable code: the recognition network outputs the decoded identifiable code to the corresponding module of the app, and calls the corresponding function.

4. the identification method of the hidden coding that a kind of directional graphic element is formed according to claim 2, it is characterized in that: described coding graphic element is the pattern with directional, and according to the different corresponding different identifiable identifiable directional orientations coding; the positioning graphic element is a rectangular pattern, including a rectangular solid line outer frame, an inner filling color, a preset pattern located in the center, and a blank area reserved for the coding graphic element.

5. the identification method of the hidden coding that a kind of directional graphic element is formed according to claim 4, it is characterized in that: described coding graphic element is set to two rows, and every row is set to five columns; Inside the rectangular frame of the primitive, two lines of coding primitives are set at the top and bottom of the positioning primitive respectively.

6. A recognition network applied to the recognition method of the hidden coding that the directional graphic element described in any one of claims 1-5 is formed, it is characterized in that: comprise backbone convolution network, fusion network and prediction network, described The backbone convolutional network, fusion network and prediction network are connected in series from front to back;

The backbone convolutional network is used to extract multi-scale features in the picture, and output feature maps of multiple scales;

The fusion network is used for fusion of multiple scale features, accepts feature maps of multiple scales input by the backbone convolution network, and outputs a fusion feature map after fusion of multiple scale features;

The prediction network is used for predicting the category confidence and position information of the target to be detected, accepts the fusion feature map output by the fusion network, and outputs the prediction result.

7. the training method of the identification network of the hidden coding that a kind of directional graphic element composition according to claim 6 is characterized in that, comprises the following steps:

S01, generate sample set: use python script to randomly generate training set and test set in batches;

S02, data enhancement: perform data enhancement on the samples to obtain enhanced sample images and enhanced sample labels;

S03, train the model: use the training set to train the recognition network, and save the network parameters after multiple rounds of iteration until the model converges;

S04, test the model: load the network parameters into the recognition network, use the test set as the input, and compare the difference between the predicted result and the real result, so as to judge whether the model is robust to unfamiliar input samples.

8. the training method of the identification network of the hidden coding that a kind of directional graphic element is formed according to claim 7, it is characterized in that: in step S01, python script can correspondingly scale on random background according to the coding of random generation, Rotate and arrange primitives to generate sample images. A single sample contains sample images and sample labels, and a large number of samples form a sample set.

9. the training method of the identification network of the hidden coding that a kind of directional graphic element composition according to claim 7 is characterized in that: in step S02, the operation of data enhancement comprises random cropping, random brightness adjustment, random color disturbance, Random Saturation Adjustment, Random Sharpening, and Random Blur, in random combinations of the above.

10. the training method of the identification network of the hidden coding that a kind of directional graphic element composition according to claim 7 is characterized in that:

In step S03, the training process of the recognition network includes the following steps:

S031, for a given input sample, input the sample image into the network, complete the forward propagation calculation process, and finally the network outputs the predicted label;

S032, calculate the error according to the real label of the input sample and the predicted label obtained in step S031;

S033, the error calculated in step S032 is back-propagated along the network output end to the input end, and the gradient descent optimization strategy is used to adjust the parameters of each layer of the network, thereby completing a round of iteration process;

S034, the training set is divided into a plurality of small batches of samples, and for each batch of samples, the processes of S031, S032 and S033 are repeated, and the cycle is iterated until the error calculated in step S032 is basically unchanged, the model has converged, and the training process is over.