CN110472516B - A method, device, equipment and system for constructing a person image recognition system - Google Patents

Abstract

The present invention discloses a method, device, equipment and system for constructing a person image recognition system, the method comprising: obtaining a basic framework for constructing a person image recognition system; determining a plug-in to be loaded in response to a received plug-in loading instruction, wherein the plug-in loading instruction carries configuration parameters pointing to at least one business plug-in; generating a business code for constructing the person image recognition system based on a configuration script and the configuration parameters of the plug-in to be loaded; integrating the business code with the basic framework to construct the person image recognition system; wherein the business plug-in comprises at least one selected from a group consisting of an image acquisition plug-in and a neural network model plug-in, and the configuration parameters of the neural network model plug-in point to at least one neural network model for person image recognition. The development efficiency of developers can be greatly improved, and different development requirements of different business scenarios can be met.

Description

A method, device, equipment and system for constructing a person image recognition system

Technical Field

The present invention relates to the field of Internet communication technology, and in particular to a method, device, equipment and system for constructing a person image recognition system.

Background Art

Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields.

Existing online systems are often based on a certain type of application scenario, and different business codes often need to be written for each different type of application scenario. Even for each segmented business scenario under a certain type of application scenario, a code library is established to build the online system according to the different needs of each segmented business scenario. For example, if the business processes in business scenario 1 and business scenario 2 are the same or similar, the repeated development of the same type of modules in the business scenario will bring about high development and maintenance costs. Therefore, for the development of online systems with the same or similar business processes, a more effective solution needs to be provided to reduce code development costs and maintenance expenses.

Summary of the invention

In order to solve the problems of low efficiency and high cost in the development of online systems with the same or similar business processes in the existing technology, the present invention provides a method, device, equipment and system for constructing a person image recognition system:

In one aspect, the present invention provides a method for constructing a person image recognition system, the method comprising:

Get the basic framework for building a person image recognition system;

In response to a received plug-in loading instruction, determining a plug-in to be loaded, wherein the plug-in loading instruction carries configuration parameters pointing to at least one service plug-in;

Based on the configuration script and the configuration parameters of the plug-in to be loaded, generating a business code for building the person image recognition system;

Integrate the business code with the basic framework to build the character image recognition system;

The business plug-in includes at least one selected from the group consisting of an image acquisition plug-in and a neural network model plug-in, and the configuration parameters of the neural network model plug-in point to at least one neural network model for character image recognition.

On the other hand, a device for constructing a person image recognition system is provided, the device comprising:

Basic framework acquisition module: used to obtain the basic framework for building a person image recognition system;

A plug-in confirmation module to be loaded: used to determine a plug-in to be loaded in response to a received plug-in loading instruction, wherein the plug-in loading instruction carries configuration parameters pointing to at least one business plug-in;

A business code generation module: used to generate business codes for constructing the person image recognition system based on a configuration script and configuration parameters of the plug-in to be loaded;

Fusion module: used to fuse the business code with the basic framework to build the character image recognition system;

The business plug-in includes at least one selected from the group consisting of an image acquisition plug-in and a neural network model plug-in, and the configuration parameters of the neural network model plug-in point to at least one neural network model for character image recognition.

On the other hand, an electronic device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for constructing a person image recognition system as described above.

On the other hand, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the method for constructing a character image recognition system as described above.

The present invention provides a method, device, equipment and system for constructing a person image recognition system, which has the following technical effects:

The present invention is applied to the development of a person image recognition system. It can greatly improve the development efficiency of developers and meet different development requirements of different business scenarios. In the form of a plug-in, it can reduce the workload of repeated development and facilitate maintenance.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a schematic diagram of an application environment provided by an embodiment of the present invention;

FIG2 is a flow chart of a method for constructing a person image recognition system provided by an embodiment of the present invention;

3 is a schematic diagram of a business process of a corresponding system applied to a small smart store business scenario provided by an embodiment of the present invention;

4 is a schematic diagram of a business process of a corresponding system applied to a large-scale smart shopping mall business scenario provided by an embodiment of the present invention;

5 is a schematic diagram of a flow chart of updating and using a plug-in for a person image recognition system provided by an embodiment of the present invention;

6 is a schematic diagram of a business process of a person image recognition system provided by an embodiment of the present invention;

FIG. 7 is also a schematic diagram of a business process of a person image recognition system provided by an embodiment of the present invention;

9 is a block diagram of a device for constructing a person image recognition system provided by an embodiment of the present invention;

FIG. 10 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

Please refer to FIG. 1, which is a schematic diagram of an application environment provided by an embodiment of the present invention, and may include a client 01 and a server 02, wherein the client and the server are connected via a network. A user sends a person image to the server via the client, and the server recognizes the received person image. It should be noted that FIG. 1 is only an example.

Specifically, the client 01 may include physical devices such as smart phones, desktop computers, tablet computers, laptops, digital assistants, smart wearable devices, etc., and may also include software running in physical devices, such as web pages provided to users by some service providers, or applications provided to users by these service providers.

Specifically, in the embodiments of this specification, the server 02 may include an independently operated server, or a distributed server, or a server cluster composed of multiple servers. The server 02 may include a network communication unit, a processor, a memory, and the like.

It should be noted that for character images that are associated with user information, when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

The following introduces a specific embodiment of a method for constructing a person image recognition system of the present invention. FIG2 is a flow chart of a method for constructing a person image recognition system provided by an embodiment of the present invention. This specification provides method operation steps as described in the embodiment or flow chart, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the order of many steps and does not represent the only execution order. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, a parallel processor or a multi-threaded processing environment). Specifically, as shown in FIG2, the method may include:

S201: Acquire a basic framework for building a person image recognition system;

In an embodiment of the present invention, a character image recognition system applied to a certain type of application scenario often has the same or similar business processes. For example, when applied to a smart store application scenario where character image recognition is required, the functional module shown in FIG3 is set for a small smart store business scenario, and the functional module shown in FIG4 is set for a large smart shopping mall business scenario. As can be seen from FIGS. 3 and 4, the two business scenarios have the same or similar business processes: video input device (hardware module) -> image acquisition module (used to obtain a stream from the video input device) -> algorithm module (used to identify the acquired image) -> storage module -> auxiliary module (such as for processing logs, monitoring, etc.). Among them, compared with the functional modules set for small store business scenarios, the functional modules set for large shopping mall business scenarios have more camera types set in the hardware modules, and more dimensions for image recognition in the algorithm modules.

Considering the same or similar business processes of the person image recognition system applied to a certain type of application scenario, a corresponding basic framework can be set up, which can provide a basic code library, a public class library, a basic service interface, etc. for the person image recognition system built for a specific business scenario. Obtaining the basic framework for building a person image recognition system can reduce the large development costs caused by repeated development to a certain extent. For example, for the business scenarios shown in 3 and 4, it is no longer necessary to develop an image acquisition module from scratch.

Of course, application scenarios that require character image recognition are not limited to smart stores (stores, shopping malls), but can also be smart retail, smart communities, etc.

S202: Determine a plug-in to be loaded in response to a received plug-in loading instruction, wherein the plug-in loading instruction carries configuration parameters pointing to at least one service plug-in;

In an embodiment of the present invention, the business plug-in is a functional module setting corresponding to the business process for implementing the person image recognition system. The business plug-in may include at least one selected from the group consisting of an image acquisition plug-in and a neural network model plug-in. According to the configuration parameters of at least one business plug-in in the plug-in loading instruction, it can be determined whether a certain business plug-in needs to be loaded. The business plug-in that needs to be loaded can be selected in a more fine-grained manner in combination with the specific needs of the business scenario.

In a specific embodiment, the neural network model plug-in corresponds to an algorithm module, and the configuration parameters of the neural network model plug-in point to at least one neural network model for character image recognition. The neural network model for character image recognition can be a neural network model for face detection, body detection, body tracking, head-body binding, face quality score, posture detection, etc. In actual applications, a smart store can perform character image recognition on users who come to the store based on the constructed character image recognition system. The user's activities in the smart store can be captured by a video input device (such as a camera), and the neural network model plug-in can perform character image recognition on the original images captured by the aforementioned shooting.

The neural network model corresponding to face detection can determine whether the original image contains a face; if there is a face, the face area can be cropped from the original image, and the position, size and face attribute analysis results of the face area can also be returned. Furthermore, the face attribute analysis results can include gender, expression (neutral, smiling, laughing), age (error estimated to be less than 5 years old), whether wearing glasses (ordinary glasses, sunglasses), whether wearing a hat, whether wearing a mask, etc.

The neural network model corresponding to human body detection can determine whether the original image contains a human body; if a human body is present, the human body region can be cropped from the original image, and the position, size and human body attribute analysis results of the human body region can also be returned. Furthermore, the human body attribute analysis results may include gender, height, weight, etc.

The neural network model corresponding to human body tracking can determine the movement trajectory of user A during the monitoring period based on the original images of user A captured by multiple cameras during the monitoring period. For example, user A and five other users are walking in a smart store together, and five cameras located at different locations in the smart store have captured original images of user A and the other five users. Based on the original images from the five cameras, the movement trajectory of user A (located from the six users) during the monitoring period can be determined.

Through the binding of the head and body to the corresponding neural network model, the original image of only the face (or the original image of only the torso + limbs) can be bound to the corresponding torso + limbs reference image (or face reference image) based on the recognition result of human body detection. For example, when user B walks in a smart store and a camera takes a picture of user B, the original image collected may not point to the face area of user B's face. The body area in the original image can be segmented out and bound to the corresponding face reference image (corresponding to user B) to obtain the recognition result pointing to user B.

The neural network model corresponding to the face quality score can evaluate the quality score of the original image according to the quality evaluation factors (such as the blur of the image, the occlusion of the target area in the image, etc.), and then the image that reaches the quality score threshold can be used for face detection, human body detection, face clustering (used to group the various face areas in an image set according to identity), face retrieval (for example, based on the face area of the original image, compare with N face images in the existing face library, find the most similar face or faces, and give a similarity ranking), etc. For example, since the cameras are set at different locations in the smart store, the shooting angles of different cameras are also different. User C at position A may be photographed by multiple cameras, and the original images captured by some cameras may be limited by angles, so the scores corresponding to the original images of user C are low. If user C shakes his head at this time, there may be blurry images in the original images of user C captured, and these original images with lower quality scores can be filtered.

The corresponding neural network model of posture detection can determine whether the facial posture or body posture in the original image meets the preset standard posture, for example, confirm whether a user is photographed facing the corresponding camera.

In practical applications, the neural network model configured by the neural network model plug-in (corresponding functions, mutual setting order, specific algorithms used, etc.) can be selected according to the needs of the application scenario. For example, a first neural network model used as a segmenter and a second neural network model used as a classifier can be selected. When in use, the original image can be set to be first input into the first neural network model for segmentation to obtain the target image, and then the target image can be input into the second neural network model for classification to obtain the recognition result. Specifically, the first neural network model can perform image segmentation based on MaskRCNN (instance segmentation algorithm). MaskRCNN combines the characteristics of FPN (Feature Pyramid Networks, Feature Pyramid Network Algorithm)/RPN (Region Proposal Network, Region Candidate Network Algorithm)/Classification (Classification Algorithm)/Detection (Detection Algorithm) well, and is suitable for image detection and segmentation. The basic network in the first neural network model can use the residual network ResNet101 to take into account both speed and accuracy. The basic network in the second neural network model can use Inception (a network model structure)-ResNet (residual network)-V2. Of course, the configuration of the neural network model in the neural network model plug-in is not limited to the above, and can be configured taking performance factors into consideration.

In another specific embodiment, the business plug-in may also include at least one selected from the group consisting of a log processing plug-in and a monitoring plug-in, wherein the log processing plug-in is used to process the log generated by the character image recognition system, and the monitoring plug-in is used to monitor the operating status of the character image recognition system.

The log processing module corresponding to the log processing plug-in can be managed in a distributed manner, such as in the form of a distributed database cluster. As shown in FIG8 , the distributed database cluster in the embodiment of this specification can also include a distributed database cluster corresponding to the blockchain network, which can include distributed nodes 801, 808, 803, 804, 805, and 806. The distributed nodes can be connected via a communication link, such as a wired communication link or a wireless communication link.

The monitoring module corresponding to the monitoring plug-in can issue an alarm and display the alarm information to the developer when it detects that the monitoring indicator of a certain monitoring object meets the alarm condition in the alarm rule. The alarm information includes the information of the monitoring indicator and the information of the associated indicator different from the monitoring indicator. The location and time of the abnormality can be effectively located, and the problem can be solved. The setting of the monitoring object is divided into plug-ins, and even further settings can be made for the functional units in a plug-in.

S203: Generate a business code for constructing the person image recognition system based on the configuration script and the configuration parameters of the plug-in to be loaded;

In the embodiment of the present invention, the configuration script may be a SHELL script (a program written using the functions of a shell; shell is a command parser). Through the SHELL script, the required parameters (configuration parameters of the plug-in to be loaded) are input, which can be used to construct the business code of the person image recognition system. For example:

"→Desktop bash generator.sh

Start generating, please enter parameters as prompted

Please enter the project name: test

Whether to use soft decoding, enter true/false, the default is true: true

Select the plugin you want to load

1. Face Detection

2. Face quality score

3. Human body detection

4. Human body tracking

5. Human head and body binding

6. Posture detection

Split by space: 1 2 3 4 5 6

Whether a log processing plug-in is required, enter true/false, the default is false: false

Whether to need monitoring plug-in, enter true/false, the default is true: true

Loading...

Generate base code...

Loading algorithm module...

Loading edge modules...

Generation completed"

In this way, the various functional modules in the character image recognition business process are integrated in the form of plug-ins. The image acquisition module, algorithm module and auxiliary module can be released independently without interfering with each other, ensuring that the update of a single module does not affect the overall business logic, which is easy to manage and maintain.

S204: integrating the business code with the basic framework to construct the person image recognition system;

In an embodiment of the present invention, the business code can be injected into the basic framework to obtain the person image recognition system. Specifically, according to the code injection rules configured in the basic framework, the business code can be injected into the basic framework to obtain a fusion code; the fusion code is compiled to obtain an executable code, and the executable code is run to implement the online application of the person image recognition system. Developers no longer need to write the code corresponding to the basic framework, which can reduce the workload caused by writing a large amount of repeated code and related debugging, testing, etc. in the development process. Developers only need to focus on the business logic itself corresponding to the specific business scenario.

In a specific embodiment, as shown in FIG5 , when the plug-in to be loaded includes the neural network model plug-in, the fusion of the business code and the basic framework to construct the person image recognition system includes:

S501: In response to the received plug-in update instruction, determining an initial network model corresponding to the neural network model plug-in, and acquiring a sample image of a person;

According to the plug-in update instruction, the initial network model can be determined to be a neural network model for face detection and/or a neural network model for body detection. Accordingly, a sample image of a person is obtained.

S502: Based on the person sample image, training the initial network model to obtain a target network model;

The initial network model can be trained based on sample images of people to obtain an intermediate network model; then the labeled test image is input into the intermediate network model for prediction to obtain the prediction result corresponding to the test image; then, based on the test image, according to the difference between the corresponding prediction result and the labeled mark, the model parameters corresponding to the intermediate network model are adjusted until the loss value corresponding to the difference converges to a preset threshold, so as to obtain the target network model.

S503: Based on the target network model, update the neural network model plug-in.

Updating the neural network model plug-in can make the corresponding algorithm module more adaptable to the needs of the application scenario. Integrating the various functional modules in the character image recognition business process in the form of plug-ins has low coupling and dependence between modules during updates, which is easier to maintain. Of course, corresponding parameter adjustments can also be made to the image acquisition plug-in, log processing plug-in, monitoring plug-in, etc. for updating. The updates of each plug-in can also be carried out synchronously, with a low degree of mutual influence. The update of each plug-in is not limited to the parameter adjustment of the preset basic components, etc., but also includes modification and supplementation of the corresponding basic components.

In another specific embodiment, as shown in FIG5 , when the plug-in to be loaded includes the image acquisition plug-in and the neural network model plug-in, the fusion of the business code and the basic framework to construct the person image recognition system further includes:

S601: Based on the image acquisition plug-in, the person image to be processed is acquired;

In practical applications, as shown in Figures 6 and 7, the original video stream can be obtained from a video input device (such as a ball camera, a gun camera, an AI camera); the original video stream is decoded to obtain a video frame, and the video frame includes at least one of the image of the person to be processed. The image acquisition module corresponding to the image acquisition plug-in can pull the original video stream from the camera, decode it to obtain a video frame, and then input the video frame into the algorithm module corresponding to the neural network model plug-in for processing. The character image recognition process performed in the algorithm module may include face detection, face quality score calculation, and human body detection, and the recognition results can be stored in a database for persistent storage.

S602: Based on the neural network model plug-in, the image of the person to be processed is input into each of the neural network models for recognition, so as to obtain each recognition result corresponding to the image of the person to be processed.

For different business scenarios, it may be necessary to combine neural network models pointing to different person image recognition functions in the algorithm module corresponding to the neural network model plug-in. As shown in Figures 6 and 7, the embodiment of this specification proposes the concept of a customized algorithm pipeline to support different business logics with higher flexibility. The four algorithm modules of face detection, face quality filtering, face detection, and face similarity matching can be combined, and the algorithm module pointed to by the corresponding neural network model combination is defined as a face recognition pipeline. Based on the customized algorithm pipeline, you can only focus on the final output result and ignore the intermediate process, which is similar to combining multiple neural network models into a function, giving input and getting output. The algorithm module uses GPU (Graphics Processing Unit) for calculation and processing. Based on the customized algorithm pipeline, the corresponding neural network model combination only needs to copy the corresponding data to the video memory once when in use, rather than copying the data once for each neural network model used, which can greatly improve the operating efficiency, while saving video memory resources and improving performance. Among them, the hardware environment for the operation of the person image recognition system can be an X86 server equipped with a GPU.

Furthermore, the image acquisition module corresponding to the image acquisition plug-in can create a data writing thread, and the data writing thread corresponds to a shared memory queue; the data corresponding to the image of the person to be processed (such as the original data frame) is written into the shared memory queue through the data writing thread. Correspondingly, the algorithm module corresponding to the neural network model plug-in can create a data reading thread, and the data reading thread corresponds to the shared memory queue; the data corresponding to the image of the person to be processed (such as the original data frame) is read from the shared memory queue through the data reading thread; and then the image of the person to be processed is input into each of the neural network models for recognition, so as to obtain each recognition result corresponding to the image of the person to be processed. Communicating by means of a shared memory queue can ensure that the updates of the image acquisition module and the algorithm module will not affect each other.

The data writing thread writes the original data frame into the queue. The data written into the queue can be based on time sequence or based on the classification settings of different video input devices (for example, the camera is set with a corresponding identification mark, and the cameras 1-3 located at the entrance of the smart store correspond to the identification mark A, and the cameras 7-12 located at the cashier of the smart store correspond to the identification mark D). Correspondingly, the data reading thread can read data from the queue according to different dimensions. For example, batches can be processed uniformly according to dimensions such as time or number of images to improve computing efficiency. The existence of a shared memory queue allows the curve graph of the production speed of the image acquisition module and the consumption speed of the algorithm module to have spikes. For example, at a certain moment, the image acquisition module produces too fast. Due to the existence of the queue, the data will not be lost immediately. The algorithm module can slowly keep up with the production speed of the image acquisition module, so that the data input into the neural network model can be smoother.

It can be seen from the technical solutions provided in the above embodiments of this specification that the embodiments of this specification are applied to the development of a person image recognition system. The functional modules in the person image recognition business process are integrated in the form of plug-ins, and the business code for building the person image recognition system is generated based on the configuration script and the configuration parameters of the plug-in to be loaded. The basic framework and the business code are integrated to construct a person image recognition system. It can greatly improve the development efficiency of developers and meet the different development needs of different business scenarios. The form of a plug-in can reduce the workload of repeated development and is also easy to maintain.

The embodiment of the present invention further provides a device for constructing a person image recognition system, as shown in FIG9 , the device includes:

Basic framework acquisition module 910: used to acquire a basic framework for building a person image recognition system;

The plug-in confirmation module 920 to be loaded is used to determine the plug-in to be loaded in response to the received plug-in loading instruction, wherein the plug-in loading instruction carries configuration parameters pointing to at least one business plug-in;

Business code generation module 930: used to generate business code for building the person image recognition system based on the configuration script and the configuration parameters of the plug-in to be loaded;

Fusion module 940: used to fuse the business code with the basic framework to construct the person image recognition system;

The business plug-in includes at least one selected from the group consisting of an image acquisition plug-in and a neural network model plug-in, and the configuration parameters of the neural network model plug-in point to at least one neural network model for character image recognition.

In a specific embodiment, when the plug-in to be loaded includes the neural network model plug-in, the fusion module 940 includes:

A plug-in update instruction receiving unit: used to determine the initial network model corresponding to the neural network model plug-in in response to the received plug-in update instruction, and obtain a sample image of a person;

Model training unit: used for training the initial network model based on the person sample image to obtain a target network model;

Plug-in updating unit: used to update the neural network model plug-in based on the target network model.

It should be noted that the device in the device embodiment and the method embodiment are based on the same inventive concept.

An embodiment of the present invention provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a method for constructing a person image recognition system as provided in the above method embodiment.

Further, FIG. 10 shows a hardware structure diagram of an electronic device for implementing a method for constructing a person image recognition system provided by an embodiment of the present invention, and the device may participate in or include the device provided by an embodiment of the present invention. As shown in FIG. 10 , the device 100 may include one or more (1002a, 1002b, ..., 1002n are used in the figure to illustrate) processors 1002 (the processor 1002 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input/output interface (I/O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power supply and/or a camera. It can be understood by those skilled in the art that the structure shown in FIG. 10 is only for illustration, and it does not limit the structure of the above-mentioned electronic device. For example, the electronic device 100 may also include more or fewer components than those shown in FIG. 10 , or have a configuration different from that shown in FIG. 10 .

It should be noted that the one or more processors 1002 and/or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the electronic device 100 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

The memory 1004 can be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the method in the embodiment of the present invention. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, to implement the above-mentioned method for constructing a person image recognition system. The memory 1004 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1004 may further include a memory remotely arranged relative to the processor 1002, and these remote memories may be connected to the electronic device 100 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The transmission device 1006 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the electronic device 100. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 1006 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the electronic device 100 (or mobile device).

An embodiment of the present invention also provides a storage medium, which can be set in an electronic device to store at least one instruction, at least one program, a code set or an instruction set related to a method for constructing a person image recognition system in a method embodiment. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for constructing a person image recognition system provided in the above method embodiment.

Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

It should be noted that the sequence of the embodiments of the present invention described above is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and electronic device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (12)

1. A method of constructing a character image recognition system, the method comprising:

Acquiring a base frame for constructing a character image recognition system, the base frame indicating a preset scene, the base frame including at least one of: a basic code library, a public class library and a basic service interface;

determining a plurality of plug-ins to be loaded based on configuration parameters, carried in a plug-in loading instruction, indicating each of the plurality of service plug-ins, wherein the plurality of plug-ins to be loaded comprise an image acquisition plug-in and at least two neural network model plug-ins;

Generating service codes for constructing the character image recognition system based on a configuration script, configuration parameters of each of the plurality of plug-ins to be loaded and corresponding use sequences of the at least two neural network model plug-ins, wherein the use sequences are used for representing flow directions of images acquired based on the image acquisition plug-ins among the at least two neural network model plug-ins, the configuration parameters of the neural network model plug-ins indicate functions to be used and/or algorithms to be used of a neural network model for character image recognition, and the neural network model for character image recognition indicates the neural network model for face detection, human body tracking, human body binding, human face quality score and gesture detection;

fusing the business codes and the basic framework to construct the character image recognition system;

Wherein after said fusing said business code with said base frame to construct said character image recognition system, said method further comprises: acquiring a character image to be processed based on the image acquisition plug-in; and under the condition that the use sequence indicates that the image acquired based on the image acquisition plug-in does not have the flow direction between the at least two neural network model plug-ins, the at least two neural network model plug-ins are used for identifying the character image to be processed so as to obtain at least two identification results corresponding to the character image to be processed.

2. The method of claim 1, wherein prior to said fusing the business code with the base frame to construct the character image recognition system, the method further comprises:

Responding to the received plug-in updating instruction, determining an initial network model corresponding to the neural network model plug-in, and acquiring a character sample image;

Training the initial network model based on the character sample image to obtain a target network model;

and updating the neural network model plug-in based on the target network model.

3. The method of claim 1, wherein the acquiring the image of the person to be processed based on the image acquisition plug-in comprises:

obtaining an original video stream from a video input device;

And decoding the original video stream to obtain a video frame, wherein the video frame comprises at least one character image to be processed.

4. The method according to claim 1, characterized in that:

The step of acquiring the image of the person to be processed based on the image acquisition plug-in comprises the following steps:

creating a data writing thread, wherein the data writing thread corresponds to a shared memory queue;

Writing data corresponding to the character image to be processed into the shared memory queue through the data writing thread;

The step of inputting the character image to be processed into each neural network model for recognition based on the at least two neural network model plug-ins to obtain at least two recognition results corresponding to the character image to be processed comprises the following steps:

Creating a data reading thread, wherein the data reading thread corresponds to the shared memory queue;

Reading data corresponding to the character image to be processed from the shared memory queue through the data reading thread;

Inputting the character image to be processed into each neural network model for recognition so as to obtain at least two recognition results corresponding to the character image to be processed.

5. The method of claim 1, wherein the business plug-in further comprises at least one selected from the group consisting of a log processing plug-in for processing logs generated by the people image recognition system and a monitoring plug-in for monitoring an operational status of the people image recognition system.

6. A construction apparatus of a personal image recognition system, the apparatus comprising:

the basic framework acquisition module: a base frame for use in constructing a character image recognition system, the base frame indicating a preset scene, the base frame comprising at least one of: a basic code library, a public class library and a basic service interface;

Plug-in confirmation module to be loaded: the method comprises the steps of determining a plurality of plug-ins to be loaded based on configuration parameters, carried in plug-in loading instructions, indicating each of a plurality of service plug-ins, wherein the plug-ins to be loaded comprise an image acquisition plug-in and at least two neural network model plug-ins;

A service code generation module: generating service codes for constructing the character image recognition system based on a configuration script, configuration parameters of each of the plurality of plug-ins to be loaded and corresponding use sequences of the at least two neural network model plug-ins, wherein the use sequences are used for representing flow directions of images acquired based on the image acquisition plug-ins among the at least two neural network model plug-ins, the configuration parameters of the neural network model plug-ins indicate functions to be used and/or algorithms to be used of a neural network model for character image recognition, and the neural network model for character image recognition indicates neural network models for face detection, human body tracking, human head binding, human face quality score and gesture detection;

and a fusion module: the business codes are fused with the basic framework to construct the character image recognition system;

Wherein the device is further for: acquiring a character image to be processed based on the image acquisition plug-in; and under the condition that the use sequence indicates that the image acquired based on the image acquisition plug-in does not have the flow direction between the at least two neural network model plug-ins, the at least two neural network model plug-ins are used for identifying the character image to be processed so as to obtain at least two identification results corresponding to the character image to be processed.

7. The apparatus of claim 6, further comprising a plug-in update module, the plug-in update module comprising:

Plug-in update instruction receiving unit: the method comprises the steps of responding to a received plug-in updating instruction, determining an initial network model corresponding to the neural network model plug-in, and acquiring a character sample image;

model training unit: training the initial network model based on the character sample image to obtain a target network model;

Plug-in updating unit: and updating the neural network model plugin based on the target network model.

8. The apparatus of claim 6, wherein the acquiring the image of the person to be processed based on the image acquisition plug-in comprises: obtaining an original video stream from a video input device; and decoding the original video stream to obtain a video frame, wherein the video frame comprises at least one character image to be processed.

9. The apparatus according to claim 6, wherein:

The step of acquiring the image of the person to be processed based on the image acquisition plug-in comprises the following steps: creating a data writing thread, wherein the data writing thread corresponds to a shared memory queue; writing data corresponding to the character image to be processed into the shared memory queue through the data writing thread;

The step of inputting the character image to be processed into each neural network model for recognition based on the at least two neural network model plug-ins to obtain at least two recognition results corresponding to the character image to be processed comprises the following steps: creating a data reading thread, wherein the data reading thread corresponds to the shared memory queue; reading data corresponding to the character image to be processed from the shared memory queue through the data reading thread; inputting the character image to be processed into each neural network model for recognition so as to obtain at least two recognition results corresponding to the character image to be processed.

10. The apparatus of claim 6, wherein the business plug-in further comprises at least one selected from the group consisting of a log processing plug-in for processing logs generated by the people image recognition system and a monitoring plug-in for monitoring an operational status of the people image recognition system.

11. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the method of constructing the persona image recognition system of any one of claims 1-5.

12. A computer-readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by a processor to implement a method of constructing a persona image recognition system of any of claims 1-5.