CN118397664A - Method and device for detecting fingerprint residues of shell - Google Patents

Abstract

The embodiment of the present application discloses a method and device for detecting fingerprint residue on a shell, the method may include: obtaining a fingerprint image, the fingerprint image being an image obtained by photographing a fingerprint on the surface of a target shell; performing image processing on the fingerprint image to determine a target area in the fingerprint image and a fingerprint area in the target area where fingerprint patterns are collected; determining the contrast between the fingerprint area and the target area based on the pixel information of the fingerprint area and the pixel information of the target area; judging the fingerprint residue performance of the target shell based on the area of the fingerprint area and the contrast. The embodiment of the present application can effectively use more information in the image to determine the fingerprint residue performance, and improve the objectivity and accuracy of the detection scheme of fingerprint residue on the shell.

Description

A method and device for detecting fingerprint residue on housing

Technical Field

The present application relates to the field of fingerprint recognition technology, and in particular to a method and device for detecting fingerprint residue on a housing.

Background technique

Fingerprint recognition technology is a multidisciplinary field involving image processing, pattern recognition and information security, and aims to identify information by identifying an individual's unique fingerprint image. In this field, the fingerprint residue test of the shell is mainly used to test the material's anti-fingerprint and anti-fouling properties.

At present, the detection of fingerprint residue on the shell is mainly judged by staff observing the fingerprint image to determine the anti-fingerprint performance of the shell. Different staff may have different judgment criteria for fingerprint residue on the shell, and fail to make full use of other information in the image, resulting in limited objectivity and accuracy of the test results. It is necessary to provide a detection method for fingerprint residue on the shell, which can effectively use more information in the image to determine the fingerprint residue performance and improve the objectivity and accuracy of the detection scheme of fingerprint residue on the shell.

Summary of the invention

The technical problem to be solved by the embodiments of the present application is to provide a method and device for detecting fingerprint residues on a shell, which can effectively use more information in an image to determine fingerprint residue performance and improve the objectivity and accuracy of the detection scheme for fingerprint residues on the shell.

In a first aspect, an embodiment of the present application provides a method for detecting fingerprint residue on a shell, which may include: acquiring a fingerprint image, wherein the fingerprint image is an image obtained by photographing a fingerprint on the surface of a target shell; performing image processing on the fingerprint image to determine a target area in the fingerprint image and a fingerprint area in the target area where fingerprint patterns are collected; determining a contrast ratio between the fingerprint area and the target area based on pixel information of the fingerprint area and pixel information of the target area; and judging the fingerprint residue performance of the target shell based on the area of the fingerprint area and the contrast ratio.

In the existing detection methods of fingerprint residues on shells, the anti-fingerprint performance of the shell is usually evaluated by staff observing the fingerprint clarity, fingerprint visibility and other visible data of the fingerprint image. This method is only based on the subjective judgment of the staff and lacks full use of other information in the image, resulting in low objectivity and accuracy of the detection results of residual fingerprints on the shell. In response to this technical problem, a more objective detection method of fingerprint residues on the shell is provided in the embodiment of the present application. After determining the target area based on the fingerprint image and the fingerprint area including the fingerprint lines in the target area, the fingerprint area is placed in the target area for comparison to obtain the contrast between the fingerprint area and the target area. Finally, the above contrast and the area of the fingerprint area are compared with the preset standard value to objectively judge the fingerprint residue performance (i.e., anti-fingerprint performance and anti-fouling performance) of the target shell. Compared with the prior art in which the fingerprint residue performance of the shell is judged subjectively by the staff, the fingerprint residue performance can be judged by objectively comparing the data information in the fingerprint image in the embodiment of the present application, that is, the fingerprint area can be judged by the above contrast. The degree of color or brightness difference between the target areas (for example, when comparing the fingerprint residue performance on shells of different materials, because the color difference is large, the difference in fingerprint residue performance between different shells can be more accurately judged by the contrast of RGB channel values between the fingerprint area and the target area, or, when comparing the fingerprint residue performance on shells of the same material, because the color difference is small and the grayscale information contains more feature information, the difference in fingerprint residue performance between different shells can be more accurately judged by the contrast of grayscale values between the fingerprint area and the target area), and the number or density of fingerprint residues can be judged by the area of the above fingerprint area, so as to more accurately and objectively judge the fingerprint residue performance, thereby improving the accuracy and objectivity of the detection of fingerprint residues on the shell.

In a possible implementation, if the pixel information includes RGB channel values; the determination of the contrast between the fingerprint area and the target area based on the pixel information of the fingerprint area and the pixel information of the fingerprint image includes: determining the sum of each channel value in the RGB channel values of the fingerprint area and the sum of each channel value in the RGB channel values of the target area; comparing the sum of each channel value in the RGB channel values of the fingerprint area with the sum of each channel value in the RGB channel values of the target area to obtain the contrast of each channel. In the embodiment of the present application, the color difference information between the fingerprint area and the target area can be obtained by comparing the sum of each channel value in the RGB channel values of the fingerprint area and the target area. The greater the color difference between the fingerprint area and the target area, the higher the fingerprint residue degree; further, when comparing the fingerprint residue performance on the shells of different materials, because the shells of different materials usually have obvious color differences, the difference in fingerprint residue performance between different shells can be more accurately judged by the contrast of the RGB channel values of the fingerprint area and the target area, thereby improving the accuracy of the shell fingerprint residue test results when comparing shells of different materials.

In a possible implementation, the image processing of the fingerprint image to determine the target area in the fingerprint image and the fingerprint area in the target area where the fingerprint lines are collected includes: determining the target area based on the shooting parameters of the fingerprint image; converting the image of the target area into a first grayscale image; performing image feature enhancement processing on the first grayscale image to obtain a second grayscale image; and binarizing the second grayscale image to determine the fingerprint area in the target area that meets the fingerprint features. In the embodiment of the present application, after determining the target area in the fingerprint image by shooting parameters, the image of the target area can be converted into a grayscale image, and image feature enhancement processing can be performed on it to effectively remove noise and highlight fingerprint features, and the pixel values in the grayscale image after denoising processing can be converted into black and white colors by binarization processing, and the area in the black pixel value that meets the fingerprint features is determined as the fingerprint area, and the fingerprint area containing fingerprint lines is accurately located, thereby improving the objectivity of the shell fingerprint residue detection and the accuracy of the results.

In a possible implementation, the image processing of the fingerprint image to determine the target area in the fingerprint image and the fingerprint area in the target area where the fingerprint lines are collected includes: determining the target area based on the shooting parameters of the fingerprint image; converting the image of the target area into a third grayscale image; partitioning the third grayscale image based on the shooting light source and the area of the third grayscale image to obtain multiple sub-areas; performing image feature enhancement processing on each sub-area in the third grayscale image to obtain a fourth grayscale image; binarizing the fourth grayscale image and merging the connected areas shared by each sub-area in the fourth grayscale image after the binarization processing to determine the fingerprint area. In the embodiment of the present application, after dividing the image into multiple sub-areas, threshold segmentation is performed for each sub-area, and the shared connected areas are merged to reduce the error caused by uneven light sources, thereby further improving the accuracy of extracting the fingerprint area. Specifically, in the embodiment of the present application, the target area is first determined by the shooting parameters in the fingerprint image, and after the image of the target area is converted into a grayscale image, the grayscale image is divided into multiple sub-areas, so that each sub-area is processed independently, thereby reducing the error caused by the unevenness of the light source; further, by performing threshold segmentation and feature enhancement processing (such as denoising and morphological processing) on each sub-area, the characteristics of the fingerprint pattern are highlighted, thereby improving the recognition accuracy of the fingerprint area; finally, by merging the shared connected areas, a continuous and complete fingerprint area is obtained, which reduces the impact of the uneven light source while ensuring the accuracy of the detection result of the fingerprint residue on the shell.

In a possible implementation, if the pixel information includes a grayscale value; the determination of the contrast between the fingerprint area and the target area based on the pixel information of the fingerprint area and the pixel information of the target area includes: determining the sum of the grayscale values of the fingerprint area and the sum of the grayscale values of the target area; and determining the contrast as the comparison result of the sum of the grayscale values of the fingerprint area and the sum of the grayscale values of the target area. In the embodiment of the present application, by comparing the sum of the grayscale values of the fingerprint area with the sum of the grayscale values of the target area, if the greater the grayscale difference between the fingerprint area and the target area, the higher the fingerprint residue degree; further, when testing the same shell material, because the color difference between shells of the same material is small, and the grayscale information contains more feature information, the difference between the fingerprint residue performance can be judged by comparing the grayscale difference information between different shells. Therefore, the difference in fingerprint residue performance between the same shell materials can be more accurately and objectively judged through the above grayscale difference information.

In a possible implementation, the fingerprint image is processed to determine the target area in the fingerprint image and the fingerprint area in the target area where the fingerprint pattern is collected, including: determining the target area based on shooting parameters; converting the image of the target area into a first grayscale image; extracting the background area from the first grayscale image and removing the background area in the first grayscale image; performing image feature enhancement processing on the first grayscale image after removing the background area to obtain a second grayscale image; and binarizing the second grayscale image to determine the fingerprint area in the target area that meets the fingerprint feature. When performing fingerprint residue detection on shells of the same material, because the color difference between shells of the same material is small, and the grayscale information contains more feature information, the background extraction and removal steps can be added in the image processing (such as image grayscale calculation, image feature enhancement processing and binarization processing, etc.) process to remove the color information in the image and effectively reduce the interference of non-fingerprint areas, and more accurately judge the difference between shells of the same material based on the feature information contained in the grayscale information when performing subsequent image processing, so as to improve the objectivity and accuracy of the detection scheme of fingerprint residue on the shell.

In a possible implementation, the image processing of the fingerprint image to determine the target area in the fingerprint image and the fingerprint area in the target area where fingerprint lines are collected includes: determining the target area based on shooting parameters; converting the image of the target area into a third grayscale image; partitioning the third grayscale image based on the shooting light source and the area of the third grayscale image to obtain multiple sub-areas; extracting the background area from each sub-area in the third grayscale image, and removing the background area in each sub-area in the third grayscale image; performing image feature enhancement processing on each sub-area in the third grayscale image after removing the background area to obtain a fourth grayscale image; and determining the fingerprint area by binarizing the fourth grayscale image and merging the connected areas shared by each sub-area in the fourth grayscale image after the binarization processing. In the embodiment of the present application, the error caused by uneven light source can be reduced through image processing methods such as adaptive partitioning, threshold segmentation, and common connected area merging, and background extraction and removal steps are added in the above image processing process to remove color information. In the subsequent image processing, the difference between shells of the same material can be more accurately judged based on the feature information contained in the grayscale information, thereby improving the objectivity and accuracy of the detection scheme for fingerprint residues on the shell.

In a possible implementation, the image feature enhancement process includes: performing denoising on the corresponding grayscale image through image filtering; and performing morphological processing on the grayscale image after denoising. In the embodiment of the present application, image filtering can be used to reduce the noise in the image and improve the quality and clarity of the image; further, morphological processing can be used to further repair the incomplete structure and shape in the image and enhance the features of the fingerprint area in the image, so as to more accurately extract the fingerprint area in the subsequent image processing process, thereby improving the accuracy of the detection scheme for fingerprint residues on the shell.

In a possible implementation, the morphological processing of the grayscale image after denoising includes: removing the uniform background area in the grayscale image after denoising to obtain the foreground area; removing the peripheral area in the foreground area to obtain the image after the peripheral area is removed; the uniform background includes uniform light source and/or uniform texture, and the peripheral area is the area in the foreground area except the connected area where the pixel values are concentrated; the image after the peripheral area is removed is morphologically processed. In the embodiment of the present application, before morphological processing, the uniform background area (for example, the background with uniform light source and/or uniform texture) in the grayscale image after denoising can be removed, and more accurate fingerprint area recognition can be achieved based on the centroid concentration of the connected area in the grayscale image, that is, determining the area where the fingerprint lines are located, and removing the large block noise outside the area; further, if the connected area in the foreground area is not detected, it can be judged that the fingerprint residue is weak. In addition, the envelope of the area where the fingerprint lines are located can be determined by image segmentation or morphological processing, and the noise outside the envelope can be removed to further optimize the image quality of the target area.

In a possible implementation, the fingerprint residual performance of the target shell is judged based on the area of the fingerprint area and the contrast, including: if the area of the fingerprint area is smaller than a preset area, and the contrast is smaller than a preset contrast, then the fingerprint residual degree of the target shell is judged to be low. Compared with the prior art in which the fingerprint residual performance of the shell is judged subjectively by the staff, in the embodiment of the present application, the fingerprint residual performance can be judged more objectively by comparing the area of the fingerprint area, and the color or grayscale contrast between the fingerprint area and the target area with a preset standard value (such as the standard area of the fingerprint area, or the standard contrast value), so as to improve the accuracy and objectivity of the detection scheme of the fingerprint residue on the shell.

In one possible implementation, when testing and comparing the fingerprint residual performance of multiple shells, the fingerprint images corresponding to different shells can be images obtained by taking pictures under the same shooting conditions, such as the same shooting light source, shooting technique, shooting equipment and other conditions. In the subsequent comparison of the fingerprint residual performance differences of different shells, more objective and accurate comparison results can be obtained, avoiding the influence of data anomalies caused by different shooting conditions.

In a second aspect, an embodiment of the present application provides a detection device for fingerprint residue on a housing, which may include:

A fingerprint image acquisition unit, used to acquire a fingerprint image, wherein the fingerprint image is an image obtained by photographing the fingerprint on the surface of the target housing;

A fingerprint area determination unit is used to perform image processing on the fingerprint image to determine a target area in the fingerprint image and a fingerprint area in the target area where fingerprint patterns are collected;

a contrast determination unit, which determines the contrast between the fingerprint area and the target area based on the pixel information of the fingerprint area and the pixel information of the target area;

The fingerprint residual performance judging unit judges the fingerprint residual performance of the target housing based on the area of the fingerprint region and the contrast.

In a possible implementation manner, if the pixel information includes RGB channel values; the contrast determination unit is specifically configured to:

Determine the sum of each channel value in the RGB channel values of the fingerprint area and the sum of each channel value in the RGB channel values of the target area;

The sum of each channel value in the RGB channel values of the fingerprint area is compared with the sum of each channel value in the RGB channel values of the target area to obtain the contrast of each channel.

In a possible implementation manner, the fingerprint area determination unit is specifically used to:

Determining the target area based on shooting parameters of the fingerprint image;

Converting the image of the target area into a first grayscale image;

Performing image feature enhancement processing on the first grayscale image to obtain a second grayscale image;

The second grayscale image is binarized to determine a fingerprint region in the target region that meets the fingerprint feature.

In a possible implementation manner, the fingerprint area determination unit is specifically used to:

Determining the target area based on shooting parameters of the fingerprint image;

Converting the image of the target area into a third grayscale image;

Performing partition processing on the third grayscale image based on the shooting light source and the area of the third grayscale image to obtain a plurality of sub-areas;

Performing image feature enhancement processing on each sub-region in the third grayscale image to obtain a fourth grayscale image;

The fingerprint region is determined by binarizing the fourth grayscale image and merging the connected regions shared by each subregion in the fourth grayscale image after the binarization process.

In a possible implementation manner, if the pixel information includes a grayscale value; the contrast determination unit is specifically configured to:

Determining the sum of the grayscale values of the fingerprint area and the sum of the grayscale values of the target area;

A comparison result of the sum of the grayscale values of the fingerprint area and the sum of the grayscale values of the target area is determined as the contrast.

In a possible implementation manner, the fingerprint area determination unit is specifically used to:

determining the target area based on shooting parameters;

Converting the image of the target area into a first grayscale image;

Extracting a background area from the first grayscale image, and removing the background area from the first grayscale image;

Performing image feature enhancement processing on the first grayscale image after removing the background area to obtain a second grayscale image;

The second grayscale image is binarized to determine a fingerprint region in the target region that meets the fingerprint feature.

In a possible implementation manner, the fingerprint area determination unit is specifically used to:

determining the target area based on shooting parameters;

Converting the image of the target area into a third grayscale image;

Performing partition processing on the third grayscale image based on the shooting light source and the area of the third grayscale image to obtain a plurality of sub-areas;

Extracting a background region from each sub-region in the third grayscale image, and removing the background region from each sub-region in the third grayscale image;

Performing image feature enhancement processing on each sub-region in the third grayscale image after removing the background region, to obtain a fourth grayscale image;

The fingerprint region is determined by binarizing the fourth grayscale image and merging the connected regions shared by each subregion in the fourth grayscale image after the binarization process.

In a possible implementation manner, the fingerprint area determination unit is specifically used to:

De-noising the corresponding grayscale image through image filtering;

Perform morphological processing on the denoised grayscale image.

In a possible implementation manner, the fingerprint area determination unit is specifically used to:

The uniform background area in the denoised grayscale image is removed to obtain the foreground area;

Removing the peripheral area in the foreground area to obtain an image after the peripheral area is removed; the uniform background includes uniform light source and/or uniform texture, and the peripheral area is an area in the foreground area excluding the connected area where the pixel values are concentrated;

Morphological processing is performed on the image after the peripheral area is removed.

In a possible implementation, the fingerprint residual performance determination unit is specifically configured to:

If the area of the fingerprint region is smaller than the preset area, and the contrast is smaller than the preset contrast, it is determined that the fingerprint residue of the target housing is low.

In a third aspect, an embodiment of the present application provides a computer storage medium for storing computer software instructions used in a housing fingerprint residue detection device provided in the second aspect, which includes a program designed for executing the above aspect.

In a fourth aspect, an embodiment of the present application provides a computer program, which includes instructions. When the computer program is executed by a computer, the computer can execute the process executed in the detection device for detecting fingerprint residue on a shell in the second aspect.

In a fifth aspect, the present application provides an electronic device, the electronic device comprising a processor, a display screen and a sensor, the processor being the processor involved in any one of the implementations in the second aspect, the display screen being the display screen involved in any one of the implementations in the second aspect, and the sensor being the sensor involved in any one of the implementations in the second aspect. The electronic device may also include a communication interface for the terminal to communicate with other devices or a communication network.

In a sixth aspect, the present application provides a smart phone having the function of implementing any one of the detection methods for fingerprint residue on a housing in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

FIG1A is a schematic flow chart of a solution for detecting fingerprint residue on a housing provided in an embodiment of the present application.

FIG. 1B is a schematic diagram of a shooting scene provided in an embodiment of the present application.

FIG. 1C is a schematic diagram of determining fingerprint residue performance of housings of different materials provided in an embodiment of the present application.

FIG. 2A is a schematic diagram of a flow chart of another housing fingerprint residue testing solution provided in an embodiment of the present application.

FIG. 2B is a flowchart of an image grayscale calculation scenario 1 provided in an embodiment of the present application.

FIG. 2C is a schematic diagram of target area determination provided in an embodiment of the present application.

FIG. 2D is a schematic diagram of an inversion calculation provided in an embodiment of the present application.

FIG3A is a flowchart of a second image grayscale calculation scenario provided in an embodiment of the present application.

FIG. 3B is a schematic diagram of an uneven light source provided in an embodiment of the present application.

FIG. 4A is a flowchart of a third image grayscale calculation scenario provided in an embodiment of the present application.

FIG. 4B is a schematic diagram of uniform background extraction provided in an embodiment of the present application.

FIG5A is a flowchart of a fourth image grayscale calculation scenario provided in an embodiment of the present application.

FIG. 6A is a flowchart of an image-to-grayscale calculation and background extraction scenario 1 provided in an embodiment of the present application.

FIG. 7A is a flowchart of a second image-to-grayscale calculation scenario provided in an embodiment of the present application.

FIG8A is a flowchart of a third image-to-grayscale calculation scenario provided in an embodiment of the present application.

FIG9A is a flowchart of a fourth image grayscale calculation scenario provided in an embodiment of the present application.

FIG. 10 is a schematic structural diagram of a device for detecting fingerprint residue on a housing provided in an embodiment of the present application.

FIG. 11 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

Detailed ways

The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process, a processor, an object, an executable file, an execution thread, a program and/or a computer running on a processor. By way of illustration, both applications and computing devices running on a computing device can be components. One or more components may reside in a process and/or an execution thread, and a component may be located on a computer and/or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. For example, a component may communicate through a local and/or remote process according to a signal having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and/or a network, such as the Internet interacting with other systems through signals).

First, some terms in this application are explained to facilitate understanding by those skilled in the art.

(1) Fingerprint connected domain refers to the fingerprint area in the image that is connected and has obvious boundaries. It is a key part of fingerprint recognition technology and involves the structure, features and their mutual relationship of the fingerprint image. The extraction of fingerprint connected domain usually includes feature extraction, edge detection, connected region analysis and other steps, aiming to effectively identify and locate the fingerprint area in the fingerprint image.

(2) Fingerprint image preprocessing refers to a series of processing steps performed on the original fingerprint image before fingerprint recognition or fingerprint feature extraction. These steps are aimed at improving image quality, enhancing fingerprint features, and reducing noise and interference to improve the accuracy and performance of subsequent fingerprint recognition algorithms.

(3) Binarization refers to the process of converting an image into only black and white colors. It involves threshold processing of grayscale images or color images. The binarization process usually includes the stages of selecting a threshold, applying the threshold to segment the image, and generating a binary image. The purpose is to simplify the image data and highlight the contours of the object of interest.

(4) RGB channels refer to the three independent color channels of red, green, and blue used in image processing and computer vision. It involves decomposing each pixel of the image into the intensity values of the three primary colors of red, green, and blue. The processing of RGB channels usually includes color separation, channel operation, and color synthesis, with the aim of achieving precise color control and image enhancement.

(5) Image Grayscale Value refers to the brightness value of each pixel in a grayscale image. It involves the process of converting a color image into a single brightness level. The processing of image grayscale values usually includes grayscale conversion, histogram analysis, and grayscale enhancement, with the aim of simplifying image data and highlighting important visual information.

(6) Residual Fingerprint Area refers to the area covered by the detectable fingerprint pattern left on the surface. It involves measuring and analyzing the degree of fingerprint residue on various surfaces. The calculation process of residual fingerprint area usually includes fingerprint image acquisition, image processing, region segmentation and area calculation, with the purpose of accurately quantifying the coverage of fingerprints.

(7) Connected components refer to the regions in a binary image that are composed of adjacent pixels with the same pixel value. It involves identifying and marking continuous groups of pixels in an image. The processing of connected components usually includes image binarization, connectivity detection, region labeling, and feature extraction, with the goal of segmenting and analyzing different objects in the image.

First, the technical problems to be solved by this application are analyzed and proposed. In the prior art, the technology for detecting fingerprint residues on the housing may include the following solutions:

Solution: The existing detection of fingerprint residue on the shell usually tests the fingerprint area extracted from the fingerprint image, and needs to be performed under the same environment, the same light source illumination and non-reflective conditions.

The above solution is currently a common technology on the market, but it also has the following disadvantages:

Disadvantage 1: Limited accuracy. Existing detection technology for fingerprint residue on housings mainly relies on the staff's visual recognition of the fingerprint area in the fingerprint image, and evaluates the housing's anti-fingerprint performance through data visible to the naked eye such as fingerprint clarity and fingerprint visibility, without fully utilizing the deeper information that may be provided in the fingerprint image. For example, the area of the residual fingerprint, the grayscale value and RGB channel value of the fingerprint area, the grayscale value and RGB channel value of the target area in the fingerprint image, etc. Therefore, relying solely on staff to perform detection through personnel identification may not be able to detect the characteristics of fingerprint residues at a deeper level, resulting in limited accuracy of the detection results and an inability to provide accurate evaluation.

Disadvantage 2: Lack of objectivity. Existing testing schemes for residual fingerprints on housings are usually based on subjective judgments by staff, rather than on objective data to detect the residual fingerprints. When comparing housings of the same material and different materials, the difference in characteristic information is large due to the difference in materials, and the difference between different housings cannot be objectively compared, thus affecting the objectivity of the detection scheme for residual fingerprints on the housing.

In order to solve the problem that the current detection scheme of fingerprint residue on the shell does not meet the actual business needs and achieve the goal of meeting the needs of users and actual business, the technical problems to be solved by this application include one or more of the following two aspects, taking into account the shortcomings of the existing technology:

1. Improve the accuracy of the detection scheme for fingerprint residue on the shell (disadvantage 1). The existing detection method for fingerprint residue on the shell mainly judges the fingerprint residue performance through data visible to the human eye, but fails to make full use of the deeper information that may be provided in the fingerprint image, resulting in insufficient accuracy of the detection result. Therefore, a detection method for fingerprint residue on the shell is needed that can make deeper use of the information in the fingerprint image and judge the fingerprint residue performance based on the above information to improve the accuracy of the detection result.

2. Improve the objectivity of the detection scheme for fingerprint residue on the shell (disadvantage 2). The existing detection scheme for fingerprint residue on the shell is mainly based on subjective judgment of the staff, and there is no targeted treatment when testing shells of the same material and different materials, resulting in small differences between the detection results of different shells, and the comparison results are not objective enough. Therefore, a detection method for fingerprint residue on the shell is needed that can judge the fingerprint residue performance based on objective data and perform targeted treatment on shells of the same or different materials to improve the objectivity of the detection results.

In combination with the detection method of fingerprint residue on the shell provided in the embodiment of the present application, the technical problems raised in the present application are specifically analyzed and solved. Please refer to Figure 1A. Figure 1A is a flow chart of a detection scheme for fingerprint residue on the shell provided in the embodiment of the present application. The method may include the following steps S100-S103.

Step S100: Acquire a fingerprint image.

Specifically, the fingerprint image is an image obtained by photographing the fingerprint on the surface of the target housing. In the embodiment of the present application, the housing may be a structure composed of different shell materials or materials, wherein the housing may include a glass back panel, a screen back panel, a touch panel, and other structures that require quantitative and qualitative judgment of the anti-fingerprint performance and anti-fouling performance of the shell material (or material).

In a possible implementation, when testing and comparing the residual fingerprint performance of multiple shells, the fingerprint images taken based on each shell can be taken under the same shooting conditions, such as the same shooting light source, shooting technique, shooting equipment and other conditions, so that more objective and accurate comparison results can be obtained when comparing the residual fingerprint performance differences of different shells in the subsequent comparison, avoiding the influence of data anomalies caused by different shooting conditions. In the embodiment of the present application, in order to objectively evaluate the residual fingerprint under a fixed shooting environment, the shooting condition can be selected as vertical shooting under a uniform light source to minimize the influence caused by reflections. Exemplarily, an embodiment of the present application provides a method for photographing and processing fingerprint images in a laboratory environment. Please refer to Figure 1B. Figure 1B is a schematic diagram of a photographing scene provided in an embodiment of the present application. Figure 1B may include a picture card bracket 01-A, a uniform light source 01-B, a residual fingerprint housing 01-C and a photographing device 01-D. Assume that the photographing site is a camera assessment and measurement (CMA) laboratory, and the photographing device includes two uniform light sources 01-B and a mobile phone or camera for photographing (i.e., the photographing device 01-D). The mobile phone is set to professional mode for photographing. The specific photographing method is as follows: the housing or the whole machine of the residual fingerprint is vertically placed in the middle of the uniform light source, that is, the residual fingerprint housing 01-C is fixed with the picture card bracket 01-A to ensure that it is vertically placed in the middle of the uniform light source. The photographing device 01-D is located 40 cm in front of the residual fingerprint housing 01-C, and is vertically aligned with the residual fingerprint housing 01-C for photographing, and the two uniform light sources 01-B are respectively arranged on both sides of the housing, and the light source is adjusted to a brightness of D65 1000lux to ensure that the light source of the shooting environment is uniform. In summary, the embodiments of the present application ensure the quality and consistency of the captured images by using a uniform light source and professional shooting equipment in a fixed shooting environment to vertically shoot the residual fingerprint housing or the entire machine, and by precisely adjusting the light source and shooting parameters, minimize the impact of reflections and shadows on the image, thereby improving the accuracy and reliability of fingerprint image processing and analysis.

Step S101: performing image processing on the fingerprint image to determine a target area in the fingerprint image and a fingerprint area in the target area where fingerprint patterns are collected.

Specifically, in the embodiment of the present application, the system first receives a fingerprint image, which may be an image containing fingerprint lines. The system may perform a preprocessing operation on the fingerprint image to enhance the image quality and facilitate subsequent fingerprint area identification. The preprocessing operation may include one or more steps such as image graying, image filtering and denoising, histogram equalization, and binarization processing to more accurately determine the target area and the fingerprint area.

In a possible implementation, the image processing of the fingerprint image to determine the target area in the fingerprint image and the fingerprint area in the target area where fingerprint lines are collected includes: determining the target area based on the shooting parameters of the fingerprint image; converting the image of the target area into a first grayscale image; performing image feature enhancement processing on the first grayscale image to obtain a second grayscale image; and binarizing the second grayscale image to determine the fingerprint area in the target area that meets the fingerprint features. Specifically, in the embodiment of the present application, after determining the target area in the fingerprint image through shooting parameters (for example, assuming that the pixel area of the fingerprint image is 1200*1200, the staff or system can determine the area with the most concentrated fingerprints in the fingerprint image as the target area based on the shooting device, shooting distance and shooting conditions, such as an area with a pixel area of 500*500 in the fingerprint image, which includes the fingerprint area where the fingerprint lines are located), the image of the target area is converted into a grayscale image, and the grayscale image is subjected to image feature enhancement processing (such as denoising and morphological processing), and the grayscale image after the above image feature enhancement processing is binarized (that is, the pixel values in the grayscale image are converted into black and white colors based on a preset threshold), and finally the area in the black pixel value that meets the fingerprint characteristics is determined as the fingerprint area, and the fingerprint residual performance is more accurately evaluated based on the fingerprint area in the subsequent. For the specific scene description corresponding to the image processing in the embodiment of the present application, please refer to the corresponding embodiments in Figures 2B-2D below, which will not be repeated in the embodiment of the present application.

In a possible implementation, the image processing of the fingerprint image to determine the target area in the fingerprint image and the fingerprint area in the target area where the fingerprint lines are collected includes: determining the target area based on the shooting parameters of the fingerprint image; converting the image of the target area into a third grayscale image; partitioning the third grayscale image based on the shooting light source and the area of the third grayscale image to obtain multiple sub-areas; performing image feature enhancement processing on each sub-area in the third grayscale image to obtain a fourth grayscale image; binarizing the fourth grayscale image and merging the connected areas shared by each sub-area in the fourth grayscale image after the binarization processing to determine the fingerprint area. In the embodiment of the present application, after dividing the image into multiple sub-areas, threshold segmentation is performed for each sub-area, and the shared connected areas are merged to reduce the error caused by uneven light sources, thereby further improving the accuracy of extracting the fingerprint area. Specifically, after determining the target area based on the fingerprint image (for example, assuming that the pixel area of the fingerprint image is 1200*1200, the staff or the system can determine the area with the most concentrated fingerprints in the fingerprint image as the target area based on the shooting equipment, shooting distance and shooting conditions, such as an area with a pixel area of 500*500 in the fingerprint image, which includes the fingerprint area where the fingerprint lines are located), the image grayscale of the target area is calculated to obtain a grayscale image, and the grayscale image is divided into multiple sub-areas based on the shooting light source and shooting area of the grayscale image; further, after performing image feature enhancement processing (such as denoising processing and/or morphological processing) on the above-mentioned multiple sub-areas respectively, threshold segmentation is performed on the multiple sub-areas after the image feature enhancement processing through binarization processing to obtain a corresponding fourth grayscale image, wherein the connected area of each sub-area in the above-mentioned fourth grayscale image may include multiple discontinuous fingerprint areas, and the common connected areas can be merged to accurately locate and extract the fingerprint area, thereby reducing the error caused by uneven light source. For the description of the specific scene corresponding to the image processing under the condition of uneven light source in the embodiments of the present application, please refer to the corresponding embodiments in Figures 3A to 5A below, which will not be described in detail in the embodiments of the present application.

In a possible implementation, the image processing of the fingerprint image to determine the target area in the fingerprint image and the fingerprint area in the target area where fingerprint lines are collected includes: determining the target area based on shooting parameters; converting the image of the target area into a first grayscale image; extracting the background area from the first grayscale image and removing the background area from the first grayscale image; performing image feature enhancement processing on the first grayscale image after removing the background area to obtain a second grayscale image; and binarizing the second grayscale image to determine the fingerprint area in the target area that meets the fingerprint features. In the embodiment of the present application, the target area in the fingerprint image is subjected to image processing, including grayscale conversion, background area extraction, image feature enhancement and binarization processing, so as to determine the fingerprint area in the target area where the fingerprint pattern is collected, wherein, after the image of the target area is converted into a grayscale image, the background area is extracted and removed, the color information can be removed, and the interference of the non-fingerprint area can be effectively reduced, and after subsequent image processing, the difference between shells of the same material can be more accurately judged by comparing the grayscale information, so that the subsequent image feature enhancement and binarization processing are more concentrated on the fingerprint area, so as to improve the accuracy and objectivity of the difference in the residual performance of the shells of the same material. For the specific scene description corresponding to the image processing after the background extraction and the uniform light source in the embodiment of the present application, please refer to the embodiment corresponding to Figure 6A below, which will not be repeated in the embodiment of the present application.

In a possible implementation, the fingerprint image is processed to determine the target area in the fingerprint image and the fingerprint area in the target area where the fingerprint lines are collected, including: determining the target area based on shooting parameters; converting the image of the target area into a third grayscale image; partitioning the third grayscale image based on the shooting light source and the area of the third grayscale image to obtain multiple sub-areas; extracting the background area from each sub-area in the third grayscale image, and removing the background area in each sub-area in the third grayscale image; performing image feature enhancement processing on each sub-area in the third grayscale image after removing the background area to obtain a fourth grayscale image; binarizing the fourth grayscale image and merging the connected areas shared by each sub-area in the fourth grayscale image after the binarization processing to determine the fingerprint area. In the embodiment of the present application, the error caused by uneven light source is reduced by adaptive partitioning, threshold segmentation, and merging of shared connected areas, and the accuracy of the fingerprint residue detection scheme of the shell is improved. Specifically, in the embodiment of the present application, after determining the target area in the fingerprint image based on the shooting parameters (for example, assuming that the pixel area of the fingerprint image is 1200*1200, the staff or system can determine the area with the most concentrated fingerprints in the fingerprint image as the target area based on the shooting equipment, shooting distance and shooting conditions, such as an area with a pixel area of 500*500 in the fingerprint image, which includes the fingerprint area where the fingerprint lines are located), by performing background extraction on the image of the target area, removing color information, and obtaining a grayscale image, multiple sub-areas can be obtained by adaptively partitioning the grayscale image, and after subsequent image processing, the difference between shells of the same material can be more accurately judged by comparing the grayscale information; further, the multiple sub-areas after background extraction can be subjected to image feature enhancement, threshold segmentation (that is, binarization of the multiple grayscale images after image feature enhancement) and connected area merging, etc., to reduce the error caused by uneven light source, so that the fingerprint area finally determined is more accurate. In summary, in the embodiments of the present application, the influence of uneven light sources is reduced by adaptively partitioning, threshold segmenting, and merging common connected areas of the image of the target area, and the interference of non-fingerprint areas is reduced by background extraction of multiple grayscale images before image feature enhancement, thereby improving the accuracy and objectivity of the difference in residual performance of fingerprints on shells of the same material. For the specific scene description corresponding to image processing after background extraction and uneven light sources in the embodiments of the present application, please refer to the embodiments corresponding to Figures 7A-9A below, which will not be repeated in the embodiments of the present application.

In a possible implementation, the image feature enhancement process includes: performing denoising on the corresponding grayscale image through image filtering; and performing morphological processing on the grayscale image after denoising. In the embodiment of the present application, image filtering can effectively reduce the noise in the image and improve the quality and clarity of the image; and morphological processing can further improve the structure and shape of the image and enhance the features of the fingerprint area in the image, so as to more accurately extract the fingerprint area in the subsequent image processing process, thereby improving the accuracy of the detection scheme of the fingerprint residue on the shell.

In a possible implementation, the morphological processing of the grayscale image after denoising includes: removing the uniform background area in the grayscale image after denoising to obtain the foreground area; removing the peripheral area in the foreground area to obtain the image after the peripheral area is removed; the uniform background includes uniform light source and/or uniform texture, and the peripheral area is the area in the foreground area other than the connected area where the pixel values are concentrated; and performing morphological processing on the image after the peripheral area is removed. In an embodiment of the present application, before performing morphological processing on the grayscale image after denoising, the uniform background area (for example, the background area with uniform light source and/or uniform texture) in the grayscale image after denoising can be removed, and more accurate fingerprint area recognition can be achieved based on the centroid concentration of the connected area in the grayscale image, that is, determining the area where the fingerprint lines are located, and removing the large block noise outside the area; further, if the connected area in the foreground area is not detected, it can be judged that the fingerprint residue is weak. In addition, the envelope of the area where the fingerprint lines are located can be determined by image segmentation or morphological processing, and the noise outside the envelope can be removed.

Step S102: Determine the contrast between the fingerprint area and the target area based on the pixel information of the fingerprint area and the pixel information of the target area.

Specifically, based on whether the material of the shell to be compared is the same, different pixel information of the target area and the fingerprint area can be compared. For example, if the material of the shell to be compared is the same, the fingerprint residual performance of the shell can be more accurately judged by comparing the grayscale values of the target area and the fingerprint area; or, if the material of the shell to be compared is different, the fingerprint residual performance of the shell can be more accurately judged by comparing the RGB channel values of the target area and the fingerprint area.

In a possible implementation, if the pixel information includes RGB channel values; the determination of the contrast between the fingerprint area and the target area based on the pixel information of the fingerprint area and the pixel information of the target area includes: determining the sum of each channel value in the RGB channel values of the fingerprint area and the sum of each channel value in the RGB channel values of the target area; comparing the sum of each channel value in the RGB channel values of the fingerprint area with the sum of each channel value in the RGB channel values of the target area to obtain the contrast of each channel. In the embodiment of the present application, the color difference information between the fingerprint area and the target area can be obtained by comparing the sum of each channel value in the RGB channel values of the fingerprint area and the target area, thereby improving the accuracy of the shell fingerprint residue test result when comparing shells of different materials. Specifically, after first obtaining the pixel information (i.e., RGB channel values) of the target area and the fingerprint area, the sum of each channel value in the RGB channel values of the fingerprint area and the sum of each channel value in the RGB channel values of the target area are determined. The system can compare the sum of the channel values of the same type in the RGB channel values of the fingerprint area and the target area to obtain the contrast of each of the three RGB channels. Based on the contrast, the degree of color difference between the fingerprint area and the target area can be judged (the greater the degree of color difference between the fingerprint area and the target area, the higher the degree of fingerprint residue), thereby obtaining more accurate results in the subsequent judgment of fingerprint residue performance. In addition, when testing shells of different materials, because shells of different materials usually have obvious color differences, the above-mentioned color difference information can be used to more accurately and objectively judge the difference in fingerprint residue performance between different shell materials. For the specific scene description corresponding to the contrast of the RGB channel values of the fingerprint area and the target area in the embodiment of the present application, please refer to the embodiments corresponding to Figures 2A-5A below, which will not be repeated in the embodiments of the present application.

In a possible implementation, if the pixel information includes a grayscale value; the determination of the contrast between the fingerprint area and the target area based on the pixel information of the fingerprint area and the pixel information of the target area includes: determining the sum of the grayscale values of the fingerprint area and the sum of the grayscale values of the target area; and determining the contrast as the result of comparing the sum of the grayscale values of the fingerprint area and the sum of the grayscale values of the target area. In the embodiment of the present application, the grayscale difference between the fingerprint area and the target area can be determined by comparing the sum of the grayscale values of the fingerprint area with the sum of the grayscale values of the target area. For example, when testing the same shell material, because the color difference between shells of the same material is small, and the grayscale information contains more feature information, the difference between the fingerprint residual performance can be judged by comparing the grayscale difference information between different shells, thereby improving the accuracy of judging the fingerprint residual performance of the shell. Specifically, in the embodiment of the present application, the sum of the grayscale values of the fingerprint area and the sum of the grayscale values of the target area can be determined through the pixel information (i.e., grayscale value) of the target area and the fingerprint area in the fingerprint image, and compared to determine the degree of grayscale difference between the fingerprint area and the target area (for example, when the same shell material is tested multiple times, the greater the grayscale difference between the fingerprint area and the target area, the higher the degree of fingerprint residue), so as to obtain more accurate results in the subsequent fingerprint residue performance judgment. In addition, when testing the same shell material, because the shells of the same material have smaller color differences and more feature information in the grayscale information, the above-mentioned grayscale difference information can be used to more accurately and objectively judge the difference in fingerprint residue performance between the same shell materials. For the specific scene description corresponding to the grayscale value contrast ratio of the fingerprint area and the target area in the embodiment of the present application, please refer to the embodiments corresponding to Figures 6A-9A below, which will not be repeated in the embodiment of the present application.

Step S103: judging the fingerprint residue performance of the target housing based on the area of the fingerprint region and the contrast.

Specifically, in the embodiment of the present application, the system first obtains the area and contrast information of the fingerprint area. The area of the fingerprint area can be calculated by the number of pixels of the fingerprint area extracted in the previous step, and the size of the area reflects the coverage of the fingerprint area in the entire target area; the contrast can be determined by calculating the difference in grayscale value or RGB channel value between the fingerprint area and the target area, and the contrast reflects the significance and clarity of the fingerprint area. After obtaining the area and contrast of the fingerprint area, the system can analyze these indicators to determine the fingerprint residual performance of the target shell. Exemplarily, the system can pre-set one or more thresholds or standards to evaluate the degree of fingerprint residue. For example, the area threshold and contrast threshold can be pre-set by the system or staff. When the system detects that the area and contrast of the fingerprint area exceed the predetermined threshold at the same time, it can be judged that the target shell has obvious fingerprint residue (that is, the fingerprint residue is high).

In a possible implementation, the fingerprint residual performance of the target shell is judged based on the area of the fingerprint area and the contrast, including: if the area of the fingerprint area is less than the preset area and the contrast is less than the preset contrast, then the fingerprint residual degree of the target shell is judged to be low. In the embodiment of the present application, if the area of the fingerprint area is less than the preset area, and the contrast between the fingerprint area and the target area (for example, if the shells of the same material are compared, the contrast is the contrast of the grayscale value, or if the shells of different materials are compared, the contrast is the contrast of the RGB channel value) is less than the preset contrast, then the fingerprint residual degree of the target shell is judged to be low, and the fingerprint residual situation is more accurately evaluated by the size of the fingerprint area and the color or grayscale difference with the target area, so as to improve the accuracy and objectivity of the detection scheme of the shell fingerprint residue. For example, see Figure 1C, Figure 1C is a schematic diagram of judging the fingerprint residual performance of a shell of different materials provided in the embodiment of the present application, as shown in Figure 1C, Figure 1C shows the fingerprint residual images of samples of different materials and the corresponding area and contrast standards, and Figure 1C contains three main material categories: PC shell material, touch panel and screen (such as a flat flexible screen). The fingerprint images under each category show the fingerprint residue of the material sample under standard light source and shooting conditions; for PC shell material, fingerprint image 1 shows the fingerprint residue taken under standard light source conditions. The system can extract the fingerprint area through image processing algorithm and calculate the area and contrast of the fingerprint area (for example, the contrast of the RGB channel value of the fingerprint area and the target area in fingerprint image 1); the fingerprint image 2 and fingerprint image 3 corresponding to the touch panel and the screen respectively can refer to the description of the embodiment corresponding to fingerprint image 1, which will not be repeated here; as shown in Figure 1C, if the fingerprint residue area in one or more fingerprint images of fingerprint image 1, fingerprint image 2 and fingerprint image 3 is detected to be less than 2000 square pixels, and the contrast should be less than 30%, it is judged that the shell corresponding to the fingerprint image has good anti-fingerprint performance and anti-fouling performance. Through the application on different shell materials, the examples provided in the embodiments of this application can verify the universality and effectiveness of the detection method of shell fingerprint residue in a variety of practical application scenarios.

Optionally, in step S101 of the above method, when fingerprint residues are detected on multiple different shells, based on whether the material of the shells is the same, the image processing may include two different processes, specifically referring to FIG2A, FIG2A is a flow chart of another shell fingerprint residue test scheme provided in an embodiment of the present application; FIG2A may include (i) image grayscale calculation 201 and (ii) image conversion grayscale calculation and background extraction 202. Based on different processes, the pixel information when judging the contrast between the fingerprint area and the target area is also different. For example, (i) image grayscale calculation 201 is when comparing the fingerprint residue of shells of the same material, the RGB channel of the fingerprint area and the target area is used to compare the fingerprint residue of the shells. The fingerprint residue is determined by the contrast of the sum of each channel value in the channel value and the area of the fingerprint area; (ii) image to grayscale calculation and background extraction 202 is to obtain more grayscale information in the fingerprint area by an additional background extraction step when comparing the fingerprint residues of shells of different materials, and the fingerprint residue is determined by the contrast of the sum of the grayscale values of the fingerprint area and the target area, and the area of the fingerprint area. Among them, (i) image grayscale calculation 201 and (ii) image to grayscale calculation and background extraction 202 can include one or more of the four scenes of scene 1, scene 2, scene 3 and scene 4 in Figure 2A during specific implementation, and can specifically include the following scene descriptions:

（i） Image grayscale calculation 201.

Scene 1: The light source is uniform and the uniform background is not removed.

Specifically, please refer to Figure 2B, which is a flowchart of an image grayscale calculation scenario 1 provided in an embodiment of the present application. The process may include the following steps S2001-S2004.

Step S2001: Fingerprint image capture and input.

Specifically, a fingerprint image is obtained, which is a fingerprint image obtained by photographing the fingerprint on the surface of the target shell. The fingerprint images corresponding to different shells can be images obtained by photographing under the same shooting conditions, such as the same shooting light source, shooting technique, shooting equipment and other conditions, so that a more objective and accurate comparison result can be obtained when comparing the fingerprint residual performance differences of different shells in the subsequent comparison, avoiding the influence of data anomalies caused by different shooting conditions.

Step S2002: extracting the region of interest (RIO).

Specifically, after acquiring the fingerprint image, the target area and the fingerprint area with fingerprint patterns collected in the target area can be extracted from the fingerprint image by performing RIO area extraction on the fingerprint image, wherein step S2002 may include the following steps S2002A-S2002D.

Step S2002A: Image grayscale calculation.

Specifically, before the image grayscale calculation, the target area is first determined by the shooting parameters of the fingerprint image (for example, assuming that the pixel area of the fingerprint image is 1200*1200, the staff or system can determine the area with the most concentrated fingerprints in the fingerprint image as the target area based on the shooting equipment, shooting distance and shooting conditions, such as an area with a pixel area of 500*500 in the fingerprint image, which includes the fingerprint area where the fingerprint lines are located). The image grayscale calculation is used to convert the fingerprint image from a color image to a grayscale image, which only contains brightness information and removes color information. Exemplarily, the grayscale calculation can adopt the weighted average method, that is, the grayscale value is calculated according to the brightness values of the three RGB channels of the image. For example, the grayscale calculation formula can be: grayscale value = 0.299 R + 0.587 G + 0.114*B. The grayscale calculation formula is obtained by weighted calculation based on the visibility of different colors to the human eye, so that the RGB channel information of each pixel in the fingerprint image (usually composed of three channels of red, green, and blue) is synthesized into a grayscale value, and based on the grayscale value, the fingerprint image is converted into a grayscale image including only grayscale information. The above grayscale calculation formula is only one possible implementation method, and other implementation methods may also be used, which is not limited in the embodiments of the present application. In addition, the target area determined by the shooting parameters of the fingerprint image can be seen in Figure 2C. Figure 2C is a target area determination schematic diagram provided in an embodiment of the present application. Figure 2C may include fingerprint image schematic diagram 02-A and image schematic diagram 02-B of the target area. As shown in fingerprint image schematic diagram 02-A, it is assumed that the pixel area of fingerprint image schematic diagram 02-A is 1500*1500, wherein the white line frame is the area where the fingerprint is located; the staff or the system can determine the area with the most concentrated fingerprints in the fingerprint image schematic diagram 02-A as the target area based on the shooting equipment, shooting distance and shooting conditions, as shown in the image schematic diagram 02-B of the target area, the pixel area of the image schematic diagram 02-B of the target area is 150*150, so that the image schematic diagram 02-B of the target area also includes the white line frame (i.e., the area where the fingerprint is located), and greatly reduces other areas that do not contain valid information. Subsequently, the fingerprint area and fingerprint residual performance can be more accurately determined based on the target area.

Step S2002B: Image filtering.

Specifically, after the image of the target area is converted into a grayscale image, the grayscale image can be denoised by image filtering. Exemplarily, in the embodiment of the present application, the grayscale image can be denoised by log-Gabor filter and/or median filter. The log-Gabor filter is used to extract fingerprint pattern features in the image and enhance the details and edge information in the image; the median filter is a nonlinear filtering method, which is mainly used to remove noise (such as salt and pepper noise) in the image while retaining the edge details of the image. Through the above-mentioned image filtering process, the interference and noise in the image can be preliminarily reduced, and the clarity and continuity of the fingerprint pattern can be improved.

Step S2002C: mathematical morphology calculation.

Specifically, after the grayscale image is denoised by image filtering, the denoised grayscale image can be subjected to morphological processing (i.e., mathematical morphological calculation), and the morphological processing can include corrosion, expansion, opening operation, closing operation, etc. For example, in the embodiment of the present application, the holes in the fingerprint lines in the denoised grayscale image can be filled by the expansion operation to make it more complete; at the same time, the small holes or breaks in the fingerprint lines can be eliminated by the closing operation, thereby improving the continuity and clarity of the grayscale image and further strengthening the fingerprint features in the image. The above-mentioned mathematical morphological calculation is only one possible implementation method, and other implementation methods can also be used, which are not limited in the embodiment of the present application.

Step S2002D: Adaptive Otsu's Method (OSTU) is used to binarize and extract the fingerprint position.

Specifically, after the grayscale image is processed by image filtering and mathematical morphology calculation, the system can use adaptive OSTU binarization to extract the fingerprint position (i.e., fingerprint area) in the grayscale image. Adaptive OSTU binarization is an automatic threshold selection technology based on grayscale histogram. Its principle is to determine the optimal segmentation threshold by maximizing the inter-class variance, thereby segmenting the image into two parts: foreground (fingerprint pattern) and background. This method can not only adapt to the grayscale distribution characteristics of different images, but also be flexibly adjusted according to the characteristics of the local area, so that the fingerprint location information can be extracted more accurately. In the embodiment of the present application, the purpose of adaptive OSTU binarization is to segment the image into two parts: foreground and background. Therefore, the output is an image containing only black and white pixel values, black (representing the background) and white (representing the foreground), that is, the grayscale image will become a binary image to indicate the presence or absence of an object. Through adaptive OSTU binarization, the system can dynamically adjust the threshold to ensure that the best segmentation effect can be obtained in different areas. Even if there is uneven illumination, noise interference, etc. in the image of the target area, the system can still accurately extract the fingerprint location information.

Step S2003: directly calculating the area of the fingerprint region.

Specifically, in the embodiment of the present application, the area of the fingerprint area can be calculated based on the binary fingerprint area (i.e., fingerprint position) extracted in step S2002, and the area of the fingerprint area is output as the residual fingerprint area. For example, the system first scans the fingerprint area after binarization. Because the black pixels in the fingerprint area image represent the fingerprint lines, the area of the fingerprint area can be obtained by counting the number of all black pixels in the fingerprint area, wherein the statistical method includes but is not limited to row-by-row scanning, column-by-column scanning, automatic extraction algorithm, and regional growth algorithm. In addition, in the process of calculating the area of the fingerprint area, because the resolution of different images may be different, the number of pixels corresponding to the fingerprint area of the same area in images of different resolutions is also different. Therefore, when calculating the area, the system can convert by introducing the resolution information of the image, for example, by converting the pixel area to the actual area through the pixel density (pixels per inch, DPI) to obtain the area of the fingerprint area.

Step S2004: Place the fingerprint position into the original image and calculate the fingerprint contrast of the RGB channels respectively.

Specifically, the fingerprint position extracted in step S2002 can be placed in the image of the target area (i.e., the image of the target area in step S2001), the contrast between the RGB channel values of the fingerprint area and the target area can be calculated, and the fingerprint residual contrast of the three RGB channels can be output; further, when detecting the fingerprint residual performance, after the user or staff touches the shell material with a finger (i.e., leaves a fingerprint mark on the shell material), the user or staff can wait for a certain time (e.g., 5 minutes) and wipe the area contacted by the finger (i.e., the fingerprint area), and then manually check and judge the fingerprint residual performance, or shoot the shell material, and judge the fingerprint residual performance of the fingerprint image shot this time through the embodiments described in steps S2001-S2005, so as to more accurately determine the fingerprint residual performance; or, in the embodiment of the present application, the user or staff touches the shell material with a finger (i.e., leaves a fingerprint mark on the shell material), and waits for a certain time (e.g., 3 minutes), and then manually check and judge the fingerprint residual performance, or shoot the shell material, and judge the fingerprint residual performance of the fingerprint image shot this time through the embodiments described in steps S2001-S2005, so as to more accurately determine the fingerprint residual performance. For example, when testing shells of different materials, because shells of different materials usually have obvious color differences, the difference in fingerprint residual performance between different shell materials can be judged more accurately and objectively through the color difference information (i.e., the contrast of RGB channel values), and the contrast of the above three RGB channels can be calculated by the contrast formula. For details, please refer to Figure 2D, which is an inversion calculation schematic diagram provided in an embodiment of the present application. Figure 2D may include a first inversion calculation schematic diagram 02-C and a second inversion calculation schematic diagram 02-D; as shown in the first inversion calculation schematic diagram 02-C , assuming that the overall area of the target area is A, the area of the fingerprint area within the target area is B, and the area of other areas outside the fingerprint area is A-B; the average R channel value in area B is β, and the average R channel value in area A-B is α. In general, the color depth of the fingerprint area is generally greater than that of the area other than the fingerprint area (that is, the color of area B is darker than that of area A-B), and the dark grayscale statistics are that the darker the color, the lower the grayscale. Therefore, the overall image can be inverted, that is, the first inversion calculation diagram 02-C is converted into the second inversion calculation diagram 02-D. At this time, β is greater than α, and A is greater than B.

At this time, the calculated contrast index can be expressed as the sum of the R channel values of the image in area B is greater than the sum of the R channel values of the image in area A. The contrast formula can be expressed as:

After determining the R channel value contrast between the fingerprint area and the target area (i.e., the R channel value contrast between the B area and the A area) based on the above contrast formula, the G channel value and the B channel value in the RGB channel can be substituted into the above contrast formula respectively to calculate the fingerprint residual contrast of the three RGB channels respectively.

First, the monotonicity analysis of the simplified formula is:

(1) When the area of A is fixed (when taking photos in real life, the area of A also needs to be fixed using fixed equipment and distance), contrast is positively correlated with B;

(2) When A and B (residual area) are fixed, contrast and Negative correlation, the larger β is, the smaller α is. The smaller it is, the greater the overall contrast (the greater the difference between β and α, the greater the contrast (significant contrast), which also conforms to the subjective meaning of contrast);

Secondly, analyze the influencing factors and correlations of several variables in the formula:

Overall area A: related to the shooting device, shooting conditions (distance L, notebook thickness d), and the selected fingerprint area ROI.

Fingerprint area (dirty area) B: B is the fingerprint area extracted by the automatic extraction algorithm. B is related to the fingerprint algorithm extraction accuracy. B is related to the average grayscale depth β of the fingerprint and the average grayscale depth α of the background.

Background average grayscale α and fingerprint average depth β: related to material (Material), shooting equipment (device), shooting light source (light-source), and fingerprint residue condition (condition).

Finally, in special cases, special values are analyzed:

Assume that the fingerprint detection area B is fixed (if the fingerprint can be detected, β>α), the greater the difference between β and α, the greater the contrast.

Scene 2: The light source is uneven and the uniform background is not removed.

Specifically, please refer to Figure 3A, which is a flowchart of an image grayscale calculation scenario 2 provided in an embodiment of the present application. The process may include the following steps S3001-S3004.

Step S3001: Fingerprint image capture and input.

Step S3002: Region of Interest (RIO) extraction.

Specifically, after acquiring the fingerprint image, the target area and the fingerprint area with fingerprint patterns collected in the target area can be extracted from the fingerprint image by performing RIO area extraction on the fingerprint image, wherein step S3002 can include the following steps S3002A-S3002H.

Step S3002A: Image grayscale calculation.

Step S3002B: adaptively partition the image of the target area.

Specifically, when the light source of the image of the target area is detected to be uneven, the system will adaptively divide the image into multiple areas according to the characteristics of the image of the target area and the light source conditions, so as to better adapt to the local features of the image. For example, please refer to Figure 3B, which is a schematic diagram of uneven light source provided in an embodiment of the present application. As shown in Figure 3B, the black line frame in Figure 3B is the area where the fingerprint is located. In Figure 3B, there are uneven reflections of varying degrees. Calculation according to normal steps may affect the calculation results of the fingerprint area. Therefore, when the system detects that the image of the target area has uneven light source conditions such as the central area brightness being higher than the surrounding area (i.e., the area other than the fingerprint area), the image of the target area can be partitioned according to 3×3 and/or 4×4 grids to more accurately process each area, thereby minimizing processing errors.

Step S3002C: Image filtering.

Step S3002D: mathematical morphology calculation.

Step S3002E: Partition image threshold segmentation.

Specifically, after the image grayscale calculation, adaptive partitioning, image filtering and denoising processing and mathematical morphology calculation are performed on the image of the target area, multiple sub-areas of the grayscale image of the target area can be obtained, and the average brightness of each sub-area is calculated respectively, and an appropriate threshold is set based on the average brightness difference of each sub-area. Finally, based on the threshold, the pixels in each sub-area can be divided into two categories: the pixels above the threshold are classified as fingerprint features, and the pixels below the threshold are classified as background; the fingerprint area can be more accurately extracted from the image of the target area through the threshold segmentation of the above-mentioned binarization processing. Exemplarily, if the average brightness in a sub-area among the multiple sub-areas is high, it may mean that there are fingerprint lines in the sub-area, so a relatively low threshold can be selected for segmentation to ensure that the fingerprint area can be effectively extracted; and for the sub-area with a low average brightness, it may be a background area, so a higher threshold can be selected for segmentation to reduce the interference of the background. Through this method of partitioned image threshold segmentation, the unevenness of the light source can be handled more flexibly, and the accuracy and stability of subsequent fingerprint area extraction can be improved.

Step S3002F: Merge partition images.

Specifically, after performing the threshold segmentation of the partitioned image, it is necessary to merge the segmented regions to obtain the overall fingerprint region. Assuming that the grayscale image of the target area is partitioned into 3×3 and 4×4 (i.e., multiple grayscale images are partitioned one by one), the common connected regions in the 3×3 or 4×4 grids can be merged to merge adjacent fingerprint regions in the partitioned image into a larger connected region, thereby obtaining a more complete fingerprint region.

Step S3002G: Fingerprint connected area determination.

Specifically, after the partition images are merged, the merged fingerprint area can be judged for connected areas to ensure the continuity of the extracted fingerprint area. Exemplarily, the system can judge the connected areas of the merged fingerprint area, identify and retain the connected fingerprint area, and then remove the disconnected parts. For example, the system will detect whether there are broken or discontinuous parts in the merged fingerprint area, and remove these parts, so as to ensure that the final extracted fingerprint area is continuous and improve the continuity and clarity of the extracted fingerprint area.

Step S3002H: directly remove non-fingerprint areas.

Specifically, after the fingerprint connected area judgment is performed, the system will directly eliminate the non-fingerprint area to reduce the amount of calculation for subsequent processing and improve processing efficiency. For example, assuming that in the process of fingerprint connected area judgment, the system has identified some areas that are not connected to the fingerprint, the above-mentioned unconnected areas will be eliminated, and only the part connected to the fingerprint (ie, the fingerprint area) will be retained. In summary, in the embodiment of the present application, multiple sub-areas can be obtained by adaptively partitioning the grayscale image of the target area, and a threshold can be determined based on the light source conditions in each sub-area in the grayscale image of the target area, and the common connected areas of each sub-area can be merged to determine the fingerprint area, which greatly reduces the error caused by the uneven light source, thereby improving the objectivity of the shell fingerprint residue test scheme and the accuracy of the results.

Step S3003: directly calculating the area of the fingerprint region.

Step S3004: Place the fingerprint position into the original image and calculate the fingerprint contrast of the RGB channels respectively.

The specific descriptions of the above-mentioned steps S3001, S3003 and S3004 can refer to the embodiments corresponding to steps S2001, S2003 and S2004 in the aforementioned Figure 2B, and the specific descriptions of steps S3002A, S3002C and S3002D can refer to the embodiments corresponding to steps S2002A-S2002C, which will not be repeated here.

Scene 3: The light source is uniform and the uniform background has been removed.

Specifically, please refer to Figure 4A, which is a flowchart of an image grayscale calculation scenario three provided in an embodiment of the present application. The process may include the following steps S4001-S4004.

Step S4001: Fingerprint image capture and input.

Step S4002: Region of Interest (RIO) extraction.

Specifically, after acquiring the fingerprint image, the target area and the fingerprint area with fingerprint patterns collected in the target area can be extracted from the fingerprint image by performing RIO area extraction on the fingerprint image, wherein step S4002 can include the following steps S4002A-S4002E.

Step S4002A: Image grayscale calculation.

Step S4002B: Image filtering.

Step S4002C: remove the uniform background and eliminate the noise based on the centroid concentration of the connected area.

Specifically, after performing preliminary noise removal on the grayscale image of the target area through image filtering, the uniform background area in the grayscale image can be extracted, and the foreground area can be obtained after removing the uniform background area. The position of the fingerprint area is determined according to the concentration of the center of gravity of the connected area in the foreground area, and the large block of noise that is not in the fingerprint area is removed. Exemplarily, after performing preliminary noise removal on the grayscale image through image filtering, the system can determine the uniform background area by identifying the background area with uniform grayscale value in the grayscale image, and the uniform background area does not contain fingerprint features and the grayscale value changes slightly; when the system extracts the uniform background area, the system can remove these background areas from the image to obtain the foreground area, wherein the foreground area may include fingerprint features and other possible interference information (such as deeper noise); further, the system can identify all connected areas in the foreground area (referring to adjacent pixel blocks with similar grayscale values in the image), and calculate the center of gravity position of each connected area (that is, the average position of all pixel points in the area) ), and judge the position of the fingerprint area based on the distribution of the centroid position. If the system detects that the centroid position of the connected area is concentrated in a certain area in the grayscale image, then the area is judged to be the fingerprint area; or, if the system detects that the centroid position (i.e., Euclidean distance) of the connected area is scattered throughout the entire image, then the system can judge that there is no significant fingerprint feature in the area and it may be a noise area; after determining the fingerprint area and the noise area, the system can remove large blocks of noise that are not in the fingerprint area (for example, by marking and removing those connected areas with scattered centroid positions and large areas to clean up the noise in the image), so that the foreground area obtained after processing is purer and the fingerprint features are more obvious, which is convenient for subsequent feature extraction and recognition. Please refer to Figure 4B, which is a schematic diagram of uniform background extraction provided in an embodiment of the present application. Figure 4B may include a target area schematic diagram 04-A, an inverted image 04-B, a filtered image 04-C, a uniform background extraction image 04-D and a binary fingerprint area 04-E. As shown in the target area schematic diagram 04-A, the target area schematic diagram 04-A is an image of the target area intercepted from the fingerprint image. Because the visible trace of the target area schematic diagram 04-A is low, it is impossible to accurately judge the fingerprint residual performance of the shell. Therefore, the target area schematic diagram 04-A can be inverted by calculation, that is, the target area schematic diagram 04-A is converted into an inverted image 04-B, and the pixel information and fingerprint pattern information in the inverted image 04-B become clearly visible; further, the system can filter and morphologically process the inverted image 04-B (that is, the above-mentioned steps S4002B and S4002B). 02D), reduce the noise in the inverted image 04-B to obtain the filtered image 04-C. The system can determine the uniform background area by identifying the background area with uniform grayscale value in the filtered image 04-C, and remove the uniform background area to obtain the foreground area; further, the system can judge the position of the fingerprint area according to the concentration of the center of gravity of the connected area in the foreground area, and remove the large block of noise that is not in the fingerprint area to obtain a grayscale image with clearer fingerprint features, that is, the uniform background extraction image 04-D. The white noise in the uniform background extraction image 04-D is reduced, and the area where the fingerprint pattern is located is clearer. Finally, the uniform background extraction image 04-D can be binarized (that is, the following step S4002E) to obtain a binary fingerprint area 04-E, in which the noise and background area have been removed and only the fingerprint area extracted from the image of the target area is included.

Step S4002D: mathematical morphology calculation.

Step S4002E: Adaptively perform OSTU binarization to extract fingerprint positions.

Step S4003: directly calculating the area of the fingerprint region.

Step S4004: Place the fingerprint position into the original image and calculate the fingerprint contrast of the RGB channels respectively.

The specific descriptions of the above-mentioned steps S4001, S4003 and S4004 can refer to the embodiments corresponding to steps S2001, S2003 and S2004 in Figure 2B above, and the steps in steps S4002A-S4002E except step S4002C can refer to the embodiments corresponding to steps S2002A-S2002D, which will not be repeated here.

Scene 4: The light source is uneven and the uniform background has been removed.

Specifically, please refer to Figure 5A, which is a flowchart of an image grayscale calculation scenario four provided in an embodiment of the present application. The process may include the following steps S5001-S5004.

Step S5001: Fingerprint image capture and input.

Specifically, a fingerprint image is obtained, which is a fingerprint image obtained by photographing the fingerprint on the surface of the target shell. The fingerprint images corresponding to different shells can be images obtained by photographing under the same shooting conditions, such as the same shooting light source, shooting technique, shooting equipment and other conditions, so that a more objective and accurate comparison result can be obtained when comparing the fingerprint residual performance differences of different shells in the subsequent comparison, avoiding the influence of data anomalies caused by different shooting conditions.

Step S5002: extracting the region of interest (RIO).

Specifically, after acquiring the fingerprint image, the target area and the fingerprint area with fingerprint patterns collected in the target area can be extracted from the fingerprint image by performing RIO area extraction on the fingerprint image, wherein step S5002 can include the following steps S5002A-S5002I.

Step S5002A: Image grayscale calculation.

Specifically, the image grayscale calculation is used to convert the image of the target area from a color image to a grayscale image, and the above-mentioned multiple grayscale images only contain brightness information and remove color information. Exemplarily, the grayscale calculation can adopt a weighted average method, that is, the grayscale value is calculated according to the brightness values of the three RGB channels of the image. For example, the grayscale calculation formula can be: grayscale value = 0.299 R + 0.587 G + 0.114*B. The grayscale calculation formula is obtained by weighted calculation based on the visibility of different colors to the human eye, so that the RGB channel information (usually composed of three channels of red, green, and blue) of each pixel in the image of the target area is synthesized into a grayscale value, and based on the grayscale value, the image of the target area is converted into a grayscale image including only grayscale information.

Step S5002B: adaptively partition the image of the target area.

Specifically, when it is detected that the light source of the grayscale image of the target area is uneven, the system can adaptively divide the image into multiple sub-areas according to the area and light source conditions of the grayscale image of the target area to better adapt to the local characteristics of the image.

Step S5002C: Image filtering.

Specifically, after the grayscale image of the target area is adaptively partitioned into multiple sub-areas, the multiple sub-areas can be denoised by image filtering such as log-Gabor filter and/or median filter to preliminarily reduce interference and noise in the image and improve the clarity and continuity of the fingerprint lines.

Step S5002D: remove the uniform background and eliminate the noise based on the centroid concentration of the connected area.

Specifically, after preliminary noise removal is performed on multiple sub-regions in the grayscale image of the target area through image filtering, a uniform background region can be extracted from each sub-region, and the foreground region can be obtained after removing the uniform background region. The position of the fingerprint region is determined according to the concentration of the center of gravity of the connected regions in the foreground region, and large blocks of noise that are not in the fingerprint region are removed, so as to achieve secondary noise removal in the above-mentioned multiple grayscale images.

Step S5002E: mathematical morphology calculation.

Specifically, after preliminary noise removal through image filtering and deep noise removal based on the centroid concentration of connected areas, the system can perform morphological processing (i.e., mathematical morphological calculations) such as corrosion, dilation, opening and closing operations on each sub-area after denoising, thereby improving the continuity and clarity of multiple sub-areas in the grayscale image of the above-mentioned target area, and further strengthening the fingerprint features in the image.

Step S5002F: Partition image threshold segmentation.

Specifically, after performing image grayscale calculation, adaptive partitioning, image filtering and denoising processing, and mathematical morphology calculation on the target area, multiple sub-regions after grayscale calculation and denoising processing can be obtained, and the average brightness of each sub-region is calculated respectively, and an appropriate threshold is set based on the average brightness difference of each sub-region. Finally, binarization processing is performed using the threshold to more flexibly handle the non-uniformity of the light source and improve the accuracy and stability of subsequent fingerprint area extraction.

Step S5002G: Merge partition images.

Specifically, after performing the threshold segmentation of the partitioned image, the segmented regions need to be merged to obtain the overall fingerprint region. Assuming that the adaptive partitioning of the grayscale image of the above target area is 3×3 and 4×4 (i.e., multiple sub-regions), the common connected regions in the 3×3 or 4×4 grids can be merged to merge the adjacent fingerprint regions in the partitioned image into a larger connected region, thereby obtaining a more complete fingerprint region.

Step S5002H: Fingerprint connected area determination.

Specifically, after the partition images are merged, a connected region judgment may be performed on the merged fingerprint region to ensure the continuity of the extracted fingerprint region.

Step S5002I: directly remove non-fingerprint areas.

Specifically, after determining the fingerprint connected area, the system will directly remove the non-fingerprint area and only retain the part connected to the fingerprint (i.e., the fingerprint area). In addition, the fingerprint area envelope in the image of the target area can be determined by image segmentation or morphological processing, and the grayscale value in the variance grayscale uniform area can be set to 0 (black) or 255 (white) to remove the non-fingerprint area outside the fingerprint area envelope.

Step S5003: directly calculating the area of the fingerprint region.

Specifically, in the embodiment of the present application, the area of the fingerprint region can be calculated based on the binary fingerprint region (i.e., fingerprint position) extracted in step S5002, and the area of the fingerprint region is output as the residual fingerprint area. For example, the system first scans the binary fingerprint region, because in the binary fingerprint region image, black pixels represent fingerprint lines, so the area of the fingerprint region can be obtained by counting the number of all black pixels in the fingerprint region.

Step S5004: Place the fingerprint position into the original image and calculate the fingerprint contrast of the RGB channels respectively.

Specifically, the fingerprint position extracted in step S5002 can be placed in the image of the target area (i.e., the image of the target area in step S5002), the contrast between the RGB channel values of the fingerprint area and the target area can be calculated, and the fingerprint residual contrast of the three RGB channels can be output.

The embodiment descriptions in the above steps S5001 to S5004 are preliminary descriptions. For detailed descriptions, please refer to the corresponding embodiments in the above-mentioned Figures 2B to 4A, which will not be repeated here.

(ii) Image conversion to grayscale and background extraction 202.

Scene 1: The light source is uniform and the uniform background is not removed.

Specifically, please refer to Figure 6A, which is a flowchart of an image to grayscale calculation and background extraction scenario 1 provided in an embodiment of the present application. The process may include the following steps S6001-S6004.

Step S6001: Fingerprint image capture and input.

Specifically, a fingerprint image is obtained, which is a fingerprint image obtained by photographing the fingerprint on the surface of the target shell. The fingerprint images corresponding to different shells can be images obtained by photographing under the same shooting conditions, such as the same shooting light source, shooting technique, shooting equipment and other conditions, so that a more objective and accurate comparison result can be obtained when comparing the fingerprint residual performance differences of different shells in the subsequent comparison, avoiding the influence of data anomalies caused by different shooting conditions.

Step S6002: Region of Interest (RIO) extraction.

Specifically, after acquiring the fingerprint image, the target area and the fingerprint area with fingerprint patterns collected in the target area can be extracted from the fingerprint image by performing RIO area extraction on the fingerprint image, wherein step S6002 can include the following steps S6002A-S6002D.

Step S6002A: image conversion to grayscale calculation.

Specifically, the image grayscale calculation is used to convert the image of the target area from a color image to a grayscale image, which only contains brightness information and removes color information. Exemplarily, the grayscale calculation can use a weighted average method, that is, the grayscale value is calculated according to the brightness values of the three RGB channels of the image. For example, the grayscale calculation formula can be: grayscale value = 0.299 R + 0.587 G + 0.114*B. The grayscale calculation formula is obtained by weighted calculation based on the visibility of different colors to the human eye, so that the RGB channel information (usually composed of three channels of red, green, and blue) of each pixel in the image of the target area is synthesized into a grayscale value, and based on the grayscale value, the image of the target area is converted into a grayscale image including only grayscale information. The converted grayscale image only includes grayscale information converted from color information, so that the grayscale image can reflect the brightness distribution of the original color image. The above grayscale calculation formula is only a possible implementation method, and other implementation methods can also be used, which is not limited in the embodiments of the present application. When performing the image-to-grayscale calculation in step S6002A, step S6002A1 may also be included.

Step S6002A1: Background extraction.

Specifically, if the shells of the same material are compared, since the color difference of the shells is small, more grayscale information can be determined by performing background extraction on the grayscale image to more accurately extract and analyze fingerprint features. Exemplarily, in the embodiment of the present application, after the image of the target area is converted to grayscale to obtain a grayscale image, the background area can be identified by analyzing the distribution and change of the grayscale value. For example, the system can use a histogram analysis method to determine the grayscale value range of the background area, and by counting the frequency of each grayscale value in the image, find the grayscale value interval that occupies a large proportion and has a small change, and determine these intervals as the background area; further, the system can perform threshold segmentation on the grayscale image to distinguish the background area from the foreground area. For example, the system can select one or more thresholds based on the pixel information or brightness information in the grayscale image, and determine the pixel points with grayscale values below the threshold as the background area; after the threshold segmentation is completed, the system can mark and remove the identified background area, so that in the subsequent image processing process, the data processing process can be simplified, and the contrast of the shells of the same material can be highlighted, and the accuracy and efficiency of image processing can be improved.

Step S6002B: Image filtering.

Specifically, after the image of the target area is converted into a grayscale image and the background is extracted, the grayscale image can be denoised by image filtering. Exemplarily, in the embodiment of the present application, the grayscale image can be denoised by log-Gabor filter and/or median filter. The log-Gabor filter is used to extract the fingerprint pattern features in the image and enhance the details and edge information in the image; the median filter is a nonlinear filtering method, which is mainly used to remove noise (such as salt and pepper noise) in the image while retaining the edge details of the image. Through the above-mentioned image filtering process, the interference and noise in the image can be preliminarily reduced, and the clarity and continuity of the fingerprint pattern can be improved.

Step S6002C: mathematical morphology calculation.

Specifically, after the grayscale image is denoised by image filtering, the denoised grayscale image can be subjected to morphological processing (i.e., mathematical morphological calculation), and the morphological processing can include corrosion, expansion, opening operation, closing operation, etc. For example, in the embodiment of the present application, the holes in the fingerprint lines in the denoised grayscale image can be filled by the expansion operation to make it more complete; at the same time, the small holes or breaks in the fingerprint lines can be eliminated by the closing operation, thereby improving the continuity and clarity of the grayscale image and further strengthening the fingerprint features in the image. The above-mentioned mathematical morphological calculation is only one possible implementation method, and other implementation methods can also be used, which are not limited in the embodiment of the present application.

Step S6002D: Adaptively perform OSTU binarization to extract fingerprint position.

Specifically, after the grayscale image is processed by image filtering and mathematical morphology calculation, the system can use adaptive OSTU binarization to extract the fingerprint position (i.e., fingerprint area) in the grayscale image. Adaptive OSTU binarization is an automatic threshold selection technology based on grayscale histogram. Its principle is to determine the optimal segmentation threshold by maximizing the inter-class variance, thereby segmenting the image into two parts: foreground (fingerprint pattern) and background. This method can not only adapt to the grayscale distribution characteristics of different images, but also be flexibly adjusted according to the characteristics of the local area, so that the fingerprint location information can be extracted more accurately. In the embodiment of the present application, adaptive OSTU binarization is used to segment the image into two parts: foreground and background, so the output is an image containing only black and white pixel values, black (representing the background) and white (representing the foreground), that is, the grayscale image will become a binary image, which is used to indicate the presence or absence of an object. Through adaptive OSTU binarization, the system can dynamically adjust the threshold to ensure that the best segmentation effect can be obtained in different areas. Even if there is uneven lighting, noise interference, etc. in the image of the target area, the system can still accurately extract the fingerprint location information.

Step S6003: directly calculating the area of the fingerprint region.

Specifically, in the embodiment of the present application, the area of the fingerprint area can be calculated based on the binary fingerprint area (i.e., fingerprint position) extracted in step S6002, and the area of the fingerprint area is output as the residual fingerprint area. For example, the system first scans the binary fingerprint area. Because in the binary fingerprint area image, black pixels represent fingerprint lines, the area of the fingerprint area can be obtained by counting the number of all black pixels in the fingerprint area, wherein the statistical method includes but is not limited to row-by-row scanning, column-by-column scanning, automatic extraction algorithm, and regional growth algorithm. In addition, in the process of calculating the area of the fingerprint area, because the resolution of different images may be different, the number of pixels corresponding to the fingerprint area of the same area in images of different resolutions is also different. Therefore, when calculating the area, the system can convert by introducing the resolution information of the image, for example, by converting the pixel area to the actual area through the pixel density (pixels per inch, DPI) to obtain the area of the fingerprint area.

Step S6004: Calculate the cumulative grayscale value of the fingerprint area.

Specifically, the fingerprint position extracted in step S6002 can be placed in the image of the target area (i.e., the image of the target area in step S6002), the contrast between the grayscale values of the fingerprint area and the target area can be calculated, and the fingerprint residual contrast of the grayscale value can be output. Exemplarily, when testing a shell of the same material, because the shells of the same material usually have a small color difference, the difference in fingerprint residual performance between different shell materials can be more accurately and objectively judged through the grayscale difference information, and the contrast of the above grayscale value can be calculated by the contrast formula. Assume that the overall area of the target area is A, the area of the fingerprint area in the target area is B, and the surrounding area outside the fingerprint area is A-B; the average grayscale value in the B area is β, and the average grayscale value in the A-B area is α. In general, the color depth of the fingerprint area is generally greater than that of the area other than the fingerprint area (i.e., the color of the B area is darker than that of the A-B area), but the dark grayscale statistics are that the darker the color, the lower the grayscale, so the image can be inverted as a whole. At this time, β is greater than α, and A is greater than B.

At this time, the calculated contrast index can be expressed as the sum of the image grayscale values in area B is greater than the sum of the image grayscale values in area A. The contrast formula can be expressed as:

First, the monotonicity analysis of the simplified formula is:

(1) When the area of A is fixed (when taking photos in real life, the area of A also needs to be fixed using fixed equipment and distance), contrast is positively correlated with B;

(2) When A and B (residual area) are fixed, contrast and Negative correlation, the larger β is, the smaller α is. The smaller it is, the greater the overall contrast (the greater the difference between β and α, the greater the contrast (significant contrast), which also conforms to the subjective meaning of contrast);

Secondly, analyze the influencing factors and correlations of several variables in the formula:

Overall area A: related to the shooting device, shooting conditions (distance L, notebook thickness d), and the selected fingerprint area ROI.

Fingerprint area (dirty area) B: B is the fingerprint area extracted by the automatic extraction algorithm. B is related to the fingerprint algorithm extraction accuracy. B is related to the average grayscale depth β of the fingerprint and the average grayscale depth α of the background.

Background average grayscale α and fingerprint average depth β: related to material (Material), shooting equipment (device), shooting light source (light-source), and fingerprint residue condition (condition).

Finally, in special cases, special values are analyzed:

Assume that the fingerprint detection area B is fixed (if the fingerprint can be detected, β>α), the greater the difference between β and α, the greater the contrast.

Scene 2: The light source is uneven and the uniform background is not removed.

Specifically, please refer to Figure 7A, which is a flowchart of an image to grayscale calculation scenario 2 provided in an embodiment of the present application. The process may include the following steps S7001-S7004.

Step S7001: Fingerprint image capture and input.

Step S7002: Region of Interest (RIO) extraction.

Specifically, after acquiring the fingerprint image, the target area and the fingerprint area with fingerprint patterns collected in the target area can be extracted from the fingerprint image by performing RIO area extraction on the fingerprint image, wherein step S7002 can include the following steps S7002A-S7002H.

Step S7002A: image conversion to grayscale calculation.

Step S7002A1: Background extraction.

Step S7002B: adaptively partition the image of the target area.

Specifically, when the light source of the image of the target area is detected to be uneven, the system will adaptively divide the image into multiple areas according to the characteristics of the image of the target area and the light source conditions to better adapt to the local characteristics of the image. For example, if the image of the target area is detected to have uneven reflections of varying degrees, the calculation according to the normal steps may affect the calculation results of the fingerprint area. Therefore, when the image of the target area is detected to have uneven light sources such as the central area being brighter than the target area, the system can partition the image of the target area according to a 3×3 and/or 4×4 grid to process each area more accurately, thereby improving the accuracy of the image processing of the target area and minimizing the processing error.

Step S7002C: Image filtering.

Step S7002D: mathematical morphology calculation.

Step S7002E: Partition image threshold segmentation.

Specifically, after the image grayscale calculation, adaptive partitioning, image filtering and denoising processing and mathematical morphology calculation are performed on the image of the target area, multiple sub-areas in the grayscale image of the target area can be obtained, and the average brightness of each sub-area can be calculated respectively, and an appropriate threshold can be set based on the average brightness difference of each sub-area. Based on the threshold, the pixels in the sub-area can be divided into two categories: the pixels above the threshold are classified as fingerprint features, and the pixels below the threshold are classified as background; the fingerprint area can be more accurately extracted from the image of the target area through the threshold segmentation of the above-mentioned binarization processing. Exemplarily, if the average brightness in a sub-area among the multiple sub-areas is high, it may mean that there is a fingerprint in the sub-area, so a relatively low threshold can be selected for segmentation to ensure that the fingerprint area can be effectively extracted; and for the area with low average brightness, it may be the background area, so a higher threshold can be selected for segmentation to reduce the interference of the background. Through this method of partitioned image threshold segmentation, the light source non-uniformity can be handled more flexibly, and the subsequent fingerprint area extraction accuracy and stability can be improved.

Step S7002F: Merge partition images.

Specifically, after performing the threshold segmentation of the partitioned image, it is necessary to merge the segmented regions to obtain the overall fingerprint region. Assuming that the image of the target region is partitioned into 3×3 and 4×4 (i.e., the above-mentioned multiple sub-regions), the common connected regions in the 3×3 or 4×4 grids can be merged to merge the adjacent fingerprint regions in the partitioned image into a larger connected region, thereby obtaining a more complete fingerprint region.

Step S7002G: Fingerprint connected area determination.

Specifically, after the partition images are merged, the merged fingerprint area can be judged for connected areas to ensure the continuity of the extracted fingerprint area. Exemplarily, the system can judge the connected areas of the merged fingerprint area, identify and retain the connected fingerprint area, and then remove the disconnected parts. For example, the system will detect whether there are broken or discontinuous parts in the merged fingerprint area, and remove these parts, so as to ensure that the final extracted fingerprint area is continuous and improve the continuity and clarity of the extracted fingerprint area.

Step S7002H: directly remove non-fingerprint areas.

Specifically, after the fingerprint connected area judgment is performed, the system will directly eliminate the non-fingerprint area to reduce the amount of calculation for subsequent processing and improve processing efficiency. For example, assuming that in the process of fingerprint connected area judgment, the system identifies some areas that are not connected to the fingerprint, the above-mentioned unconnected areas will be eliminated, and only the part connected to the fingerprint (i.e., the fingerprint area) will be retained. In summary, in the embodiment of the present application, multiple sub-areas can be obtained by adaptively partitioning the grayscale image of the target area, and a threshold is determined based on the light source situation in each sub-area, and the common connected area of each sub-area is merged to determine the fingerprint area, which greatly reduces the error effect caused by the uneven light source, thereby improving the objectivity of the shell fingerprint residue test scheme and the accuracy of the results. In addition, the fingerprint area envelope in the image of the target area can also be determined by image segmentation or morphological processing, and the non-fingerprint area outside the fingerprint area envelope can be removed by setting the grayscale value in the variance grayscale uniform area to 0 (black) or 255 (white).

Step S7003: directly calculate the area of the fingerprint region.

Step S7004: Calculate the cumulative grayscale value of the fingerprint area.

For the specific descriptions of the above-mentioned steps S7001, S7003 and S7004, please refer to the embodiments corresponding to steps S6001, S6003 and S6004 in Figure 6A above. For steps S7002A, S7002C and S7002D, please refer to the embodiments corresponding to steps S6002A-S6002C, which will not be repeated here.

Scene 3: The light source is uniform and the uniform background has been removed.

Specifically, please refer to Figure 8A, which is a flowchart of an image to grayscale calculation scenario three provided in an embodiment of the present application. The process may include the following steps S8001-S8004.

Step S8001: Fingerprint image capture and input.

Step S8002: Region of Interest (RIO) extraction.

Specifically, after acquiring the fingerprint image, the target area and the fingerprint area with fingerprint patterns collected in the target area can be extracted from the fingerprint image by performing RIO area extraction on the fingerprint image, wherein step S8002 can include the following steps S8002A-S8002E.

Step S8002A: image conversion to grayscale calculation.

Step S8002A1: Background extraction.

Step S8002B: Image filtering.

Step S8002C: remove the uniform background and eliminate the noise based on the centroid concentration of the connected area.

Specifically, after performing preliminary noise removal on the grayscale image of the target area through image filtering, the uniform background area in the grayscale image can be extracted, and the foreground area can be obtained after removing the uniform background area. The position of the fingerprint area is determined according to the concentration of the center of gravity of the connected area in the foreground area, and the large block of noise that is not in the fingerprint area is removed. Exemplarily, after performing preliminary noise removal on the grayscale image through image filtering, the system can determine the uniform background area by identifying the background area with uniform grayscale value in the grayscale image, and the uniform background area does not contain fingerprint features and the grayscale value changes slightly; when the system extracts the uniform background area, the system can remove these background areas from the image to obtain the foreground area, wherein the foreground area may include fingerprint features and other possible interference information (such as deeper noise); further, the system can identify all connected areas in the foreground area (referring to adjacent pixel blocks with similar grayscale values in the image), and calculate the center of gravity position of each connected area (that is, the average position of all pixel points in the area) ), and judge the position of the fingerprint area based on the distribution of the centroid position. If the system detects that the centroid position of the connected area is concentrated in a certain area in the grayscale image, then the area is judged to be the fingerprint area; or, if the system detects that the centroid position (i.e., Euclidean distance) of the connected area is scattered throughout the entire image, then the system can judge that there is no significant fingerprint feature in the area and it may be a noise area; after determining the fingerprint area and the noise area, the system can remove large blocks of noise that are not in the fingerprint area (for example, by marking and removing those connected areas with scattered centroid positions and large areas to clean up the noise in the image), so that the foreground area obtained after processing is purer and the fingerprint features are more obvious, which is convenient for subsequent feature extraction and recognition.

Step S8002D: mathematical morphology calculation.

Step S8002E: Adaptively perform OSTU binarization to extract fingerprint positions.

Step S8003: directly calculating the area of the fingerprint region.

Step S8004: Calculate the cumulative grayscale value of the fingerprint area.

For the specific descriptions of the above-mentioned steps S8001, S8003 and S8004, please refer to the embodiments corresponding to steps S6001, S6003 and S6004 in Figure 6A above. For the steps in steps S8002A-S8002E except step S8002C, please refer to the embodiments corresponding to steps S6002A-S6002D, which will not be repeated here.

Scene 4: The light source is uneven and the uniform background has been removed.

Specifically, please refer to Figure 9A, which is a flowchart of an image grayscale calculation scenario four provided in an embodiment of the present application. The process may include the following steps S9001-S9004.

Step S9001: Fingerprint image capture and input.

Specifically, a fingerprint image is obtained, which is a fingerprint image obtained by photographing the fingerprint on the surface of the target shell. The fingerprint images corresponding to different shells can be images obtained by photographing under the same shooting conditions, such as the same shooting light source, shooting technique, shooting equipment and other conditions, so that a more objective and accurate comparison result can be obtained when comparing the fingerprint residual performance differences of different shells in the subsequent comparison, avoiding the influence of data anomalies caused by different shooting conditions.

Step S9002: Region of Interest (RIO) extraction.

Specifically, after acquiring the fingerprint image, the target area and the fingerprint area with fingerprint patterns collected in the target area can be extracted from the fingerprint image by performing RIO area extraction on the fingerprint image, wherein step S9002 can include the following steps S9002A-S9002I.

Step S9002A: image conversion to grayscale calculation.

Specifically, the image grayscale calculation is used to convert the image of the target area from a color image to a grayscale image, which only contains brightness information and removes color information. Exemplarily, the grayscale calculation can use a weighted average method, that is, the grayscale value is calculated based on the brightness values of the three RGB channels of the image. For example, the grayscale calculation formula can be: grayscale value = 0.299 R + 0.587 G + 0.114*B. The grayscale calculation formula is obtained by weighted calculation based on the visibility of different colors to the human eye, so that the RGB channel information (usually composed of three channels of red, green, and blue) of each pixel in the image of the target area is synthesized into a grayscale value, and based on the grayscale value, the image of the target area is converted into a grayscale image that only includes grayscale information.

Step S9002A1: Background extraction.

Specifically, when comparing shells of the same material, since the color difference of the shells is small, more grayscale information can be determined by performing background extraction on the grayscale image, so as to more accurately extract and analyze fingerprint features.

Step S9002B: adaptively partition the image of the target area.

Specifically, when it is detected that the light source of the grayscale image of the target area is uneven, the system will adaptively divide the grayscale image into a plurality of sub-areas according to the characteristics of the grayscale image and the light source conditions.

Step S9002C: Image filtering.

Specifically, after adaptively partitioning the grayscale image of the target area to obtain multiple sub-regions, the multiple sub-regions can be denoised by image filtering such as log-Gabor filter and/or median filter to preliminarily reduce interference and noise in the grayscale image and improve the clarity and continuity of the fingerprint lines.

Step S9002D: remove the uniform background and eliminate the noise based on the centroid concentration of the connected area.

Specifically, after preliminary noise removal of multiple sub-regions through image filtering, a uniform background region can be extracted from each sub-region, and the foreground region can be obtained after removing the uniform background region. The position of the fingerprint region can be determined according to the concentration of the center of gravity of the connected regions in the foreground region, and large blocks of noise that are not in the fingerprint region can be removed, so as to achieve secondary noise removal in the above-mentioned multiple grayscale images.

Step S9002E: mathematical morphology calculation.

Specifically, after preliminary noise removal through image filtering and deep noise removal based on the centroid concentration of connected areas, the system can perform morphological processing (i.e., mathematical morphological calculations) such as corrosion, dilation, opening and closing operations on the multiple sub-areas after denoising, thereby improving the continuity and clarity of the above-mentioned multiple grayscale images and further strengthening the fingerprint features in the image.

Step S9002F: Partition image threshold segmentation.

Specifically, after performing image grayscale calculation, adaptive partitioning, image filtering and denoising processing, and mathematical morphology calculation on the image of the target area, multiple sub-regions in the grayscale image of the target area can be obtained. The average brightness of each sub-region is calculated separately, and an appropriate threshold is set based on the average brightness difference of each grayscale image. Finally, binarization processing is performed based on the threshold to more flexibly handle the non-uniformity of the light source and improve the accuracy and stability of subsequent fingerprint area extraction.

Step S9002G: Merge partition images.

Specifically, after performing the threshold segmentation of the partitioned image, it is necessary to merge the segmented regions to obtain the overall fingerprint region. Assuming that the grayscale image of the target area is partitioned into 3×3 and 4×4 (i.e., multiple grayscale images are partitioned one by one), the common connected regions in the 3×3 or 4×4 grids can be merged to merge adjacent fingerprint regions in the partitioned image into a larger connected region, thereby obtaining a more complete fingerprint region.

Step S9002H: Fingerprint connected area determination.

Specifically, after the partition images are merged, a connected region judgment may be performed on the merged fingerprint region to ensure the continuity of the extracted fingerprint region.

Step S9002I: directly remove non-fingerprint areas.

Specifically, after determining the fingerprint connected area, the system will directly remove the non-fingerprint area and only retain the part connected to the fingerprint (i.e., the fingerprint area). In addition, the fingerprint area envelope in the image of the target area can be determined by image segmentation or morphological processing, and the grayscale value in the variance grayscale uniform area can be set to 0 (black) or 255 (white) to remove the non-fingerprint area outside the fingerprint area envelope.

Step S9003: directly calculating the area of the fingerprint region.

Specifically, in the embodiment of the present application, the area of the fingerprint region can be calculated based on the binary fingerprint region (i.e., fingerprint position) extracted in step S9002, and the area of the fingerprint region is output as the residual fingerprint area. For example, the system first scans the binary fingerprint region, because in the fingerprint region image, black pixels represent fingerprint lines, so the area of the fingerprint region can be obtained by counting the number of all black pixels in the fingerprint region.

Step S9004: Calculate the cumulative grayscale value of the fingerprint area.

Specifically, the fingerprint position extracted in step S9002 can be placed in the image of the original target area (ie, the image of the target area in step S9002), the contrast between the grayscale values of the fingerprint area and the target area can be calculated, and the fingerprint residual contrast of the three RGB channels can be output.

The embodiment descriptions in the above steps S9001 to S9004 are preliminary descriptions. For detailed descriptions, please refer to the corresponding embodiment descriptions in the above-mentioned Figures 6A to 8A, which will not be repeated here.

The above describes in detail the method of the embodiment of the present application, and the following provides the related device of the embodiment of the present application.

Please refer to Figure 10, which is a structural schematic diagram of a detection device for fingerprint residue on a shell provided in an embodiment of the present application. The detection device 1000 for fingerprint residue on a shell may include a fingerprint image acquisition unit 1001, a fingerprint area determination unit 1002, a contrast determination unit 1003 and a fingerprint residue performance judgment unit 1004, wherein each unit is described in detail as follows.

The fingerprint image acquisition unit 1001 is used to acquire a fingerprint image, where the fingerprint image is an image obtained by photographing the fingerprint on the surface of the target housing;

A fingerprint area determination unit 1002 is used to perform image processing on the fingerprint image to determine a target area in the fingerprint image and a fingerprint area in the target area where fingerprint lines are collected;

In a possible implementation, the fingerprint area determination unit 1002 is specifically configured to:

Determining the target area based on shooting parameters of the fingerprint image;

Converting the image of the target area into a first grayscale image;

Performing image feature enhancement processing on the first grayscale image to obtain a second grayscale image;

The second grayscale image is binarized to determine a fingerprint region in the target region that meets the fingerprint feature.

In a possible implementation, the fingerprint area determination unit 1002 is specifically configured to:

Determining the target area based on shooting parameters of the fingerprint image;

Converting the image of the target area into a third grayscale image;

Performing partition processing on the third grayscale image based on the shooting light source and the area of the third grayscale image to obtain a plurality of sub-areas;

Performing image feature enhancement processing on each sub-region in the third grayscale image to obtain a fourth grayscale image;

The fingerprint region is determined by binarizing the fourth grayscale image and merging the connected regions shared by each subregion in the fourth grayscale image after the binarization process.

In a possible implementation, the fingerprint area determination unit 1002 is specifically configured to:

determining the target area based on shooting parameters;

Converting the image of the target area into a first grayscale image;

Extracting a background area from the first grayscale image, and removing the background area from the first grayscale image;

Performing image feature enhancement processing on the first grayscale image after removing the background area to obtain a second grayscale image;

The second grayscale image is binarized to determine a fingerprint region in the target region that meets the fingerprint feature. In a possible implementation, the fingerprint region determination unit 1002 is specifically configured to:

determining the target area based on shooting parameters;

Converting the image of the target area into a third grayscale image;

Performing partition processing on the third grayscale image based on the shooting light source and the area of the third grayscale image to obtain a plurality of sub-areas;

Extracting a background region from each sub-region in the third grayscale image, and removing the background region from each sub-region in the third grayscale image;

Performing image feature enhancement processing on each sub-region in the third grayscale image after removing the background region, to obtain a fourth grayscale image;

The fingerprint region is determined by binarizing the fourth grayscale image and merging the connected regions shared by each subregion in the fourth grayscale image after the binarization process.

In a possible implementation, the fingerprint area determination unit 1002 is specifically configured to:

De-noising the corresponding grayscale image through image filtering;

Perform morphological processing on the denoised grayscale image.

In a possible implementation, the fingerprint area determination unit 1002 is specifically configured to:

The uniform background area in the denoised grayscale image is removed to obtain the foreground area;

Removing the peripheral area in the foreground area to obtain an image after the peripheral area is removed; the uniform background includes uniform light source and/or uniform texture, and the peripheral area is an area in the foreground area excluding the connected area where the pixel values are concentrated;

Morphological processing is performed on the image after the peripheral area is removed.

A contrast determination unit 1003, which determines the contrast between the fingerprint area and the target area based on the pixel information of the fingerprint area and the pixel information of the target area;

In a possible implementation, if the pixel information includes RGB channel values, the contrast determination unit 1003 is specifically configured to:

Determine the sum of each channel value in the RGB channel values of the fingerprint area and the sum of each channel value in the RGB channel values of the target area;

The sum of each channel value in the RGB channel values of the fingerprint area is compared with the sum of each channel value in the RGB channel values of the target area to obtain the contrast of each channel.

In a possible implementation, if the pixel information includes a grayscale value, the contrast determination unit 1003 is specifically configured to:

Determining the sum of the grayscale values of the fingerprint area and the sum of the grayscale values of the target area;

A comparison result of the sum of the grayscale values of the fingerprint area and the sum of the grayscale values of the target area is determined as the contrast.

The fingerprint residual performance judging unit 1004 judges the fingerprint residual performance of the target housing based on the area of the fingerprint region and the contrast.

In a possible implementation, the fingerprint residual performance determination unit 1004 is specifically configured to:

If the area of the fingerprint region is smaller than the preset area, and the contrast is smaller than the preset contrast, it is determined that the fingerprint residue of the target housing is low.

To facilitate understanding of the embodiments of the present application, an exemplary electronic device provided in the embodiments of the present application is first introduced below.

Please refer to Figure 11, which is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device 100 can be various types of electronic devices, and the embodiment of the present application does not limit its specific type. For example, the electronic device 100 can be a mobile phone, and can also include a tablet computer, a desktop computer, a desktop computer with a touch-sensitive surface or a touch panel, a laptop computer (laptop), a handheld computer, a notebook computer, a smart screen, a wearable device (such as a smart watch, a smart bracelet, etc.), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a car machine, a game console, and can also be an Internet of Things (IOT) device, etc. Please refer to Figure 11, and the following is a specific introduction to the various components of the electronic device 100 in conjunction with Figure 11:

The electronic device 100 may include a processor 101, a memory 102, a wireless communication module 103, a mobile communication module 104, an antenna 103A, an antenna 104A, a power switch 105, a sensor module 106, a focus motor 107, a camera 108, a display screen 109, etc. Among them, the sensor module 106 may include a gyroscope sensor 106A, an acceleration sensor 106B, an ambient light sensor 106C, an image sensor 106D, a distance sensor 106E, etc. Among them, the wireless communication module 103 may include a WLAN communication module, a Bluetooth communication module, etc. The above multiple parts can transmit data through a bus.

The processor 101 may include one or more processing units, for example: the processor 101 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. In the embodiment of the present application, the processor 101 is responsible for executing various instructions in the detection of fingerprint residues on the shell. For example, after obtaining a fingerprint image (i.e., the fingerprint image in the embodiment description corresponding to FIG. 1A), the processor 101 can execute an image processing function to extract a target area from the fingerprint image, and a fingerprint area in the target area where fingerprint lines are collected, and determine the area of the fingerprint area and the fingerprint residue contrast (such as the contrast of the RGB channel value or grayscale value of the fingerprint area and the target area in the fingerprint image) based on the fingerprint area to determine the fingerprint residue performance of the shell.

The memory 102 may be used to store computer executable program codes, and the executable program codes may include instructions. The processor 101 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the memory 102. The memory 102 may include a program storage area and a data storage area. In a specific implementation, the memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

The wireless communication function of the electronic device 100 can be implemented through the antenna 103A, the antenna 104A, the mobile communication module 104, the wireless communication module 103, the modem processor and the baseband processor.

Antenna 103A and antenna 104A can be used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas.

The mobile communication module 104 can provide solutions for wireless communications including 2G/3G/4G/5G, etc., applied to the electronic device 100. The mobile communication module 104 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 104 can receive electromagnetic waves through the antenna 104A, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 104 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 104A.

The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is transmitted to the application processor. The application processor outputs a sound signal through an audio device, or displays an image or video through the display screen 109.

The wireless communication module 103 can provide wireless communication solutions including wireless local area networks (WLAN), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., applied to the electronic device 100. The wireless communication module 103 can be one or more devices integrating at least one communication processing module. The wireless communication module 103 receives electromagnetic waves via the antenna 103A, modulates the frequency of the electromagnetic wave signal and performs filtering processing, and sends the processed signal to the processor 101. The wireless communication module 103 can also receive the signal to be sent from the processor 101, modulate the frequency of the signal, amplify it, and convert it into electromagnetic waves for radiation through the antenna 103A. In the embodiment of the present application, files transmitted from other devices can be received based on Bluetooth or other communication technologies in the wireless communication module 103, and after receiving the file, the file can be received, refused to be received, or sent to the target application window based on the user's choice.

The power switch 105 may be used to control the supply of power to the electronic device 100 .

The gyro sensor 106A can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 106A. The gyro sensor 106A can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 106A detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 106A can also be used for navigation and somatosensory game scenes.

The acceleration sensor 106B can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device. For example, the acceleration sensor 106B can be applied to applications such as horizontal and vertical screen switching and pedometers.

The ambient light sensor 106C is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 109 according to the sensed brightness of the ambient light. The ambient light sensor 106C can also be used to automatically adjust the white balance when taking pictures.

The image sensor 106D, also known as a photosensitive element, can utilize the photoelectric conversion function of the photoelectric device to convert the light image on the photosensitive surface into an electrical signal that is proportional to the light image. The image sensor can be a charge coupled device (CCD) sensor or a complementary metal-oxide-semiconductor (CMOS) sensor. Exemplarily, when the shutter is pressed, under a fixed light source and shooting technique, the image sensor 106D can convert the captured light signal into an electrical signal and pass it to the ISP for processing, so as to further analyze and process the fingerprint image (i.e., the fingerprint image in the embodiment description corresponding to FIG1A).

The distance sensor 106E can be used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some shooting scenarios, the electronic device 100 can use the distance sensor 106E to measure the distance to achieve fast focusing.

The focus motor 107 can be used for fast focusing. The electronic device 100 can control the movement of the lens through the focus motor 107 to achieve automatic focusing.

The electronic device 100 can realize the shooting function through the ISP, the camera 108, the video codec, the GPU, the display screen 109 and the application processor.

The ISP is used to process the fingerprint image data fed back by the camera 108. For example, when taking a photo, light is transmitted to the image sensor 106D through the lens, and the light signal is converted into an electrical signal. The image sensor 106D transmits the electrical signal to the ISP for processing and converts it into a digital image signal. The ISP can also optimize the noise, brightness, and color of the image to ensure the quality and recognizability of the fingerprint image.

The camera 108 can be used to capture a static image of the residual fingerprint. The object generates an optical image through the lens and projects it onto the image sensor 106D. The image sensor 106D converts the optical signal into an electrical signal and transmits it to the ISP to convert it into a digital image signal so as to extract the fingerprint area (i.e., the fingerprint area in the embodiment description corresponding to FIG. 1A) and calculate the contrast. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 108, where N is a positive integer greater than 1.

The video codec is used to compress or decompress digital images. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can open or save pictures or videos in multiple coding formats.

The electronic device 100 can realize the display function through a GPU, a display screen 109, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 109 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 101 may include one or more GPUs, which execute program instructions to generate or change display information.

The display screen 109 is used to display images, videos, etc. The display screen 109 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 109, where N is a positive integer greater than 1.

It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

The present application also provides an electronic device, which may include: a memory and a processor. The memory may be used to store a computer program; the processor may be used to call the computer program in the memory so that the electronic device executes the method executed by the electronic device side in any of the above embodiments.

The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the electronic device side in any of the above embodiments.

In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

The chip system may be composed of the chip, or may include the chip and other discrete devices.

Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processing unit, a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute the method executed by the electronic device side in any of the above embodiments.

The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device side in any of the above embodiments.

The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by a computer program to instruct the relevant hardware to complete the process, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes. In short, the above is only an embodiment of the technical solution of the present application, and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made according to the disclosure of the present application should be included in the scope of protection of the present application.

Claims (22)

1.A method for detecting fingerprint residue of a housing, the method comprising:

Acquiring a fingerprint image, wherein the fingerprint image is an image obtained by shooting a fingerprint on the surface of a target shell;

Performing image processing on the fingerprint image, and determining a target area in the fingerprint image and a fingerprint area with fingerprint lines collected in the target area;

Determining the contrast ratio of the fingerprint region and the target region based on the pixel information of the fingerprint region and the pixel information of the target region;

And judging the fingerprint residual performance of the target shell based on the area of the fingerprint area and the contrast.

2. The method of claim 1, wherein if the pixel information includes RGB channel values; the determining the contrast ratio between the fingerprint region and the target region based on the pixel information of the fingerprint region and the pixel information of the target region includes:

Determining a sum of each of the RGB channel values of the fingerprint region and a sum of each of the RGB channel values of the target region;

and comparing the sum of each channel value in the RGB channel values of the fingerprint area with the sum of each channel value in the RGB channel values of the target area to obtain the contrast of each channel.

3. The method of claim 2, wherein the performing image processing on the fingerprint image to determine a target area in the fingerprint image and a fingerprint area in the target area where fingerprint lines are acquired, comprises:

determining the target area based on shooting parameters of the fingerprint image;

converting the image of the target area into a first gray scale image;

performing image feature enhancement processing on the first gray level image to obtain a second gray level image;

And carrying out binarization processing on the second gray level image to determine a fingerprint area which accords with fingerprint characteristics in the target area.

4. The method of claim 2, wherein the performing image processing on the fingerprint image to determine a target area in the fingerprint image and a fingerprint area in the target area where fingerprint lines are acquired, comprises:

determining the target area based on shooting parameters of the fingerprint image;

converting the image of the target area into a third gray scale image;

partitioning the third gray level image based on the areas of the shooting light source and the third gray level image to obtain a plurality of subareas;

respectively carrying out image characteristic enhancement processing on each sub-region in the third gray level image to obtain a fourth gray level image;

and the fingerprint area is determined by carrying out binarization processing on the fourth gray level image and combining the communication areas shared by each sub-area in the fourth gray level image after the binarization processing.

5. The method of claim 1, wherein if the pixel information includes a gray value; the determining the contrast ratio between the fingerprint region and the target region based on the pixel information of the fingerprint region and the pixel information of the target region includes:

determining a sum of gray values of the fingerprint area and a sum of gray values of the target area;

and determining a comparison result of the sum of the gray values of the fingerprint area and the sum of the gray values of the target area as the contrast.

6. The method of claim 5, wherein the performing image processing on the fingerprint image to determine a target area in the fingerprint image and a fingerprint area in the target area where fingerprint lines are collected, comprises:

determining the target area based on shooting parameters;

converting the image of the target area into a first gray scale image;

extracting a background area from the first gray level image, and removing the background area in the first gray level image;

performing image feature enhancement processing on the first gray level image with the background area removed to obtain a second gray level image;

And carrying out binarization processing on the second gray level image to determine a fingerprint area which accords with fingerprint characteristics in the target area.

7. The method of claim 5, wherein the performing image processing on the fingerprint image to determine a target area in the fingerprint image and a fingerprint area in the target area where fingerprint lines are collected, comprises:

determining the target area based on shooting parameters;

converting the image of the target area into a third gray scale image;

partitioning the third gray level image based on the areas of the shooting light source and the third gray level image to obtain a plurality of subareas;

Extracting a background region from each sub-region in the third gray scale image, and removing the background region in each sub-region in the third gray scale image;

Respectively carrying out image characteristic enhancement processing on each sub-region in the third gray level image after the background region is removed to obtain a fourth gray level image;

and the fingerprint area is determined by carrying out binarization processing on the fourth gray level image and combining the communication areas shared by each sub-area in the fourth gray level image after the binarization processing.

8. The method according to claim 3, 4, 6 or 7, wherein the image feature enhancement processing comprises:

Denoising the corresponding gray level image through image filtering;

And carrying out morphological processing on the gray level image after the denoising processing.

9. The method of claim 8, wherein morphologically processing the denoised grayscale image comprises:

removing the uniform background area in the denoised gray image to obtain a foreground area;

Removing the peripheral area in the foreground area to obtain an image with the peripheral area removed; the uniform background comprises uniform light sources and/or uniform textures, and the peripheral area is an area except for a communication area in which pixel values are concentrated in the foreground area;

and performing morphological processing on the image with the peripheral area removed.

10. The method of any of claims 1-7, wherein the determining fingerprint residual performance of the target housing based on the area of the fingerprint region and the contrast comprises:

And if the area of the fingerprint area is smaller than the preset area and the contrast is smaller than the preset contrast, judging that the fingerprint residue of the target shell is low.

11. A device for detecting fingerprint residue in a housing, the device comprising:

the fingerprint image acquisition unit is used for acquiring a fingerprint image, wherein the fingerprint image is an image obtained by shooting a fingerprint on the surface of the target shell;

The fingerprint area determining unit is used for performing image processing on the fingerprint image, and determining a target area in the fingerprint image and a fingerprint area with fingerprint lines collected in the target area;

a contrast determining unit that determines a contrast of the fingerprint region and the target region based on pixel information of the fingerprint region and pixel information of the target region;

And a fingerprint residual performance judging unit for judging the fingerprint residual performance of the target shell based on the area of the fingerprint area and the contrast.

12. The apparatus of claim 11, wherein if the pixel information includes RGB channel values; the contrast determination unit is specifically configured to:

Determining a sum of each of the RGB channel values of the fingerprint region and a sum of each of the RGB channel values of the target region;

and comparing the sum of each channel value in the RGB channel values of the fingerprint area with the sum of each channel value in the RGB channel values of the target area to obtain the contrast of each channel.

13. The apparatus according to claim 12, wherein the determining fingerprint area unit is specifically configured to:

determining the target area based on shooting parameters of the fingerprint image;

converting the image of the target area into a first gray scale image;

performing image feature enhancement processing on the first gray level image to obtain a second gray level image;

And carrying out binarization processing on the second gray level image to determine a fingerprint area which accords with fingerprint characteristics in the target area.

14. The apparatus according to claim 12, wherein the determining fingerprint area unit is specifically configured to:

determining the target area based on shooting parameters of the fingerprint image;

converting the image of the target area into a third gray scale image;

partitioning the third gray level image based on the areas of the shooting light source and the third gray level image to obtain a plurality of subareas;

respectively carrying out image characteristic enhancement processing on each sub-region in the third gray level image to obtain a fourth gray level image;

and the fingerprint area is determined by carrying out binarization processing on the fourth gray level image and combining the communication areas shared by each sub-area in the fourth gray level image after the binarization processing.

15. The apparatus of claim 11, wherein if the pixel information includes a gray value; the contrast determination unit is specifically configured to:

determining a sum of gray values of the fingerprint area and a sum of gray values of the target area;

and determining a comparison result of the sum of the gray values of the fingerprint area and the sum of the gray values of the target area as the contrast.

16. The apparatus according to claim 15, wherein the determining fingerprint area unit is specifically configured to:

determining the target area based on shooting parameters;

converting the image of the target area into a first gray scale image;

extracting a background area from the first gray level image, and removing the background area in the first gray level image;

performing image feature enhancement processing on the first gray level image with the background area removed to obtain a second gray level image;

And carrying out binarization processing on the second gray level image to determine a fingerprint area which accords with fingerprint characteristics in the target area.

17. The apparatus according to claim 15, wherein the determining fingerprint area unit is specifically configured to:

determining the target area based on shooting parameters;

converting the image of the target area into a third gray scale image;

partitioning the third gray level image based on the areas of the shooting light source and the third gray level image to obtain a plurality of subareas;

Extracting a background region from each sub-region in the third gray scale image, and removing the background region in each sub-region in the third gray scale image;

Respectively carrying out image characteristic enhancement processing on each sub-region in the third gray level image after the background region is removed to obtain a fourth gray level image;

and the fingerprint area is determined by carrying out binarization processing on the fourth gray level image and combining the communication areas shared by each sub-area in the fourth gray level image after the binarization processing.

18. The apparatus according to claim 13, 14, 16 or 17, wherein the determining fingerprint area unit is specifically configured to:

Denoising the corresponding gray level image through image filtering;

And carrying out morphological processing on the gray level image after the denoising processing.

19. The apparatus according to claim 18, wherein the determining fingerprint area unit is specifically configured to:

removing the uniform background area in the denoised gray image to obtain a foreground area;

Removing the peripheral area in the foreground area to obtain an image with the peripheral area removed; the uniform background comprises uniform light sources and/or uniform textures, and the peripheral area is an area except for a communication area in which pixel values are concentrated in the foreground area;

and performing morphological processing on the image with the peripheral area removed.

20. The apparatus according to any one of claims 11-17, wherein the fingerprint residue performance determining unit is specifically configured to:

And if the area of the fingerprint area is smaller than the preset area and the contrast is smaller than the preset contrast, judging that the fingerprint residue of the target shell is low.

21. A computer storage medium, characterized in that the computer storage medium stores a computer program which, when executed by a processor, implements the method according to any of claims 1-10.

22. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the method of any of claims 1-10.