RU2312395C1 - Method for sorting digital images for quality printing - Google Patents

Abstract

FIELD: processing of digital images.

SUBSTANCE: method includes following operations: recording of digital images from, for example, a digital camera, into memory block, which may be a part of a computer or a printer; analysis of digital images, contained in memory block, on basis of properties: image brightness, image blur level, red eye detection; recording of metadata into files of digital images, corresponding to character of found defects; digital images, on basis of analysis results, are classified as belonging to following categories: images which satisfy high quality standards and which are usable for printing, in other words, without defects, images, usable for printing after being edited, in other words, with correctable defects, images, not usable for printing, in other words, with uncorrectable defects; in memory block, separate sections are created for storage of each one of three aforementioned categories of digital images; files of sorted digital images are recorded in memory block together with metadata about the character of found defects.

EFFECT: conservation of processing time and consumables.

6 dwg

Description

The invention relates to the field of measurement technology, and more specifically to methods for automating the selection of digital images suitable for high-quality printing.

In the process of choosing digital photos for printing, the main criterion used by the average consumer is the quality of the images. Some images may not be suitable due to poor quality, i.e. be underexposed, overexposed or have low contrast or have problematic content.

Real defects in digital photographs include, but are not limited to, exposure defects, blur and glare from the flash, noise, JPEG artifacts, color imbalance, chromatic aberration, and other defects.

As analysis of digital photographic images shows, exposure problems are the most common defect. The following are various blur options and glare from the flash. On average, one in ten photographs is corrupted by noise, JPEG artifacts, or color imbalance. Chromatic aberrations are rare. At the same time, their appearance and isolation is predictable, so that various algorithms for adaptive search and correction can be used to combat them. Other types of defects cannot be detected without additional special measurements and assumptions. Often, defects are distributed unevenly across the image, and local correction is necessary, for example, in the case of spurious illumination or complex color imbalance.

Currently, several methods are known for automatically evaluating image quality to determine whether image quality is acceptable or if improvement is required. Basically, these methods relate to exposure quality. The determination of the correspondence of the exposure to the nature of the illumination of the recorded object or scene consists in assigning the following classes of images: underexposed, overexposed, low-contrast or normal, as well as the choice of exposure correction technique. In the simplest case, such a technique can be a histogram extension, positive or negative gamma correction, or prohibition of correction.

U.S. Patent Application Laid-open No. 2003/0151674 [1] proposes a method for evaluating the quality of images received by a digital camera, which includes checking the received image to determine the acceptable quality of the photo and providing feedback to the user who decides whether to save the image or re-photograph. The quality of a photograph is evaluated based on three characteristics: a) image sharpness; b) the image quality of a person; and c) the presence or absence of a flash during the shooting process. Quality criteria are calculated for each of the three quality characteristics and compared with a predefined threshold to determine whether the achieved level exceeds a threshold value. If all thresholds are exceeded, the user is informed that the received image is of acceptable quality.

U.S. Patent Application Laid-Open No. 2005/0270381 [2] provides a method for enhancing image acquisition capabilities implemented in a digital camera. The process is based on the analysis of the parameters of the resulting image contained in the metadata, and comparing them with valid parameters to determine the imperfection of the image. If there are no defects, the image is recorded. The user can be informed about image problems. If a defect in the image is detected, it is possible to re-image.

U.S. Patent Application Laid-Open No. 2004/0190789 [3] describes a methodology for characterizing exposure data based on an analysis of a global histogram of image brightness, implying, in turn, a subsequent correction. Automatic exposure analysis and the improvement process are divided into two main stages: the analysis and classification stage and the exposure correction stage. During processing of the brightness histogram, the classifier identifies the parameters used to analyze the exposure of the image: the effective histogram width, which is used to determine the histogram alignment range, the center and centroid of the histogram, which are used to determine whether the brightness should be increased to correct an underexposed image or reduced for an overexposed image . Thus, the correction of exposure errors is reduced to a) alignment of the histogram; b) increase or c) decrease the brightness of the image.

The autofocus function of digital cameras does not always work accurately. Photos taken in autofocus may not be in focus and, as a result, may be blurred. For some photographs, the entire image may not be in focus due to the movement of the camera during shooting or the autofocus function, i.e. when the digital camera cannot focus in any part of the image. In other cases, only part of the image is focused. For example, a digital camera focuses in the background instead of the foreground if the image is not correctly focused. In this case, finding unfocused images is a difficult task, because requires a foreground and background definition.

Image blur is one of the most important factors in the perception of image quality due to the accompanying loss of detail and image sharpness. Blurring the image can occur due to a large number of reasons, including, for example, defocusing the lens on the subject, defocusing the foreground or background due to the large aperture value, the movement of the subject relative to the camera, and even atmospheric haze. Image blur in traditional, analog photography could not be significantly reduced. However, in digital photography, blurring can be largely eliminated. Existing techniques for reducing image blur usually include a blur estimation unit and a blur elimination unit. If the blur estimation unit finds an image blur value above a certain threshold, the image is processed by the blur reduction unit. Some existing techniques for reducing image blur quantify the degree of blur and accordingly select the degree of correction. Other techniques for reducing image blur use an iterative procedure that cycles through the image in the blur reduction unit until the estimated degree of blur is below a certain threshold.

Doron Shaked and Ingeborg Tastl, "Sharpness Measure: Towards Automatic Image Enhancement", Hewlett-Packard Laboratories Technical Report HPL 2004-84, May 2004 [4] describes an algorithm for measuring total image sharpness, which determines how much image sharpness should be changed to achieve optimum. The algorithm estimates the global sharpness of the image, given a single value for the entire image. Thus, the sharpness of blurry images will be increased, while sufficiently sharp images will not be processed at all. The measure uses local frequency analysis, focused on the selection of features. In this regard, the method is free from the disadvantages of alternative methods. Frequency-domain analysis methods provide excellent sharpening measures for images of similar scenes, however, they are unsatisfactory when the scenes change. Feature-oriented methods focus on the characteristics of the images, however, the assumptions required for good performance are too strict for ordinary applications. The proposed measure of sharpness correlates well with visually perceived sharpness and is largely invariant to the image content. The proposed sharpening measure can be used to run an improvement algorithm that will increase the sharpness of the original image to its nominal value. Last but not least, the proposed sharpening measure is easy to implement and requires less computation than calculating a 3 × 3 convolution.

US Pat. No. 6,771,308 [5] describes a digital camera having a blur detector. The camera itself can print photos. A detector is built into the camera to detect camera shake during shooting, a medium for recording photos and the intensity of camera shake, a device for selecting recorded photos for printing, a controller for analyzing images that determines whether the selected image can be printed or not. In the future, the camera issues a warning when it considers that the corresponding image cannot be printed.

US Pat. No. 6,298,145 [6] describes an image processing system including a blur detector that determines an image blur value based on information contained in a compressed image, provided that it contains a face. The blur detector informs that the image is suitable for printing and / or visual presentation if the blur value is less than a predetermined threshold.

A red-eye defect continues to be the most common problem faced by users of digital cameras and imaging systems. It occurs due to reflection in the eye of the object and usually appears in the digital image as a reddish area where the black pupil of the object usually appears. An unnatural reddish gleam in the eye occurs due to internal reflections from the vascular membrane behind the retina, which is saturated with blood vessels. This effect is caused, in particular, by a small angle between the flash lamp of the camera and its lens. This angle is reduced with the miniaturization of a camera having a built-in flash. This trend will continue with further miniaturization of cameras and the introduction of flash lamps in camera phones.

US Pat. No. 6,252,976 [7] describes a computer program for detecting eye color defects in an image due to a flash lamp, performing the following steps: detecting areas having a skin color in a digital image; searching for areas having a skin color for groups of pixels with characteristics corresponding to a red eye defect and correcting color of pixels based on the position of the red eye region.

US Pat. No. 6,873,743 [8] describes an automatic system for detecting and removing red eyes in real-time digital images, including a red-eye detection module that detects red eyes if they exist without user intervention. If a defect is found in the image, then the part of the image surrounding the defective area is processed by the correction module, which reduces the red component of the red eyes while maintaining the remaining color characteristics of the problem area. The invention minimizes the computational cost required to detect and correct red eyes, and is thus suitable for applications requiring real-time processing of large volumes of digital images before archiving or printing. This system can process images found on personal computers, commercial printers or digital cameras as part of the acquisition process, or before rendering on handheld computers, mobile phones and other digital devices.

Along with brightness and blur, color processing or color reproduction is a key factor in determining the quality of a digital image. However, understanding and modeling the human visual system is far from complete, and there are several numerical models that can predict the results of color perception according to the very complex nature of our preferences. As a result, various camera manufacturers are developing their own color models and approaches to produce the most spectacular photos. Even for physically identical scenes, cameras from different manufacturers can give quite different colors in photographs due to different CCDs, processing algorithms, and color models used.

Thus, the assessment of the quality of digital images prepared for photo printing is performed automatically based on the analysis of the brightness of the digital image, the degree of blurring and detection of red eyes.

After the digital image quality has been assessed, this information is recorded in a special way.

A digital camera image file usually stores metadata. In the general case, metadata is any data associated with a file that differs from the data of the file that is usually available to the user. In most cases, modern digital cameras use the Exif (Exchangeable image file) format as the metadata format, which is recommended by the Japanese Industrial Electronics and Information Technology Association (see Exchangeable image file format for digital still cameras: Exif Version 2.2. JEITA CP-3451 Technical Report. April 2002 [9]). Exif metadata can be considered as an additional source of information in the process of automatic processing of digital images, and the content of this information is determined by the characteristics of the digital camera itself and includes its installed and measured parameters. At the same time, the user can use the most diverse utilities available on the software market to view and modify metadata for their further use, for example, for improvement and printing.

In addition, an automatic algorithm capable of detecting low-quality photographs through image analysis will make it possible to enter the necessary information about the implemented or planned image corrections into the metadata.

An example of the use of metadata for printing is PDF. Its use reduces cost and eliminates many of the problems associated with sending print jobs to the printer. In this case, the printer creates a PDF in accordance with its specifications, transferring the objects, images and fonts associated with the file.

Adobe's extensible metadata platform (XMP) is also known to allow you to embed file metadata directly into the file itself (see Robin L. Dale, Günter Waibel. "Capturing Technical Metadata for Digital Still Images". RLG DigiNews. Volume 8, Number 5 , 2004 [10] and "Adobe XMP-JDF-PDF: The Alphabet of Automation." Print Media. May / June 2004 [11]). For example, a digital camera automatically writes photo information to an image file, such as date and time, camera type, exposure, and lens settings. This information is metadata. Additional metadata can be added manually, such as the location of the picture, the name of the photographer, related information, etc., which will allow you to search for important information related to the shooting. Using XMP, this metadata is attached to the photo file and accompanies it during the entire processing process. The parameters of all changes that occur with the file are recorded in the metadata. Even if the source file is converted to another format, such as EPS, PDF or Photoshop, the metadata is preserved, providing the file’s information header during the entire processing process.

Currently, many printers have the ability to directly print from a digital camera or memory card, but only a few can be called truly stand-alone. The autonomy of the printing device is determined, in our opinion, by the presence of a hard disk, a set of slots for memory cards, a PictBridge port that allows printing or transferring photos to the built-in hard disk from an arbitrary digital camera, as well as automatic or interactive procedures for correcting digital images. It is also assumed that the user can manually mark the photos necessary for printing by viewing them on the display. However, in the standard version, which is determined primarily by the marketing (price) considerations of the manufacturer, the size of this display is small enough to accurately assess the quality of the image. A large number of printer models do not have a display at all, and the process of selecting images for printing takes a lot of time, given that modern digital camera memory cards store several hundred pictures at the same time.

Closest to the claimed invention is the solution described in the laid out application US No. 2002/0109854 [12], in which the detailed method for assessing the quality of digital images is considered as the basis for image correction and determining the number of required correction procedures, the degree of a specific correction procedure, its relative weight among other procedures, if any, as well as any combination of the above. Each improvement is assigned a rating, the maximum value of which is equal to one. Thus, if the image is enhanced, i.e. edited, one class, for example, exposure correction is performed, then it is assigned a rating of one; with two transformations - rating two, etc. Accordingly, the assigned rating may depend on the type of correction and / or the degree of improvement. If the image undergoes significant adjustment or several procedures of various types are used to adjust it, then the corresponding rating is increased, for example, to two. Therefore, each image is ranked according to the degree of correction or improvement that has been applied to that image. Greater improvement corresponds to a higher rating. An image that has undergone sufficient correction in accordance with the rating received is considered fit for print. The highest priority for printing is the image with the highest rating. This application is further considered as a prototype of the claimed invention.

The problem to which the claimed invention is directed is to develop a simple and reliable method for automatically dividing existing digital images into high-quality ones, i.e. suitable for printing without editing, conditionally quality, i.e. printable after editing or other processing, and printable images.

The subject of the invention is an automatic method for sorting digital images, which includes the following operations:

- carry out the recording of digital images, for example, obtained from a digital camera, in the memory unit, which may be part of a computer or printer;

- analyze the digital images contained in the memory unit, according to the signs:

- image brightness

- the degree of blurring,

- detection of red eyes;

- write metadata corresponding to the nature of the detected defects in digital image files;

- classify digital images according to the analysis into the following categories:

- Images that meet high quality standards and are suitable for printing, i.e. without defects,

- images suitable for printing subject to editing, i.e. with correctable defects,

- images unsuitable for printing, i.e. with irreparable defects;

- create separate sections in the memory unit for storing each of the three above categories of digital images;

- they record in the memory block files of sorted digital images along with metadata about the nature of the detected defects.

One of the advantages of the claimed invention is the automatic selection of the best images from a set of digital photographs uploaded to the printer from its own memory unit or from the memory unit that controls the printer of the computer. High-quality photographs are recorded in the corresponding directory of the memory block and can later be printed in batch mode without additional analysis. This gives an advantage both in processing time and in terms of saving consumables, i.e. photo paper and print cartridges.

The features identified during the initial analysis of digital images are flagged with a special flag in Exif metadata or in special technical metadata, so that images can be printed without further classification. In addition, defect information may be recorded in the Exif header (or metadata) as “exposure defect”, “blur defect” or “red eye defect”, which will be used to further improve defective images for printing.

In the process of downloading digital images from a digital camera to a storage device or in the process of analyzing existing images, for example, on the printer’s built-in hard drive, the classification allows you to select high-quality photos in a separate subdirectory, from where printing can be performed directly or with additional printer correction. The proposed method reduces the cost of consumables and printing time.

The most significant improvements that the claimed invention provides:

- a method for sorting digital images for printing, divided into three categories:

- Images that meet high quality standards and

suitable for printing;

- Images not suitable for printing, but which may be

improved and subsequently printed;

- images not suitable for printing, for which improvement

impossible.

- This method reduces the consumption of photo paper for printing and the time spent on printing, increasing the efficiency of the photo printing process.

- The analysis of images and their evaluation are carried out automatically, which does not require expert intervention or other external participation.

- Evaluation of image quality, in particular, the definition of high-quality images, significantly increases the productivity of the corresponding software for photo printing by eliminating image improvement procedures, if they are not really required.

- Configuring the software architecture for evaluating image quality allows you to optimize the image enhancement module by adding, removing, or organizing appropriate processing filters.

Of particular note is the fact that in the claimed solution metadata characterizing the image correction process is generated and recorded in the enhanced image file, which eliminates the multiple assessment of image quality or correction. This point is extremely important, since most digital image enhancement algorithms do not include preliminary image analysis. This circumstance is critical, since repeated application of the correction procedure may lead not to improvement, but to deterioration of the final image.

The invention is explained further on the basis of the following graphic materials:

Figure 1. A block diagram of a device that implements the inventive method for sorting automatic analysis of digital images uploaded to a printer or stored in a memory unit, for example, on an integrated printer hard disk.

Figure 2. An algorithm for classifying digital images, correcting metadata in the Exit header, and sorting images before printing.

Figure 3. Digital Image Classification Algorithm.

Figure 4. The metadata correction algorithm in the Exif header.

Figure 5. Image sorting algorithm.

6. Classification results.

Figure 1 shows a diagram of the interaction of the main components of the printer, which performs automatic analysis and evaluation of digital images received from the input device 101, which can be a digital camera, memory card, computer or arbitrary device that can store images containing information, or which digital images can be read. Digital images received from input device 101 can be written to a memory unit, for example, a drive such as a printer hard disk, or another device that can store images containing information, or from which digital images can be read, for later storage or classification . The memory unit may also contain instructions used by the software in the form of a module 104 for automatic analysis and correction of digital images. It is understood that the memory block consists of two parts - the program memory, in which the code for analysis, correction and printing of digital images is executed, and the data memory, where classified images are written. The process of analyzing and evaluating images is carried out by a processor 105, which executes a program recorded in a memory unit in component 102. Information exchange is carried out via a data bus 103. After processing, the image is sent to the memory unit 102, for example, to a drive such as a printer hard disk. The metadata corresponding to the image correction is generated and written to a digital image file containing the enhanced image to prevent multiple image evaluation or correction. Subsequently, data can be read from the printer’s hard disk to one or more output devices 106, which are either a printer printing mechanism for receiving photo prints, or arbitrary devices that can store images containing information, or a device from which digital images can be read out.

Figure 2 shows the algorithm of the process of classifying digital images for printing. At step 201, digital images are received from the input device. At step 202, the resulting images are classified based on brightness analysis, blur analysis, and red-eye analysis divided into the following classes: images that meet high quality standards; images unsuitable for printing, but which can be improved and subsequently printed; Pictures not suitable for printing, for which improvement is not possible. At step 203, the classification results are recorded in the corresponding Exif field for subsequent sorting and printing of images. At step 204, high-quality images are stored in a memory block, for example, in a hard drive such as a hard disk, a printer in the Normal subdirectory, images that should be improved before printing are recorded in the Improvement subdirectory, and images that contain fatal defects are recorded in the Defective subdirectory. At step 205, quality images are printed (upon request).

Figure 3 shows the steps of the digital image classification process. At step 301, a classification by image brightness occurs.

To assess the quality of exposure, it is advisable to use the method described in [3], according to which, to assess that the image is not suitable for printing, calculate the average brightness value of the analyzed digital image. If the average image brightness is less than a certain threshold value I1 and greater than a certain threshold value I2, the image is considered unsuitable for printing, its improvement is impossible. Otherwise, the image is classified as Normal (printable); Low contrast, Underexposed or Overexposed - unsuitable for printing, requires improvement. In a specific case, it is proposed to use the following values for an 8-bit image: I1 = 10, I2, = 245.

At step 302, an analysis of the degree of blurriness of the image is performed, for example, according to the technique described in the published report by Suk Hwan Lim, Jonathan Yen and Peng Wu "Detection of Out-Of-Focus Digital Photographs", Hewlett-Packard Laboratories Technical Report HPL-2005-14 January 20, 2006 [14]. The image is assigned a sharpness range from 1 to 5, where 1 is very blurry, 5 is very good. If the image has a sharpness rank of 5, then this is a quality image suitable for printing. If the sharpness rating is 3-4, then this image can be improved, if the sharpness is 1-2, then this image is defective and cannot be improved.

At step 303, the image is analyzed for red eyes. The algorithm can be roughly divided into 3 stages:

but. detecting objects that could potentially be seen as red eyes;

b. neglecting false positives;

from. we localize (segment) red eyes.

To determine the presence of red eyes, it is recommended, in particular, to use the algorithms described in [6] and [7].

Figure 4 shows the algorithm for assigning a special flag according to the classification results to the Exif header of the image based on the classification results. In step 401, special fields and a “Defect” are created in the Exif header or technical image metadata. If the image is classified as defective in step 402, the Defective flag is written in the Quality field of the Exif header in step 403. If the image is classified as Improvement in step 404, the Improvement flag is written in the Quality field of the Exif header and type a defect (“Exposure defect”, “Blur defect”, or “Red-eye defect”) is written in the “Defect” field in step 405. Otherwise, the “Normal” flag is written in the “Quality” field of the Exif header in step 406.

Figure 5 shows a flowchart for sorting classified images into directories (folders) of the printer hard disk depending on the classification flag of the Quality field in the Exif header. If the “Defective” flag is detected in step 501, then in step 502 the image is recorded in the “Defective” directory; otherwise, in step 503, the “Quality” field in the Exif header is analyzed for the presence of the “Improvement” flag. If the flag is set to "Improvement", at step 504 the image is recorded in the "Improvement" subdirectory, otherwise at step 505, the image is recorded in the "Normal" subdirectory.

6 shows the process of classifying source digital images and recording them in the corresponding sub-directories of a memory unit, for example, a printer hard disk, according to the classification results.

As an example, seven images are presented at the input of the system, numbered from 1 to 7. Based on the classification results, images 2, 3, and 4 are defined as high-quality and suitable for printing. The "Normal" flag is written in the "Quality" field of the Exif header and the images are moved to the "Normal" directory. Images 1, 5, and 7 are classified as unsuitable for printing because they need improvement. They have the “Improvement” flag set in the “Quality” field of the Exif header, and they are recorded in the “Improvement” directory. The reason for this classification is that image 1 is rated as Underexposed, image 5 is classified as Overexposed, and image 7 is classified as having red eyes. Image data must be improved before printing. Image 6 is classified as blurry, without the possibility of improvement. It is assigned the “Defective” flag in the “Quality” field of the Exif header, and it is recorded in the “Defective” directory.

A method for sorting digital images, including the image quality assessment described above, can be implemented both in printing systems that include a hard disk, and in automatic image enhancement programs, both a plug-in and individual applications. In most implementations of the present invention, the user can take advantage of automatic quality analysis and correction without the use of additional applications. It should be noted that an application of this type, which classifies images both globally and locally, makes the performance of the automatic procedure higher, which eliminates incorrect image improvement when high-quality digital images are degraded due to incorrect classification of defects.

Thus, the present invention improves a user's ability to process large archives of digital images by analyzing metadata associated with digital images during post-processing post-processing. In the future, if there are problems with the quality of the received images, the user can access the metadata contained in the image file in order to perform its corresponding correction. The same is true for automatic mode. In particular, the user can only print images that have been rated “Normal,” without any further improvements. In another implementation, the user can record on a separate medium only the pictures from the "Normal" subdirectory for transferring them to the darkroom.

Improving the quality of images with the flag “Improvement” is carried out depending on the type of defect (“Exposure defect”, “Blur defect” or “Red eye defect”). If the image has an “Exposure Defect”, then its correction can be performed on the basis of the exposure correction algorithm described in [3]. If the image has a “Blur Defect”, it is advisable to apply sharpening methods, for example, using a number of filters that enhance high frequencies. Various implementations of such filters are given in [13]. If the image has a red-eye defect, the correction procedure is similar to that described in [7] and [8].

It is noteworthy that the inventive method can also be successfully implemented using a printing device equipped with a built-in hard disk and software for automatic assessment of image quality, which provides the user with ample opportunity to place and analyze image quality without resorting to additional application software packages.

In other words, a printer equipped with an image quality rating system will allow you to organize the printing process in batch mode, even for sets of not very high-quality images.

Claims (1)

A method of sorting digital images for high-quality printing in automatic mode, which includes the following operations: recording digital images, for example, obtained from a digital camera, in a memory unit, which may be part of a computer or printer, analyze digital images contained in the memory unit according to the image brightness, degree of blur, red eye detection, write metadata corresponding to the nature of the detected defects into digital image files, classify digital images according to the analysis into the following categories: images that meet high standards quality and printability, i.e. without defects, images suitable for printing provided that they are edited, i.e. with correctable defects, images unsuitable for printing, i.e. with irreparable defects, create separate sections in the memory unit for storing each of the three above categories of digital images, write the sorted digital image files in the memory unit along with metadata about the nature of the detected defects.

Images