CN114676360B

CN114676360B - Image processing method, device, electronic equipment and storage medium

Info

Publication number: CN114676360B
Application number: CN202210295315.8A
Authority: CN
Inventors: 郑思晓; 姜文浩; 李志锋; 刘威
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-03-23
Filing date: 2022-03-23
Publication date: 2024-09-17
Anticipated expiration: 2042-03-23
Also published as: CN114676360A

Abstract

The application discloses an image processing method, an image processing device, electronic equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: acquiring at least one original image associated with the content item; identifying a first region in the at least one original image; and performing color mapping on pixels of a second area except the first area in the at least one original image to obtain at least one target image associated with the content item. According to the application, the color of the first area in the original image is kept unchanged in the color changing process, and only the second area is subjected to color mapping, so that the target image obtained by color changing is a color matching scheme derived on the basis of the original image, the display mode of the image related to the content item is greatly enriched, the display effect of the image related to the content item is improved, and the method has great practical significance in the interactive design of the advertisement landing page especially for the content item of advertisements.

Description

Image processing method, device, electronic equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to an image processing method, an image processing device, an electronic device, and a storage medium.

Background

Currently, when a User clicks a content item to be placed, the User opens and presents the User with a UI (User Interface) Interface, in which the content item to be clicked is usually displayed with expanded content, such as details of an item, promotion content of an activity, and the like.

In the UI design for the above landing page, it is generally required that the throwing party make each image (such as a poster, a detail drawing, a model drawing, a product drawing, etc.) to be displayed in the landing page, and then display each image in the landing page according to a preset typesetting or layout, but the display mode of the image contained in the landing page is relatively single and the display effect is relatively poor.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, electronic equipment and a storage medium, which can enrich the display mode of an image associated with a content item and improve the display effect of the image associated with the content item. The technical scheme is as follows:

in one aspect, there is provided an image processing method, the method including:

acquiring at least one original image associated with the content item;

Identifying a first area in the at least one original image, wherein the first area refers to an area without color change in the original image;

and performing color mapping on pixel points of a second area except the first area in the at least one original image to obtain at least one target image associated with the content item, wherein the target image is the original image after the color of the first area is unchanged and the color of the second area is changed.

In one aspect, there is provided an image processing apparatus including:

An acquisition module for acquiring at least one original image associated with a content item;

The identification module is used for identifying a first area in the at least one original image, wherein the first area refers to an area without color change in the original image;

and the mapping module is used for carrying out color mapping on pixel points of a second area except the first area in the at least one original image to obtain at least one target image associated with the content item, wherein the target image is the original image after the color of the first area is kept unchanged and the color of the second area is changed.

In one possible implementation, the identification module includes:

the detection unit is used for carrying out target detection on any original image to obtain a candidate region in the original image;

the classification unit is used for inputting the candidate region into a color change classification model, outputting a color change detection result through the color change classification model, and detecting whether the candidate region needs color change or not;

A first determining unit configured to determine, as the first region, a candidate region for which the color change detection result indicates that color change is not required.

In one possible embodiment, the detection unit is configured to:

and carrying out rectangle detection on the original image, and determining the detected rectangle area as the candidate area.

In one possible implementation, the identification module includes:

the first identification unit is used for carrying out object identification on any original image to obtain an object identification result, and the object identification result represents whether the original image contains a target object or not;

And a second determining unit configured to determine, as the first area, an area in which the target object is located, in a case where the object recognition result indicates that the original image includes the target object.

In one possible implementation, the second determining unit is configured to:

Performing object segmentation on the original image to obtain a mask image for identifying the target object;

and determining the first area where the target object is located based on the mask image.

In one possible implementation, the mapping module includes:

the assignment unit is used for assigning the pixel points in the first area to be target pixel values in the at least one original image to obtain at least one first image;

A second identifying unit, configured to identify at least one dominant hue based on the at least one first image, where the dominant hue characterizes a color distribution tendency of pixels in the at least one first image;

the mapping unit is used for carrying out color mapping on the pixel points in the at least one first image to obtain at least one second image;

The assignment unit is further configured to assign, in the at least one second image, a pixel point located in the first area to a pixel value corresponding to a pixel point in a corresponding position in the original image, so as to obtain the at least one target image.

In one possible implementation, the second identifying unit is configured to:

splicing the at least one first image to obtain a spliced image;

Clustering pixel points in the spliced image based on color vectors of the pixel points to obtain at least one color cluster;

Determining a cluster center of the at least one color class cluster as the at least one dominant hue.

In a possible implementation manner, the mapping unit is configured to:

Converting the at least one first image from a red-green-blue RGB color space to a hue-saturation-brightness HSV color space;

For any main tone, in the HSV color space, keeping the saturation S and the brightness V of the main tone unchanged, and modifying the tone H of the main tone to obtain a color-changing tone corresponding to the main tone;

Determining a color mapping relation of pixel levels based on a main tone mapping relation of a main tone and a color-changing tone, wherein the color mapping relation represents the corresponding relation of different pixel values before and after color changing;

and mapping pixel values of pixel points in the at least one first image based on the color mapping relation to obtain the at least one second image.

In one possible embodiment, the hue difference between any two dominant hues is equal to the hue difference between the corresponding two color-changing hues.

In one possible embodiment, the apparatus further comprises:

A determining module, configured to determine a white area in the at least one original image, where a difference between a pixel value of a pixel point in the white area and a white value is less than a target threshold;

the assignment unit is further configured to assign, in the at least one second image, a pixel point located in the white area to a pixel value corresponding to a pixel point in a corresponding position in the original image.

In one aspect, an electronic device is provided that includes one or more processors and one or more memories having at least one computer program stored therein, the at least one computer program being loaded and executed by the one or more processors to implement an image processing method as described above.

In one aspect, a storage medium is provided in which at least one computer program is stored, the at least one computer program being loaded and executed by a processor to implement an image processing method as described above.

In one aspect, a computer program product or computer program is provided, the computer program product or computer program comprising one or more program codes, the one or more program codes being stored in a computer readable storage medium. The one or more processors of the electronic device are capable of reading the one or more program codes from the computer-readable storage medium, and executing the one or more program codes to enable the electronic device to perform the image processing method described above.

The technical scheme provided by the embodiment of the application has the beneficial effects that at least:

By keeping the color of the first area unchanged during the color change for the original image associated with the content item, and performing the color mapping only for the second area, the color-changed target image is another color scheme derived on the basis of the original image, which color scheme does not require manual operation of the picture repair application by the user, the first area which cannot be changed in color can be intelligently identified, and the color of the first area is unchanged before and after color change, so that the display mode of the image related to the content item is greatly enriched and the display effect of the image related to the content item is improved under the condition that the color change accuracy and the color change effect are ensured.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic view of an implementation environment of an image processing method according to an embodiment of the present application;

FIG. 2 is a flowchart of an image processing method according to an embodiment of the present application;

FIG. 3 is a flowchart of an image processing method according to an embodiment of the present application;

fig. 4 is a schematic flow chart of an image processing method provided in an embodiment of the present application;

FIG. 5 is a schematic flow chart of a preprocessing module provided by an embodiment of the present application;

FIG. 6 is a schematic flow chart of a post-processing module provided by an embodiment of the present application;

Fig. 7 is a schematic structural view of an image processing apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

The terms "first," "second," and the like in this disclosure are used for distinguishing between similar elements or items having substantially the same function and function, and it should be understood that there is no logical or chronological dependency between the terms "first," "second," and "n," and that there is no limitation on the amount and order of execution.

The term "at least one" in the present application means one or more, meaning "a plurality" means two or more, for example, a plurality of first positions means two or more first positions.

The term "comprising at least one of A or B" in the present application relates to the following cases: only a, only B, and both a and B.

User-related information (including, but not limited to, device information, personal information, behavioral information, etc.), data (including, but not limited to, data for analysis, stored data, presented data, etc.), and signals, as applied to a particular product or technology in accordance with the methods of embodiments of the present application, are subject to user approval, consent, authorization, or substantial authorization by parties, and the collection, use, and processing of the related information, data, and signals requires compliance with relevant national and regional laws and regulations and standards. For example, the original images associated with the content items referred to in the present application are all acquired with sufficient authorization.

Artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) is the theory, method, technique, and application system that simulates, extends, and extends human intelligence using a digital computer or a machine controlled by a digital computer, perceives the environment, obtains knowledge, and uses the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

Computer Vision (CV) is a science of how to "look" at a machine, and more specifically, to replace a camera and a Computer to perform machine Vision such as identifying and measuring a target by human eyes, and further perform graphic processing, so that the Computer is processed into an image more suitable for human eyes to observe or transmit to an instrument to detect. As a scientific discipline, computer vision research-related theory and technology has attempted to build artificial intelligence systems that can acquire information from images or multidimensional data. Computer vision techniques typically include image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition ), video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D techniques, virtual reality, augmented reality, synchronous positioning, and map construction, among others.

With research and advancement of artificial intelligence technology, research and application of artificial intelligence technology is being developed in various fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned, automatic driving, unmanned aerial vehicles, robots, smart medical treatment, smart customer service, etc., and it is believed that with the development of technology, artificial intelligence technology will be applied in more fields and with increasing importance value.

The scheme provided by the embodiment of the application relates to technologies of artificial intelligence, such as target detection in images, image classification and the like, and the terms related to the embodiment of the application are explained below:

Floor page (LANDING PAGE): also called landing page or guide page, refers to a UI interface (e.g. web page) that is opened and presented to a user further after a potential user clicks on a content item such as an online advertisement, search result of a search engine, and promotion mail in a web advertisement. Landing pages typically display the expanded content of the goods or services to which the clicked advertisement, search result link, promotional mail, etc., is directed, such as details of the goods or services, posters, model drawings, merchandise drawings, promotional drawings, etc.

Gaussian Blur (Gaussian blue): also known as gaussian smoothing, is an image smoothing technique that is commonly used to reduce image noise and to reduce the level of detail. The smooth image is obtained by Gaussian blur technology, the visual effect of the smooth image is like that of the smooth image observed by a ground glass, and the smooth image is obviously different from the effect of lens out-of-focus imaging and the effect of common illumination shadow. Gaussian smoothing is also used in the preprocessing stage of computer vision algorithms to enhance the image effect of the image at different scale sizes. From a mathematical perspective, the gaussian blur process of an image is the convolution of the image with a normal distribution, which is also called gaussian blur, because it is also called gaussian. Since the fourier transform of a gaussian function is another gaussian function, gaussian blur is a low pass filter for an image.

Contour detection (Contour Detection): the method is characterized in that in a digital image containing a target and a background, the background and the texture inside the target as well as the influence of noise interference are ignored, and a certain technology and method are adopted to realize the process of extracting the target contour. Contour detection is an important basis for object detection techniques.

Object Detection (Object Detection): the task of object detection is to find all objects of interest in an image and determine their positions, which is one of the core problems in the field of computer vision. The object of interest related to the embodiment of the application refers to a rectangular area in an original image, and the object detection for the rectangular area is simply referred to as rectangular detection, and the detected rectangular area is usually represented by a rectangular frame.

Object identification: the task of object recognition is to recognize whether an image contains a certain type of specified object (such as facial recognition, article recognition and the like), and the object recognition is always the most challenging problem in the field of computer vision due to different appearances, shapes and postures of various objects and the interference of factors such as illumination, shielding and the like during imaging. The embodiment of the application relates to a given target object, which is used for identifying an original image and judging whether the original image contains the target object or not.

Image segmentation: the method is characterized in that the image is represented as a set of physically significant connected areas, namely, the objects and the backgrounds in the image are respectively marked and positioned according to the prior knowledge of the objects (namely, the foreground) and the background, and the objects are separated from the background or other pseudo objects. The embodiment of the application relates to a mask image for identifying a target object, which is obtained by finely dividing the target object from an original image by utilizing an image dividing technology under the condition that an object identification result indicates that the original image contains the target object.

Clustering: clustering is a technique for finding such an internal structure, and a clustering algorithm is a well-known algorithm in unsupervised learning. The embodiment of the application relates to a data set formed by color vectors of a plurality of pixel points in an image, wherein data members in the data set are all color vectors of one pixel point, a clustering process refers to a process of classifying and organizing pixel points with similar color vectors, namely, close colors, and a result obtained based on the color vector clustering is called a color class cluster, in other words, the color class cluster refers to a subset formed by each color vector clustered to the current class cluster through the color vector.

K means clustering (K-Means Clustering): a cluster analysis algorithm for iterative solution comprises the following steps: k pixel points are randomly selected from a data set formed by color vectors of all pixel points in an image to serve as initial clustering centers, then the distance between each color vector and each clustering center is calculated, and each color vector is distributed to the closest clustering center. The color vectors assigned to the same cluster center represent one cluster of color classes. For any color class cluster, the cluster center of the current color class cluster is recalculated according to the color vectors in the cluster each time the distribution of all the color vectors is completed. The above process will be repeated until a certain termination cluster condition is met. Termination cluster conditions include, but are not limited to: no (or a minimum number of) color vectors are reassigned to different color class clusters; or no (or a minimum number of) cluster centers of the color class clusters are changed again; or, the sum of squares error is locally minimal, etc.

RGB color space: any color in the RGB color space, which can be characterized by a three-dimensional color vector of [ R, G, B ], can be obtained by using the variation of three color channels of Red (Red, R), green (Green, G), blue (Blue, B) and the superposition of the three color channels, is also called an RGB color mode, and is a color standard in the industry, and the color standard includes almost all colors perceived by human vision, and is one of the most widely used color systems.

HSV color space: any color located in the HSV color space, also called hexagonal pyramid model (Hexcone Model), is a color space created by a.r.smith in 1978 according to the visual properties of the colors, also called hexagonal pyramid model, can be obtained using the variation of Hue (Hue), saturation (Saturation), brightness (Value) and their superposition with each other, in other words, any color can be characterized using a three-dimensional color vector of [ H, S, V ].

LAB color space: a color scheme defined by the CIE (International Commission On Illumination ). Any color in nature can be expressed in the LAB color space, which is larger than the RGB color space, so any color in the RGB color space can be mapped in the LAB color space. In addition, the LAB color space describes human visual senses in a digital manner, independent of the device, so it compensates for the deficiency that the RGB color patterns must depend on the device color characteristics. In the LAB color space, the use of any color can be characterized using a three-dimensional color vector of [ L, a, b ], L representing the brightness of the color, positive for red, negative for green, positive for yellow, and negative for blue.

The system architecture of the embodiment of the present application is described below.

Fig. 1 is a schematic view of an implementation environment of an image processing method according to an embodiment of the present application. Referring to fig. 1, the implementation environment includes a terminal 101 and a server 102:

the terminal 101 is configured to provide a color matching derivation service for images associated with a content item, where the color matching derivation service refers to, given one or more images and the number N of color schemes, the machine automatically derives N color schemes for each given image, where N is an integer greater than or equal to 1. The terminal 101 has installed and running thereon an application program supporting processing of images, optionally including: at least one of a browser application, a web page making application, an advertisement making application, a picture repairing application, a photographing application, an audio-video application, a short video application, a live broadcast application, an instant messaging application, a content sharing application, or a social application.

Illustratively, the program code for processing images is embedded in the application program, so that after the user inputs at least one original image and designates the number N of color schemes, the terminal 101 can run the program code to derive N color schemes for each original image, i.e., to output N color-converted target images for each original image.

The terminal 101 and the server 102 are connected by a wired network or a wireless network.

The server 102 is an electronic device for providing a background service for the application program, and the server 102 includes: at least one of a server, a plurality of servers, a cloud computing platform, or a virtualization center. Optionally, the server 102 takes on primary image processing work and the terminal 101 takes on secondary image processing work; or the server 102 takes on secondary image processing work and the terminal 101 takes on primary image processing work; or the terminal 101 and the server 102 cooperatively perform image processing work with a distributed computing architecture.

Optionally, the server 102 is a stand-alone physical server, or a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), and basic cloud computing services such as big data and artificial intelligence platforms.

In some embodiments, the terminal 101 independently executes the image processing method, so that the computing load of the server 102 can be reduced, and the occupation of the processing resources of the server 102 in the process of processing the image is avoided. At this time, the terminal 101 invokes the program code embedded in the local application program to perform an image processing task.

In some embodiments, the terminal 101 performs the image processing method cooperatively through information interaction with the server 102, that is: after acquiring at least one original image, the terminal 101 responds to the triggering operation of a user on an intelligent color matching option, inputs the number N of one color matching scheme, sends a color matching instruction carrying the at least one original image and the number N of the color matching schemes to the server 102, the server 102 responds to the color matching instruction, performs color matching on each original image for N times only on pixels in a second area after recognizing that the color matching is not needed in the first area, can obtain N target images, repeats the operation until all the original images are traversed, finally can obtain the derived N target images corresponding to the at least one original image, the server returns the N color matching schemes to the terminal 101, and the terminal 101 can visually present the N color matching schemes so as to facilitate the user to select which color matching schemes need to be put in. At this point, some of the image processing effort can be migrated to the server 102 to maintain higher system performance on the terminal.

Alternatively, terminal 101 refers broadly to one of a plurality of terminals, the device type of terminal 101 including, but not limited to: at least one of a vehicle-mounted terminal, a television, a smart phone, a smart speaker, a smart watch, a tablet computer, a smart voice interaction device, a smart home appliance, an aircraft, an electronic book reader, an MP3 (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3) player, an MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) player, a laptop portable computer, or a desktop computer. The following embodiments are illustrated with the terminal comprising a smart phone.

Those skilled in the art will appreciate that the number of terminals 101 described above can be greater or fewer. For example, the number of the terminals 101 is only one, or the number of the terminals 101 is several tens or hundreds, or more. The embodiment of the present disclosure does not limit the number and device type of the terminals 101.

In the UI design for the above landing page, it is generally required that the throwing party make each image (such as a poster, a detail drawing, a model drawing, a product drawing, etc.) to be displayed in the landing page, and then display each image in the landing page according to a preset typesetting or layout, but the display mode of the image contained in the landing page is relatively single and the display effect is relatively poor. Moreover, the manufacturing platform for the landing page is not available, color matching derivative service for the landing page can be provided, if a throwing party wants to change the color of an image in the landing page, the image can only be imported into a picture repairing application, and the pixel value of each pixel point in the image is modified so as to realize the integral color change of the image, so that the technical threshold of the picture repairing application is higher, the technology threshold of non-professional people is not friendly, the operation difficulty is greatly improved, and on the other hand, for the landing page with a complicated structure, not all areas need to be changed, such as some commodity detail drawings, model drawings, real shot photos and the like, if the commodity is changed, the user can generate the cheat of inconsistent pictures and real objects, and therefore, the color change accuracy and effect are poor.

In view of this, an embodiment of the present application provides an image processing method, which can design an intelligent color matching derivative service for content items such as advertisements, publicity films, popularization links, and the like, so as to automatically derive landing pages of content items made by a delivery party, and apply a plurality of new color matching schemes, so that original landing pages and landing pages of new color matching schemes have rich color distribution, improve a display mode of images in the landing pages, and optimize a display effect. In addition, the image processing method can keep the first area which is not to be changed in the landing page from being changed in color, such as the first area of a photo, a commodity, a model and the like, and keep the color unchanged in the color changing process, so that the image in the landing page is prevented from generating information transfer errors before and after color changing, inaccurate commodity or service introduction pictures are prevented from being displayed to consumers, and higher color changing accuracy and good color changing effect can be kept.

Fig. 2 is a flowchart of an image processing method according to an embodiment of the present application. Referring to fig. 2, the embodiment is performed by an electronic device, and the electronic device is taken as a terminal for explanation, and the embodiment includes:

201. The terminal obtains at least one original image associated with the content item.

The content item according to the embodiment of the present application refers to an information item for carrying specific content, where the specific content includes goods, services, notices, announcements, information, etc. to be exposed, for example, the content item is various advertisements, publicity films, promotion links, bulletin messages, search results of a search engine, marketing mails, etc., and the embodiment of the present application does not specifically limit the content item.

The original image to which the embodiment of the present application relates refers to an image associated with a content item. Alternatively, the original image is an image that is produced by the delivery party and is used for displaying a landing page of the content item, or the original image is an image that is included in the content item itself, which is not particularly limited in the embodiment of the present application. The original image may be configured to different contents based on service requirements of a delivering party, for example, when a content item is a commodity advertisement, a landing page refers to a commodity detail page, and the original image includes a commodity display diagram, a model trial diagram, a commodity specification diagram, and the like.

In some embodiments, a user initiates an application on a terminal that supports processing images, displays an image editing interface in the application, and imports at least one original image associated with a content item in the image editing interface, optionally the original image is a separately produced image, and presents an expanded content of the content item by combining a plurality of separately produced images in a landing page, or the original image is a video frame selected from videos associated with the content item, the selected video frames being either a key frame or a non-key frame, the type of the original image being not particularly limited by embodiments of the present application.

In some embodiments, the original image is an image obtained by locally calling the camera component to shoot by the terminal, or the original image is an image obtained by processing the locally shot image, or the original image is an image obtained by transmitting in other terminals or a computer readable storage medium, or the original image is an image stored in advance in a local database, or the original image is an image downloaded from a cloud, and the source of the original image is not specifically limited in the embodiments of the present application.

In some embodiments, after the user imports the at least one original image in the image editing interface, a triggering operation is performed on an intelligent color matching option provided in the image editing interface, and the triggering proceeds to step 202, where the intelligent color matching option is used for performing color conversion of a partial area on each input original image, so as to obtain a color-converted target image.

202. The terminal identifies a first area in the at least one original image, wherein the first area refers to an area without color change in the original image.

The first region refers to a region of the original image where color change is not required, and thus the first region is also referred to as a non-color change region, in other words, the first region characterizes a region where color change is not required in the color change process.

Optionally, the first region comprises at least one of: in other words, since the areas which cannot be changed such as the commodity display image and the model trial image are usually made into rectangular areas, the first areas which do not need to be changed can be selected from the rectangular areas; or, in other words, the target object (such as a commodity, an article, a model, a speaker, etc. associated with the content item) in the original image does not need to be changed in color, so that the target object can be segmented from the original image, and the segmented target object is used as a first area without changing color.

In some embodiments, in the case that the first area includes a rectangular area without color change in the original image, the terminal performs rectangular detection on the original image to obtain one or more rectangular areas in the original image, inputs the rectangular areas into a rectangular classifier (a color change classification model), determines whether the input rectangular area needs to be color-changed through the rectangular classifier, and then determines the rectangular area, which indicates that the rectangular area does not need to be color-changed, as the first area, where the rectangular classifier is optionally a classification model for determining whether the input rectangular area needs to be color-changed.

In some embodiments, when the first region includes the target object in the original image, the terminal performs object recognition on the original image to determine whether the original image includes the target object, and then, when the original image includes the target object, performs object segmentation on the original image, and determines the region where the segmented target object is located as the first region.

In some embodiments, in the case that the first area includes both a rectangular area without color change and a target object, the rectangular area indicated by the rectangular classifier without color change and the area where the segmented target object is located are determined as the first area, in other words, the first area is a union between the rectangular area without color change and the target object, and the embodiment of the present application does not specifically limit the first area.

203. And the terminal performs color mapping on pixel points of a second area except the first area in the at least one original image to obtain at least one target image associated with the content item, wherein the target image is the original image after the color of the first area is unchanged and the color of the second area is changed.

In some embodiments, after determining the first area without color change in the step 202, each original image can be divided into a first area and a second area, where the second area refers to an area of the original image except the first area. Then, for each original image, only the pixel points in the second area are subjected to color mapping, and the color of the pixel points in the first area is kept unchanged, so that a target image corresponding to the original image can be obtained, the above operation is repeatedly performed until the target image corresponding to each original image is obtained, at least one target image corresponding to each original image can be obtained at this time, the target images form a derived color scheme, if the number N of color schemes indicated by the user is greater than 1, then only the above step 203 is required to be performed N times to perform N different color mapping, and N color schemes can be obtained.

Further, after acquiring the N color schemes, the terminal may display an initial scheme formed by the at least one original image and the N color schemes derived intelligently by the machine in the image editing interface, so that a user can select which color schemes to sample in the landing page for use, for example, the user can select the initial scheme or some or all of the N color schemes for use, so that when the content item is put, any consumer clicks the content item to enter the landing page, a certain color scheme is randomly used, or under the condition of full authorization and independent consent of the consumer, the color preference of the consumer is analyzed to obtain the color scheme with the highest degree of adaptation with the consumer, so as to provide a rich and personalized landing page display scheme to prompt the display effect of the image associated with the landing page.

All the above optional solutions can be combined to form an optional embodiment of the present disclosure, which is not described in detail herein.

According to the method provided by the embodiment of the application, the color of the first area is kept unchanged in the color changing process by aiming at the original image related to the content item, and the color mapping is carried out on the second area only, so that the target image obtained by color changing is another color matching scheme derived on the basis of the original image, the color matching scheme does not need manual operation of a user to repair the image, the first area which cannot be changed in color can be intelligently identified, and the color of the first area is unchanged before and after color change, so that the display mode of the image related to the content item is greatly enriched and the display effect of the image related to the content item is improved under the condition that the color change accuracy and the color change effect are ensured.

In the above embodiment, a process flow of automatically generating a color-changed target image based on an original image is simply described, but in the embodiment of the present application, the above process flow of intelligent color change will be described in detail, and fig. 3 is a flowchart of an image processing method provided in the embodiment of the present application. Referring to fig. 3, the embodiment is performed by an electronic device, and the electronic device is taken as a terminal for explanation, and the embodiment includes:

301. The terminal obtains at least one original image associated with the content item.

Step 301 is similar to step 201, and will not be described here.

302. The terminal identifies a first area in the at least one original image, wherein the first area refers to an area without color change in the original image.

Optionally, the first region comprises at least one of: in other words, since the areas which cannot be changed such as the commodity display image and the model trial image can be detected as targets, the first areas which do not need to be changed can be selected from the candidate areas; or, in other words, the target object (such as a commodity, an article, a model, a speaker, etc. associated with the content item) in the original image does not need to be changed in color, so that the target object can be segmented from the original image, and the segmented target object is used as a first area without changing color.

Next, the manner of identifying the first region will be described with respect to the case where two first regions, i.e., the candidate region requiring no color change and the region where the target object is located, are detected, respectively.

Case one, candidate region without color change

In some embodiments, in the case that the first area includes candidate areas without color change, for any original image in the at least one original image, the terminal performs object detection on the original image to obtain candidate areas in the original image, in other words, one or more candidate areas can be determined from the original image through object detection, and some of the candidate areas are capable of color change, some of the candidate areas are not required to be color-changed, and candidate areas without color change need to be identified as the first area through a subsequent color change classification model.

In some embodiments, since the candidate area that cannot be color-changed, such as the general merchandise display diagram, the model trial diagram, and the like, is made into a rectangular area in the UI interface associated with the content item, such as the landing page or the display page, the above-mentioned target detection process may be a process of detecting the rectangular area by the pointer, in other words, the terminal performs rectangular detection on the original image, and determines the detected rectangular area as the candidate area.

In some embodiments, the terminal performs rectangle detection by: 1) Carrying out Gaussian blur on the original image to obtain a blurred image; 2) Binarizing the blurred image to obtain a binary image; 3) Detecting a contour contained in the binary image, and performing polygon approximation on the detected contour; 4) And determining that the rectangular area is detected when the approximated polygon is a convex quadrilateral and the included angle between adjacent sides meets the target condition.

Optionally, when acquiring the blurred image, besides the gaussian blur mode, the mean blur, median blur, bilateral blur and other modes may be used to smooth the original image to obtain the blurred image, and the blur mode is not specifically limited in the embodiment of the present application.

Optionally, when acquiring the binary image, an oxford method (OTSU) algorithm is used to perform binary segmentation on the blurred image, the OTSU algorithm is also called a maximum inter-class variance method, and after performing binary segmentation on the image according to the threshold obtained by the OTSU algorithm, the inter-class variance of the foreground and background images is the maximum. The OTSU algorithm is considered as an optimal algorithm for selecting the image binary segmentation threshold, is simple to calculate and is not influenced by the brightness and contrast of the image, so that the OTSU algorithm is widely applied to digital image processing. The OTSU algorithm divides an image into a background part and a foreground part according to the gray scale characteristic of the image. Since variance is a measure of the uniformity of the gray level distribution, the larger the inter-class variance between the background and the foreground, the larger the difference between the two parts constituting the image, and the smaller the difference between the two parts when the foreground is divided into the background or the background is divided into the foreground. Thus, a segmentation that maximizes the inter-class variance means that the probability of misclassification is minimal.

Illustratively, a segmentation threshold is calculated on the blurred image by the OTSU algorithm, then, for each pixel in the blurred image, the pixel value of the pixel is set to 1 if the pixel value of the pixel is greater than the segmentation threshold, and the pixel value of the pixel is set to 0 if the pixel value of the pixel is less than the segmentation threshold, so that a binary image can be obtained, and the pixel value of each pixel in the binary image is either 1 or 0.

Optionally, the contour detection is performed on the binary image to obtain a contour feature map of the binary image, and the polygon approximation is performed on the contour feature map, in other words, the contours in the contour feature map are approximated by using the polygons, so that each polygon obtained by approximation can be determined.

Optionally, for any approximated polygon, if the polygon is a quadrilateral and is a convex polygon, and the angles between adjacent sides meet the target condition, it is determined that a rectangular area resulting from the convex quadrilateral approximation is detected.

Optionally, the target condition means that an angle difference between the adjacent edges and 90 degrees is smaller than an angle threshold, where the angle threshold is any value larger than 0, for example, the angle threshold is 5 degrees, 10 degrees or other values, or the target condition means that an absolute value of a cosine difference between a cosine value of the angle between the adjacent edges and 0 (a cosine value of 90 degrees is 0) is smaller than a cosine threshold, where the cosine threshold is any value larger than or equal to 0 and smaller than or equal to 1, for example, the cosine threshold is 0.1, 0.2 or other values, which is not limited in detail in the embodiment of the present application.

In some embodiments, whether each polygon obtained by approximation is a convex quadrilateral and whether the included angle between adjacent sides meets the target condition is determined, and after the polygon obtained by approximation is traversed, each rectangular region contained in the original image can be obtained.

It should be noted that, steps 1) to 4) of the rectangle detection can be implemented by using a function or code logic encapsulated in the Open CV code library.

In other embodiments, the terminal performs rectangle detection by: inputting the original image into a rectangular detection model, extracting edge characteristics of the original image through a plurality of hidden layers included in the rectangular detection model, determining whether the original image contains a rectangular area or not through the edge characteristics, and outputting the detected rectangular area if the original image contains the rectangular area, wherein the detected rectangular area is schematically marked in the original image, or buckled out of the original image, and the buckled rectangular area is output, or only the position coordinates of the detected rectangular area are output. Alternatively, the rectangular detection model is any image processing model based on deep learning, the rectangular detection model is obtained by training based on a positive sample image containing a rectangular region and a negative sample image not containing a rectangular region, the rectangular detection model is used for detecting whether the rectangular region is contained in the input image, and the detected rectangular region is also output in the case that the input image contains the rectangular region. Optionally, the model structure of the rectangular detection model includes, but is not limited to: DNN (Deep Neural Networks, deep neural network) model, CNN (Convolutional Neural Networks, convolutional neural network) model, MLP (Muti-Layer preference) model, and the like, the model structure of the rectangular detection model is not particularly limited in the embodiment of the application.

Illustratively, for each rectangular region detected, the upper left corner vertex coordinates (x, y) of the rectangular region and the width w and height h of the rectangular region are used to characterize, in other words, for each rectangular region, the position coordinates of this rectangular region in the original image are characterized using a four-tuple of { x, y, w, h }.

Alternatively, the representation may be performed using the lower right corner vertex coordinates of the rectangular area, the width and the height of the rectangular area, or the representation may be performed using the lower left corner vertex coordinates of the rectangular area, the width and the height of the rectangular area, or the representation may be performed using the upper right corner vertex coordinates of the rectangular area, the width and the height of the rectangular area, and the embodiment of the present application does not specifically limit the representation manner of the position coordinates of the rectangular area.

In some embodiments, for detecting any candidate region, inputting the candidate region into a color-changing classification model, and outputting a color-changing detection result through the color-changing classification model, where the color-changing detection result is used to characterize whether the input candidate region needs to change colors, and the color-changing classification model is used to detect whether the input candidate region needs to change colors, where the color-changing classification model is obtained based on a pre-labeled first region without color change and a pre-labeled second region with color change.

Alternatively, the color change classification model is a classification model, a candidate region is input into the color change classification model, an image feature of the candidate region is extracted through the color change classification model, the image feature is used to predict a matching probability P that the candidate region is a first region without color change, when the matching probability P is greater than or equal to a matching threshold P ', an output color change detection result indicates that the candidate region does not need color change (such as outputting a classification label "no color change"), and when the matching probability P is less than the matching threshold P', an output color change detection result indicates that the candidate region needs color change (such as outputting a classification label "color change").

Optionally, the model structure of the color-changing classification model includes, but is not limited to: resNet101 (residual network 101) model, resNet model, resNet model, logistic regression (Logistic Regression) model, decision tree (Decision tree) model, support vector machine (Support Vector Machine, SVM) model, naive Bayes (Naive Bayes) model, and the like.

Illustratively, when any one of the detected candidate regions is a rectangular region, the color change classification model is provided as a rectangular classifier, the rectangular classifier is used for detecting whether the input rectangular region needs to be color changed, for example, the rectangular classifier is used for performing two-class detection of 'color change', 'no color change', for the input rectangular region, after image feature extraction and matching probability prediction of the rectangular classifier, a classification label predicted for the rectangular region is output, for example, when the classification label 'color change' is output, the color change detection result of the rectangular region is determined to be that color change is required, and when the classification label 'no color change' is output, the color change detection result of the rectangular region is determined to be that color change is not required.

In some embodiments, based on the color change detection result of each candidate region by the color change classification model, the terminal determines a candidate region that does not need color change as the first region, for example, determines a rectangular region that does not need color change as the first region, where the candidate region can be a circular region, an irregular region, or the like besides the rectangular region, which is not specifically limited in the shape of the candidate region according to the embodiments of the present application.

Case two, region where target object is located

In some embodiments, in the case that the first area includes an area where the target object is located, for any original image in the at least one original image, the terminal performs object recognition on the original image to obtain an object recognition result, where the object recognition result characterizes whether the original image includes the target object, that is, the object recognition is only a process of determining whether the original image includes the target object, and not a process of finely identifying the area where the target object is located.

In some embodiments, the target objects include, but are not limited to: the kind of the target object is not particularly limited in the embodiment of the present application, such as a face, a designated article, a designated person, a designated building, a designated animal, a designated plant, a designated avatar, and the like.

In some embodiments, the terminal inputs the original image into an object recognition model, extracts deep features associated with the object types in the original image through the object recognition model, analyzes whether the original image contains a target object through the deep features, and obtains an object recognition result. Illustratively, when the target object includes a face, the terminal inputs the original image into a face recognition model, performs face recognition on the original image through the face recognition model to output a face recognition result, and the face recognition result characterizes whether the original image includes a face of any object; when the target object comprises a cat, the terminal inputs the original image into an object recognition model, and performs object recognition on the original image through the object recognition model so as to output an object recognition result, wherein the object recognition result represents whether the original image comprises the cat or not.

The above object recognition process can also be regarded as an image classification process, i.e. the original image is subjected to a "target object included"/"target object not included" classification detection by the object recognition model, where the object recognition model is trained as a classification model for detecting whether the target object is included in the input image, and typically, different object recognition models are trained for different target objects, and a RETINAFACE (retinal face) model is used for face recognition, schematically, for the target object "face".

In some embodiments, in the case that the object recognition result indicates that the original image contains the target object, the terminal performs object segmentation on the original image to obtain a mask image for identifying the target object. Optionally, the mask image is a binary image, in which a pixel with a pixel value of 1 (referred to as a foreground pixel) represents an area where the target object is located, and a pixel with a pixel value of 0 (referred to as a background pixel) represents an area where the target object is not located. Optionally, the mask image is a color image, and is divided into a plurality of connected areas in the color image, where each connected area represents a divided object, and the connected areas where the same object is located are filled with the same color, for example, when green is used to represent the target object, an area formed by green pixels in the mask image represents an area where the target object is located.

In some embodiments, the terminal inputs the original image into an object segmentation model, performs segmentation processing on the original image through the object segmentation model, and outputs a mask image for identifying the target object. Optionally, the model architecture of the object segmentation model includes, but is not limited to: MODNet model, watershed algorithm, full convolution (Fully Convolutional Networks, FCN) model, etc., the model architecture of the object segmentation model is not particularly limited in the embodiments of the present application.

In some embodiments, the area where the target object is located can be found by the above mask image, so that the first area where the target object is located can be determined based on the mask image. Illustratively, for each target object, one mask image is used to characterize the region in which the target object is located.

In the above process, by determining the area where the target object is located as the first area when the object identification result indicates that the original image contains the target object, it is ensured that the area where the target object is located can be accurately identified from the original image after the specific target object is specified, and the area where the target object is located is ensured not to be color-changed, and the method can be flexibly applicable to service requirements for ensuring that different objects are not color-changed, for example, when the object is set as a cup of a commodity to which the content item is directed, the area where each cup is located can be identified from the original image as the first area, and the cups are ensured not to be color-changed in the processing process.

303. And the terminal assigns the pixel points in the first area to target pixel values in the at least one original image to obtain at least one first image.

In some embodiments, since none of the pixels in the first area need to participate in the color-changing algorithm, the terminal may assign the pixels in the first area to the target pixel value in each original image, so as to obtain at least one first image.

By assigning the pixel in the first area as the target pixel value, the interference of the dominant hue identification in the subsequent step 306 can be reduced, in other words, if the pixel in the first area is not assigned as the target pixel value, the color vector of the pixel in the first area is caused to participate in the clustering process based on the color vector of the pixel, so that the color vector of the pixel in the first area which does not participate in color conversion affects the identification of the dominant hue in the second area which needs to participate in color conversion, therefore, by assigning the pixel in the first area as the target pixel value, the pixel which does not participate in color conversion can be guaranteed to have the same pixel value (target pixel value), and thus, all the pixel have the same color vector, and the color vector is clustered into the same color cluster, and the interference of the pixel in the first area on the dominant hue identification can be greatly reduced by excluding the color cluster when the dominant hue is screened.

In one example, each pixel point in the first area is assigned a white pixel value (255, 255, 255), or the target pixel value may also be a black pixel value (0, 0), or other colors, and the embodiment of the present application does not specifically limit the target pixel value.

In some embodiments, when the first area includes a rectangular area where color change is not required, all pixel points in the rectangular area are assigned as target pixel values, when the first area includes an area where the target object is located, position coordinates of the pixel points in the area where the target object is located can be determined according to the mask image, so that the pixel points in the corresponding positions are found in the original image based on the position coordinates, and the pixel points in the corresponding positions are assigned as target pixel values, for example, when the mask image is a binary image and the pixel points with the value of 1 represent the area where the target object is located, position coordinates of all the pixel points with the value of 1 are found, and the pixel points in the corresponding positions are found in the original image based on the position coordinates, and are assigned as target pixel values.

In some embodiments, the terminal may not execute the step 303, that is, does not assign a value to a pixel in the first area of the original image, and only needs to exclude the pixel in the first area when screening the dominant hue.

304. And the terminal splices the at least one first image to obtain a spliced image.

In some embodiments, the terminal performs a transverse or longitudinal stitching on the at least one first image, or performs a multi-grid stitching to obtain a stitched image, where the embodiment of the application does not specifically limit a typesetting manner during stitching, for example, a longitudinal stitching manner is used to stitch the at least one first image to obtain the stitched image.

In some embodiments, since the first images may have inconsistent sizes, the first images may need to be adjusted to the same width before being longitudinally stitched to obtain the stitched image, or the first images may need to be adjusted to the same height before being transversely stitched to obtain the stitched image, or the first images may need to be adjusted to the same width and height before being stitched to obtain the stitched image in a multi-grid (e.g. 4-grid and 9-grid) manner, that is, before being stitched, the first images may be adjusted in size to meet the stitching requirement and then be stitched to obtain the stitched image.

305. And the terminal clusters the pixel points in the spliced image based on the color vectors of the pixel points to obtain at least one color cluster.

In some embodiments, the terminal forms a data set for a plurality of pixel points in the stitched image, and clusters the color vectors of the plurality of pixel points in the data set by using a clustering algorithm, that is, the color vectors with similar color vectors, that is, close distance, are guaranteed to be classified into the same color class cluster, and at least one color class cluster formed by clustering the color vectors of the plurality of pixel points in the stitched image can be obtained when the clustering is stopped.

In some embodiments, any pixel in the dataset uses the (R, G, B) three-dimensional vector to characterize the color vector of that pixel. In the clustering process, a K-Means (K-Means) clustering algorithm, a K-Nearest Neighbor (KNN) clustering algorithm or other clustering models based on deep learning can be used, and the embodiment of the application does not specifically limit the used clustering algorithm.

Illustratively, a K-Means clustering algorithm is taken as an example for explanation, and on a data set formed by color vectors of a plurality of pixel points in a spliced image, the color vectors of K (K is more than 1) pixel points are randomly selected as initial K clustering centers; then, the distance between the color vector of the pixel and the color vector of the K cluster centers (such as the euclidean distance) is calculated for each pixel in the data set, the color vector of the pixel is allocated to the cluster center closest to the K cluster centers, and the color vector allocated to the same cluster center represents one color cluster. After each color vector in the dataset is allocated, for any color cluster, an average vector needs to be recalculated as a new cluster center according to each color vector belonging to the color cluster. And continuously executing the operations of grouping the color vectors and updating the clustering center until the clustering termination condition is met, and stopping clustering, wherein K groups of clusters when the clustering is stopped are candidate K color clusters. Optionally, the termination cluster condition includes, but is not limited to: no (or a minimum number of) color vectors are reassigned to different color class clusters; or, no (or a minimum number of) cluster centers of the color class clusters are changed again; or, the error square and the local minimum, etc., and the termination clustering condition may be any one or a combination of at least two of the foregoing, and the embodiment of the present application does not specifically limit the termination clustering condition.

In some embodiments, when the above K-Means clustering algorithm stops clustering, candidate K color class clusters are output, optionally, K color class clusters are all put into the following step 306 to obtain a dominant hue, optionally, not all the K color class clusters need to participate in the following step 306 to obtain a dominant hue, for example, since all the pixel points in the first area are assigned with the target pixel value, then the color class cluster in which the target pixel value is located needs to be excluded from the candidate K color class clusters, for example, when the target pixel value is the white pixel value (255, 255, 255), the color class cluster in which the white pixel value is deleted from the K color class clusters, for example, from the K-1 color class clusters in which the target pixel value is excluded, the K-1 color class clusters are sorted according to the number of the containing color vectors, for example, the I color class clusters in the sorting order are put into the following step 306 to obtain the dominant hue, for example, the I color class clusters are greater than or equal to the number of the dominant hue, in other words, the number of the K-1 color class clusters is less than the dominant hue, and the number of the dominant hue clusters is less than the dominant hue is the dominant hue, and the number of the candidate clusters is less than the dominant hue is filtered, and the number of the dominant hue is less than the dominant hue clusters is filtered, and the number of the dominant hue clusters is less than the dominant hue clusters is found by the number of the candidate cluster.

Optionally, only after the candidate K color clusters are excluded from the color clusters where the target pixel value is located, the remaining K-1 color clusters are put into the following step 306, or the candidate K color clusters are directly sorted from large to small according to the number of the pixel points, and M color clusters with the sorting position of top M are selected and put into the following step 306, where M is an integer greater than or equal to 1 and less than or equal to K.

Alternatively, if the terminal does not perform the step of filling the pixel points in the first area with the target pixel values in step 303, the terminal may delete the pixel points in the first area when the data set formed by the plurality of pixel points of the stitched image is initially constructed, so as to ensure that the color vectors of the pixel points in the first area do not participate in the color clustering process.

306. And the terminal determines the clustering center of the at least one color cluster as at least one dominant hue of the spliced image, wherein the dominant hue represents the color distribution tendency of the pixel points in the spliced image.

Wherein, the dominant hue characterizes the color distribution trend of the pixels in the stitched image, and the stitched image is stitched by at least one first image, in other words, the dominant hue characterizes the color distribution trend of the pixels in the at least one first image.

In some embodiments, the average vector of the color vectors of the pixels included in each color cluster is used as the cluster center of the color cluster by the terminal, and represents a dominant hue of the spliced image, and the above operation is performed on at least one color cluster to obtain at least one dominant hue.

In the foregoing steps 304-306, a possible implementation manner of obtaining at least one dominant hue by the terminal based on the at least one first image identification is provided, optionally, the terminal does not perform the foregoing step 304, without stitching each first image, where a data set for clustering is formed by using color vectors of a plurality of pixel points in a second area of each first image, and at least one dominant hue can be identified, and in this embodiment of the present application, whether to perform dominant hue identification after stitching is not specifically limited.

307. And the terminal performs color mapping on the pixel points in the spliced image to obtain a color-changing image, and cuts the color-changing image to obtain at least one second image corresponding to each of the at least one first image.

Since the stitched image is stitched by at least one first image and at least one second image is cut by the color-changing image, the above step 307 corresponds to performing color mapping on the pixel points in the at least one first image by the terminal, so as to obtain at least one second image. Optionally, if the first images are not stitched during the dominant hue recognition, there is no stitched image, and then the corresponding second images can be obtained by directly mapping the first images one by one, without cutting the color-changing image obtained by mapping the stitched image to obtain each second image, alternatively, even if the first images are stitched during the dominant hue recognition to obtain the stitched image, after the dominant hue recognition is finished, the corresponding second images can be obtained by directly mapping the first images one by one, that is, the stitched image is only used for recognizing the dominant hue and does not participate in the color mapping process, at this time, no additional cutting process is needed, and the image processing flow can be simplified.

In some embodiments, the terminal performs the color mapping by the following steps 3071-3075:

3071. the terminal converts the stitched image from the red-green-blue RGB color space to the hue-saturation-brightness HSV color space.

In some embodiments, since the RGB color space and HSV color space are characterized by essentially using different color models for the same color, the pixel value of any pixel in the stitched image can be converted from the RGB color space to the HSV color space.

Since the stitched image is stitched from at least one first image, the above step 3071 corresponds to converting the at least one first image from the RGB color space to the HSV color space, alternatively, if the stitched image is not acquired in the main color recognition stage, the at least one first image may be directly converted from the RGB color space to the HSV color space in step 3071, which is not specifically limited in the embodiment of the present application.

3072. And the terminal modifies the tone H of any main tone in the HSV color space to obtain a color-changing tone corresponding to the main tone, wherein the saturation S and the brightness V of the main tone are kept unchanged.

In some embodiments, for each dominant hue, in the HSV color space, since the color change objective is to change only hue H without modifying saturation S and brightness V, saturation S and brightness V of the dominant hue may be kept unchanged, and hue H of the dominant hue is modified to obtain a color change hue corresponding to the dominant hue.

In some embodiments, the tone sampling function in the HSV color space is configured based on the number N of color schemes to be generated by the user for the present intelligent color matching, so that for any i (1 i N) th color scheme, the corresponding color-changing tone can be uniformly sampled according to the number N of color schemes in the value range [0,360] of the tone H by substituting the value of i into the tone sampling function.

In the above process, the tone H is modified by using the same tone sampling function for each dominant tone to obtain the corresponding color-changing tone, so that the tone difference between the different dominant tones before and after modification can be kept unchanged, in other words, the tone difference between any two dominant tones is equal to the tone difference between the corresponding two color-changing tones, so that the color with too close tone can be prevented from occurring in the final target image, and the defects of poor image viewing due to too small tone difference after color changing can be avoided.

In some embodiments, the expression of the tone sampling function is as follows:

Hn＝(Ho+i·s)％360

Where H _o represents the dominant hue, H _n represents the color-changing hue corresponding to the dominant hue H _o, i represents the number of the ith color scheme (also represents the ith color mapping to the original image), N represents the total number of color schemes, s is a parameter calculated based on N, and% represents the operation using (H2t _o +i.s) to remainder 360.

3073. The terminal determines a color mapping relationship of pixel levels based on a main tone mapping relationship of a main tone and a color-changing tone, wherein the color mapping relationship characterizes the corresponding relationship of different pixel values before and after color changing.

In some embodiments, since a corresponding color conversion tone is found in the HSV color space for each dominant tone, there is a dominant tone mapping relationship between each dominant tone and each corresponding color conversion tone, and a pixel-level color mapping relationship can be constructed based on the dominant tone mapping relationship, in other words, when the dominant tone is 5, a respective color mapping relationship for each color throughout the entire color space can be constructed according to the dominant tone mapping relationship between the 5 dominant tones and the 5 color conversion tones.

In some embodiments, the stitched image is converted from the HSV color space to the LAB color space again, since the LAB color space is larger than the RGB color space, which represents that any point (any color) in the RGB color space can find a corresponding mapping point in the LAB color space, so when the color mapping relationship is constructed in the LAB color space by bilinear interpolation or trilinear interpolation based on the dominant tone mapping relationship, the color-converted image obtained by mapping the pixel values of each pixel point in the LAB color space can be converted from the LAB color space back to the RGB color space based on the higher color-conversion accuracy and better color-conversion effect by the following step 3074. Illustratively, a Palette-based Photo Recoloring (Palette-based image color changing) algorithm is adopted to construct the color mapping relationship based on the dominant tone mapping relationship, and the manner of constructing the color mapping relationship is not particularly limited in the embodiment of the present application.

In some embodiments, the color mapping relationship may also be directly constructed based on the dominant tone mapping relationship in the RGB color space, or constructed based on the dominant tone mapping relationship in the HSV color space, and the embodiment of the present application does not specifically define whether to convert to the LAB color space to implement color mapping.

3074. And the terminal maps the pixel values of the pixel points in the spliced image based on the color mapping relation to obtain a color-changing image.

In some embodiments, since the color mapping relationship represents the correspondence between different pixel values before and after color conversion, for any pixel point in the stitched image, the color-converted pixel value corresponding to the pixel value of the pixel point can be found through the color mapping relationship, the color-converted pixel value is assigned to the pixel point in the corresponding position in the color-converted image, and the above operation is repeatedly performed, so that a color-converted image in which each pixel point is color-mapped can be obtained.

3075. And the terminal cuts the color-changing image to obtain at least one second image corresponding to each of the at least one first image.

In some embodiments, since the color change only changes the pixel value of the pixel point, but does not change the size and position of the first image, it can be known that the first image before the color change and the second image after the color change have the same size and position, and the terminal cuts the color change image based on the size and position of each first image, so that the second image corresponding to each first image can be obtained.

Since the stitched image is stitched by at least one first image and at least one second image is cut by the color-changing image, the above steps 3074-3075 correspond to mapping the pixel points in the at least one first image based on the color mapping relationship by the terminal, so as to obtain at least one second image.

308. And the terminal assigns the pixel points positioned in the first area to the pixel values of the pixel points at the corresponding position in the corresponding original image in the at least one second image to obtain at least one target image associated with the content item, wherein the target image is the original image after the color of the first area is kept unchanged and the color of the second area is changed.

In some embodiments, each second image has a first image which is uniquely corresponding to the first image and an original image which is uniquely corresponding to the first image, that is, the original image, the first image and the second image have a one-to-one correspondence relationship, for each second image, the terminal assigns any pixel point in the first area in the second image to be a pixel value corresponding to a pixel point in the corresponding position in the original image, and the above operation is repeated for a plurality of times, so that a target image associated with the content item is obtained, so that only color change is performed on the second area in the target image, and the color of the first area is ensured to be unchanged.

In some embodiments, in the case that the first area includes a rectangular area without color change, the terminal can find the pixel value and the position coordinate of the pixel point included in the rectangular area without color change in the original image based on the quadruple { x, y, w, h } of each rectangular area, and for any pixel point P included in the rectangular area without color change, based on the position coordinate of the pixel point P, the pixel point P ' with the same position coordinate can be found in the second image, and then assign the pixel value of the pixel point P in the original image to the pixel point P ' with the same position in the second image, so that it can be ensured that any pixel point P ' in the rectangular area without color change will not be modified in the target image.

In some embodiments, in the case that the first area includes an area where the target object is located, the terminal can find, based on the mask image, a pixel value and a position coordinate of a pixel point included in the area where the target object is located, for any pixel point P included in the area where the target object is located, based on the position coordinate of the pixel point P, a pixel point P ' with the same position coordinate can be found in the second image, and then assign the pixel value of the pixel point P in the original image to the pixel point P ' with the same position in the second image, so that it can be ensured that any pixel point P ' in the area where the target object is located is not modified in the target image.

In the foregoing steps 303 to 308, a possible implementation manner of performing color mapping on the pixels in the second area except for the first area in the at least one original image to obtain the at least one target image associated with the content item is provided, that is, after the first area is identified, color conversion is performed on the whole image, and then the pixel values of the pixels in the first area are assigned from the original image to the target image, alternatively, color conversion may be performed only for the second area during color conversion, so that the operation of the assignment is not required, and the image processing flow can be simplified.

In some embodiments, since there are often white areas in each original image associated with a content item, these white areas often act as a background for a landing page or a presentation page, there may be situations in some business needs where it is desirable that the white areas do not change colors, to ensure that the background color is always a more pleasant, easily viewable white. For the above-mentioned business requirement, for determining a white area in the at least one original image, the difference between the pixel value of the pixel point in the white area and the white value is smaller than a target threshold, the target threshold is any value greater than or equal to 0, for example, for each original image, traversing the pixel point in the original image, if the pixel value of any pixel point is between (240, 240, 240) and (255, 255, 255), that is, the difference between the 3 channels of RGB is smaller than the target threshold 15, determining the current pixel point as belonging to the white area, and repeatedly executing the above-mentioned operations, so as to determine all the pixel points belonging to the white area in the original image; then, based on the manner similar to the above step 308, after the white area is identified for each original image, in the second image corresponding to the original image, the pixel point located in the white area is assigned to the pixel value corresponding to the pixel point at the corresponding position in the original image, so that the white area is ensured not to change color, and different service requirements can be flexibly adapted. It should be noted that, when there are other various requirements such as that the red area will not change color, the blue area will not change color, etc., similar operations are performed to ensure that any specified color will not change color in the color changing process, so that the color changing device has extremely high controllability.

In the above embodiment, detailed implementation of each step of the image processing method is described in detail, in the embodiment of the present application, taking the case that 2 original images (original images 1 and 2) are input by a user as an example, how to intelligently color the 2 original images, and derive the corresponding 2 target images.

Fig. 4 is a schematic flow chart of an image processing method provided in an embodiment of the present application, as shown in fig. 4, an original image 1 and an original image 2 are respectively input into a preprocessing module 401, and the preprocessed image 1 and the preprocessed image 2 are respectively output, where the preprocessing module 401 is configured to identify a first area without color change in the original image, and fill the first area into white, for example, the first area includes a rectangular area without color change and a target object, and at this time, for the rectangular area without color change, the preprocessing module 401 is further required to output a quadruple { x, y, w, h } for representing a position coordinate of the rectangular area without color change, where (x, y) represents an upper left corner vertex coordinate of the rectangular area without color change, and w and h represent a width and a height of the rectangular area without color change, respectively; in addition, for the target object, the preprocessing module 401 needs to output a Mask image Mask for identifying the area where the target object is located.

Then, the preprocessed image 1 and the preprocessed image 2 output by the preprocessing module 401 are stitched to obtain a stitched image, the stitched image is input into the dominant hue recognition module 402, and the dominant hue recognition module 402 is configured to recognize at least one dominant hue, for example, 5 dominant hues, in the stitched image by using a clustering manner based on color vectors.

Then, at least one dominant hue identified by the dominant hue identification module 402 is input to the color scheme generation module 403, and the color scheme generation module 403 is configured to change each dominant hue to obtain a color-changing hue.

Then, the stitched image, at least one dominant hue identified by the dominant hue identification module 402, and at least one color conversion hue generated by the color scheme generation module 403 are all converted into an LAB color space, and then input into the color conversion algorithm module 404, where each pixel point in the stitched image is subjected to color mapping in the LAB color space, and a color conversion image is output.

Then, the color-changed image is cut to obtain a color-changed image 1 and a color-changed image 2, the original image 1 and the color-changed image 1 are input into a post-processing module 405, a white area, a non-color-changed rectangular area and a target object are pasted into the color-changed image 1 from the original image 1 through the post-processing module 405, the target image 1 is obtained, the original image 2 and the color-changed image 2 are input into the post-processing module 405, and the white area, the non-color-changed rectangular area and the target object are pasted into the color-changed image 2 from the original image 2 through the post-processing module 405, so that the target image 2 is obtained.

Next, the preprocessing module 401 in fig. 4 will be described in an expanded manner.

Fig. 5 is a schematic flow chart of a preprocessing module provided in an embodiment of the present application, as shown in fig. 5, in the preprocessing module 401, for any input original image, a first area without color change is identified through two paths of detection, in a bypass, all candidate rectangular areas in the original image are detected through the rectangular detection module 4011, each candidate rectangular area detected by the rectangular detection module 4011 is sent into the rectangular classifier 4012 to perform "color change"/"color not change" classification identification, and for the rectangular area with "color not change" as a color change detection result, four-tuple { x, y, w, h } parameters of the rectangular area without color change are output; in another bypass, whether the original image contains the target object is identified by the object identifying module 4013, so as to obtain an object identifying result, and when the object identifying result indicates that the original image contains the target object, the original image is input to the object dividing module 4014 to divide the target object in the original image, and a Mask image Mask for identifying the target object is output, then, the non-color-changing rectangular area and the target object each belong to a non-color-changing area (i.e. the first area), the first area is filled with white, and then the preprocessing module 401 outputs a preprocessed image in which the first area is filled with white, and quadruples { x, y, w, h } of each non-color-changing rectangular area, and the Mask image Mask of each target object.

Next, the post-processing module 405 in fig. 4 will be described in detail.

Fig. 6 is a schematic flow chart of a post-processing module according to an embodiment of the present application, as shown in fig. 6, in the post-processing module 405, for any input original image and a corresponding color-changed image, a quadruple { x, y, w, h } of each color-unchanged rectangular area in the original image and a Mask image Mask of each target object need to be obtained, where parameters of the quadruple and Mask may be provided by the pre-processing module 401. Then, for the original image, the corresponding color-unchanged rectangular areas are obtained from the original image by using the quadruples { x, y, w, h } of the color-unchanged rectangular areas, and meanwhile, the corresponding target object is obtained from the original image by using the Mask image Mask of each target object.

The image processing method provided by the embodiment of the application can be suitable for the manufacturing process of various UI interfaces related to the content item, such as the manufacturing process of the advertisement landing page, and by utilizing the image processing method, N color schemes can be intelligently, automatically and quickly derived for an initial scheme containing at least one original image given by the landing page, and the N color schemes can be selected by a user and can be used in formal delivery, so that the user can select a proper color scheme according to service requirements, and release and apply the proper color scheme to the corresponding advertisement landing page, enrich the display mode and display effect of the advertisement landing page, save the time cost and manufacturing cost of the advertisement landing page, and improve the man-machine interaction efficiency of the manufacturing process.

Fig. 7 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application, as shown in fig. 7, the apparatus includes:

An acquisition module 701 for acquiring at least one original image associated with a content item;

the identifying module 702 is configured to identify a first area in the at least one original image, where the first area refers to an area in the original image that does not need color change;

the mapping module 703 is configured to perform color mapping on pixels of a second area, except for the first area, in the at least one original image, so as to obtain at least one target image associated with the content item, where the target image is an original image obtained by keeping the color of the first area unchanged and changing the color of the second area.

The device provided by the embodiment of the application keeps the color of the first area unchanged in the color changing process by aiming at the original image related to the content item, and only performs color mapping on the second area, so that the target image obtained by color changing is another color matching scheme derived on the basis of the original image, the color matching scheme does not need manual operation of a user for picture repairing application, the first area which cannot be changed in color can be intelligently identified, and the color of the first area is unchanged before and after color change, so that the display mode of the image related to the content item is greatly enriched and the display effect of the image related to the content item is improved under the condition that the color change accuracy and the color change effect are ensured.

In one possible implementation, based on the apparatus composition of fig. 7, the identification module 702 includes:

A first determining unit configured to determine, as the first region, a candidate region for which the color change detection result indicates that color change is not necessary.

In one possible embodiment, the detection unit is configured to:

the first recognition unit is used for carrying out object recognition on any original image to obtain an object recognition result, and the object recognition result represents whether the original image contains a target object or not;

And a second determining unit configured to determine, as the first area, an area in which the target object is located, in a case where the object recognition result indicates that the original image contains the target object.

In one possible embodiment, the second determining unit is configured to:

In one possible implementation, based on the apparatus composition of fig. 7, the mapping module 703 includes:

A second identifying unit, configured to identify at least one dominant hue based on the at least one first image, where the dominant hue represents a color distribution tendency of pixels in the at least one first image;

In one possible embodiment, the second identifying unit is configured to:

splicing the at least one first image to obtain a spliced image;

The cluster center of the at least one color class cluster is determined as the at least one dominant hue.

In a possible implementation, the mapping unit is configured to:

For any main tone, keeping the saturation S and the brightness V of the main tone unchanged in the HSV color space, and modifying the tone H of the main tone to obtain a color-changing tone corresponding to the main tone;

and mapping pixel values of the pixel points in the at least one first image based on the color mapping relation to obtain the at least one second image.

In one possible embodiment, the device based on fig. 7 is composed, the device further comprising:

A determining module, configured to determine a white area in the at least one original image, where a difference between a pixel value of a pixel point in the white area and the white value is less than a target threshold;

It should be noted that: the image processing apparatus provided in the above embodiment is only exemplified by the division of the above functional modules when processing an image, and in practical application, the above functional allocation can be performed by different functional modules according to needs, that is, the internal structure of the electronic device is divided into different functional modules to perform all or part of the functions described above. In addition, the image processing apparatus and the image processing method provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the image processing apparatus and the image processing method are detailed in the image processing method embodiment, which is not described herein again.

Fig. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application, and as shown in fig. 8, a terminal 800 is an exemplary illustration of an electronic device. Optionally, the device types of the terminal 800 include: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion picture expert compression standard audio plane 3), an MP4 (Moving Picture Experts Group Audio Layer IV, motion picture expert compression standard audio plane 4) player, a notebook computer, or a desktop computer. Terminal 800 may also be referred to by other names of user devices, portable terminals, laptop terminals, desktop terminals, and the like.

In general, the terminal 800 includes: a processor 801 and a memory 802.

Optionally, the processor 801 includes one or more processing cores, such as a 4-core processor, an 8-core processor, or the like. Optionally, the processor 801 is implemented in at least one hardware form of DSP (DIGITAL SIGNAL Processing), FPGA (Field-Programmable gate array), PLA (Programmable Logic Array ). In some embodiments, the processor 801 includes a main processor, which is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 801 is integrated with a GPU (Graphics Processing Unit, image processor) that is responsible for rendering and rendering of the content that the display screen is required to display. In some embodiments, the processor 801 further includes an AI (ARTIFICIAL INTELLIGENCE ) processor for processing computing operations related to machine learning.

In some embodiments, memory 802 includes one or more computer-readable storage media, which are optionally non-transitory. Optionally, the memory 802 also includes high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 802 is used to store at least one program code for execution by processor 801 to implement the image processing methods provided by the various embodiments of the present application.

In some embodiments, the terminal 800 may further optionally include: a peripheral interface 803, and at least one peripheral. The processor 801, the memory 802, and the peripheral interface 803 can be connected by a bus or signal lines. Individual peripheral devices can be connected to the peripheral device interface 803 via buses, signal lines, or a circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 804, a display 805, a camera assembly 806, audio circuitry 807, and a power supply 808.

Peripheral interface 803 may be used to connect at least one Input/Output (I/O) related peripheral to processor 801 and memory 802. In some embodiments, processor 801, memory 802, and peripheral interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or both of the processor 801, the memory 802, and the peripheral interface 803 are implemented on separate chips or circuit boards, which is not limited in this embodiment.

The Radio Frequency circuit 804 is configured to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 804 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 804 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. Optionally, the radio frequency circuit 804 communicates with other terminals via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (WIRELESS FIDELITY ) networks. In some embodiments, the radio frequency circuit 804 further includes NFC (NEAR FIELD Communication) related circuits, which are not limited by the present application.

The display 805 is used to display a UI (User Interface). Optionally, the UI includes graphics, text, icons, video, and any combination thereof. When the display 805 is a touch display, the display 805 also has the ability to collect touch signals at or above the surface of the display 805. The touch signal can be input to the processor 801 as a control signal for processing. Optionally, the display 805 is also used to provide virtual buttons and/or virtual keyboards, also referred to as soft buttons and/or soft keyboards. In some embodiments, the display 805 is one, providing a front panel of the terminal 800; in other embodiments, the display 805 is at least two, and is respectively disposed on different surfaces of the terminal 800 or in a folded design; in still other embodiments, the display 805 is a flexible display disposed on a curved surface or a folded surface of the terminal 800. Even alternatively, the display 805 is arranged in a non-rectangular irregular pattern, i.e. a shaped screen. Optionally, the display 805 is made of LCD (Liquid CRYSTAL DISPLAY), OLED (Organic Light-Emitting Diode), or other materials.

The camera assembly 806 is used to capture images or video. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal and the rear camera is disposed on the rear surface of the terminal. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 806 also includes a flash. Alternatively, the flash is a single-color temperature flash, or a dual-color temperature flash. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and is used for light compensation under different color temperatures.

In some embodiments, audio circuit 807 includes a microphone and a speaker. The microphone is used for collecting sound waves of users and the environment, converting the sound waves into electric signals, inputting the electric signals to the processor 801 for processing, or inputting the electric signals to the radio frequency circuit 804 for voice communication. For the purpose of stereo acquisition or noise reduction, a plurality of microphones are respectively disposed at different portions of the terminal 800. Optionally, the microphone is an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 801 or the radio frequency circuit 804 into sound waves. Alternatively, the speaker is a conventional thin film speaker, or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only an electric signal but also an acoustic wave audible to humans can be converted into an acoustic wave inaudible to humans for ranging and other purposes. In some embodiments, audio circuit 807 also includes a headphone jack.

The power supply 808 is used to power the various components in the terminal 800. Alternatively, the power supply 808 is an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 808 includes a rechargeable battery, the rechargeable battery supports wired or wireless charging. The rechargeable battery is also used to support fast charge technology.

In some embodiments, the terminal 800 also includes one or more sensors 810. The one or more sensors 810 include, but are not limited to: acceleration sensor 811, gyro sensor 812, pressure sensor 813, optical sensor 814, and proximity sensor 815.

In some embodiments, the acceleration sensor 811 detects the magnitudes of accelerations on three coordinate axes of the coordinate system established with the terminal 800. For example, the acceleration sensor 811 is used to detect components of gravitational acceleration on three coordinate axes. Optionally, the processor 801 controls the display screen 805 to display a user interface in a landscape view or a portrait view according to the gravitational acceleration signal acquired by the acceleration sensor 811. The acceleration sensor 811 is also used for acquisition of motion data of a game or a user.

In some embodiments, the gyro sensor 812 detects the body direction and the rotation angle of the terminal 800, and the gyro sensor 812 and the acceleration sensor 811 cooperate to collect the 3D motion of the user on the terminal 800. The processor 801 performs the following functions based on the data collected by the gyro sensor 812: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

Optionally, the pressure sensor 813 is disposed at a side frame of the terminal 800 and/or at a lower layer of the display 805. When the pressure sensor 813 is disposed on a side frame of the terminal 800, a grip signal of the terminal 800 by a user can be detected, and the processor 801 performs left-right hand recognition or quick operation according to the grip signal collected by the pressure sensor 813. When the pressure sensor 813 is disposed at the lower layer of the display screen 805, the processor 801 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 805. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The optical sensor 814 is used to collect the ambient light intensity. In one embodiment, the processor 801 controls the display brightness of the display screen 805 based on the intensity of ambient light collected by the optical sensor 814. Specifically, when the intensity of the ambient light is high, the display brightness of the display screen 805 is turned up; when the ambient light intensity is low, the display brightness of the display screen 805 is turned down. In another embodiment, the processor 801 also dynamically adjusts the shooting parameters of the camera assembly 806 based on the ambient light intensity collected by the optical sensor 814.

A proximity sensor 815, also known as a distance sensor, is typically provided on the front panel of the terminal 800. The proximity sensor 815 is used to collect the distance between the user and the front of the terminal 800. In one embodiment, when the proximity sensor 815 detects a gradual decrease in the distance between the user and the front face of the terminal 800, the processor 801 controls the display 805 to switch from the bright screen state to the off screen state; when the proximity sensor 815 detects that the distance between the user and the front surface of the terminal 800 gradually increases, the processor 801 controls the display 805 to switch from the off-screen state to the on-screen state.

Those skilled in the art will appreciate that the structure shown in fig. 8 is not limiting and that more or fewer components than shown can be included or certain components combined or a different arrangement of components employed.

Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application, where the electronic device 900 may have a relatively large difference due to different configurations or performances, and the electronic device 900 includes one or more processors (Central Processing Units, CPU) 901 and one or more memories 902, where at least one computer program is stored in the memories 902, and the at least one computer program is loaded and executed by the one or more processors 901 to implement the image processing method provided in the foregoing embodiments. Optionally, the electronic device 900 further includes a wired or wireless network interface, a keyboard, an input/output interface, and other components for implementing the functions of the device, which are not described herein.

In an exemplary embodiment, a computer readable storage medium is also provided, for example a memory comprising at least one computer program executable by a processor in a terminal to perform the image processing method in the respective embodiments described above. For example, the computer readable storage medium includes ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, optical data storage device, and the like.

In an exemplary embodiment, a computer program product or computer program is also provided, comprising one or more program codes, the one or more program codes being stored in a computer readable storage medium. The one or more processors of the electronic device are capable of reading the one or more program codes from the computer-readable storage medium, and executing the one or more program codes to enable the electronic device to execute to perform the image processing method in the above-described embodiment.

Those of ordinary skill in the art will appreciate that all or a portion of the steps implementing the above-described embodiments can be implemented by hardware, or can be implemented by a program instructing the relevant hardware, optionally stored in a computer readable storage medium, optionally a read-only memory, a magnetic disk or an optical disk, etc.

The foregoing description of the preferred embodiments of the present application is not intended to limit the application, but rather, the application is to be construed as limited to the appended claims.

Claims

1. An image processing method, the method comprising:

acquiring at least one original image associated with the content item;

in the at least one original image, assigning the pixel points in the first area as target pixel values to obtain at least one first image;

obtaining at least one dominant hue based on the at least one first image recognition, wherein the dominant hue characterizes the color distribution trend of pixel points in the at least one first image;

Mapping pixel values of pixel points in the at least one first image based on the color mapping relation to obtain at least one second image;

In the at least one second image, assigning pixel points positioned in a first area to pixel values corresponding to pixel points at corresponding positions in an original image to obtain at least one target image, wherein the target image is the original image after the color of the first area is kept unchanged and the color of the second area is changed, and the second area is an area except the first area in the original image;

The modifying the tone H of the main tone to obtain a color-changing tone corresponding to the main tone comprises the following steps:

Adding the product of the multiplication of the sequence number of the color matching scheme and the scheme parameter to the hue H, wherein the scheme parameter is the ratio of 360 to N+1, and N is the number of the color matching schemes;

And taking the sum value pair 360 of the addition to obtain the color-changing tone corresponding to the main tone.

2. The method of claim 1, wherein the identifying a first region in the at least one original image, the first region being a region of the original image that does not require color change comprises:

Performing target detection on any original image to obtain a candidate region in the original image;

inputting the candidate region into a color change classification model, and outputting a color change detection result through the color change classification model, wherein the color change classification model is used for detecting whether the candidate region needs color change or not;

And determining a candidate area which is indicated by the color change detection result and does not need color change as the first area.

3. The method of claim 2, wherein performing object detection on any one of the original images to obtain candidate regions in the original image comprises:

4. The method of claim 1, wherein the identifying a first region in the at least one original image, the first region being a region of the original image that does not require color change comprises:

Performing object recognition on any original image to obtain an object recognition result, wherein the object recognition result represents whether the original image contains a target object or not;

and determining the area where the target object is located as the first area when the object identification result indicates that the original image contains the target object.

5. The method of claim 4, wherein determining the region in which the target object is located as the first region comprises:

6. The method of claim 1, wherein said deriving at least one dominant hue based on said at least one first image identification comprises:

splicing the at least one first image to obtain a spliced image;

7. The method of claim 1, wherein a hue difference between any two dominant hues is equal to a hue difference between corresponding two color-changing hues.

8. The method according to claim 1, wherein the method further comprises:

determining a white area in the at least one original image, wherein the difference value between the pixel value of the pixel point in the white area and the white value is smaller than a target threshold value;

And in the at least one second image, assigning the pixel points in the white area as pixel values corresponding to the pixel points in the corresponding position in the original image.

9. An image processing apparatus, characterized in that the apparatus comprises:

The mapping module comprises an assignment unit, which is used for assigning the pixel points in the first area as target pixel values in the at least one original image to obtain at least one first image;

the mapping module comprises a second identification unit, a first image processing unit and a second image processing unit, wherein the second identification unit is used for identifying at least one dominant tone based on the at least one first image, and the dominant tone characterizes the color distribution trend of pixel points in the at least one first image;

the mapping module comprises a mapping unit for converting the at least one first image from a red-green-blue RGB color space to a hue-saturation-brightness HSV color space;

The mapping unit is further configured to, for any dominant hue, keep the saturation S and the brightness V of the dominant hue unchanged in the HSV color space, and modify the hue H of the dominant hue to obtain a color-changing hue corresponding to the dominant hue;

the mapping unit is further used for determining a color mapping relation of a pixel level based on a main tone mapping relation of a main tone and a color changing tone, and the color mapping relation represents the corresponding relation of different pixel values before and after color changing;

the mapping unit is further configured to map pixel values of pixel points in the at least one first image based on the color mapping relationship, so as to obtain at least one second image;

The assignment unit is further configured to assign pixel points located in a first area to pixel values of pixel points corresponding to positions in an original image in the at least one second image, so as to obtain at least one target image, where the target image is the original image after the color of the first area is kept unchanged and the color of the second area is changed, and the second area is an area, except for the first area, in the original image;

10. The apparatus of claim 9, wherein the identifying a first region in the at least one original image, the first region being a region in the original image that does not require color change comprises:

11. The apparatus of claim 10, wherein performing object detection on any one of the original images to obtain candidate regions in the original image comprises:

12. The apparatus of claim 9, wherein the identifying a first region in the at least one original image, the first region being a region in the original image that does not require color change comprises:

13. The apparatus of claim 12, wherein the determining the area in which the target object is located as the first area comprises:

14. The apparatus of claim 9, wherein said deriving at least one dominant hue based on said at least one first image identification comprises:

splicing the at least one first image to obtain a spliced image;

15. The apparatus of claim 9, wherein a hue difference between any two dominant hues is equal to a hue difference between corresponding two color-changing hues.

16. The apparatus of claim 9, wherein the apparatus further comprises:

17. An electronic device comprising one or more processors and one or more memories, the one or more memories having stored therein at least one computer program loaded and executed by the one or more processors to implement the image processing method of any of claims 1-8.

18. A storage medium having stored therein at least one computer program loaded and executed by a processor to implement the image processing method of any one of claims 1 to 8.

19. A computer program product, characterized in that the computer program product comprises at least one computer program, which is loaded and executed by a processor to implement the image processing method of any one of claims 1 to 8.