[go: up one dir, main page]

WO2022267466A1 - 图像处理方法、装置和电子设备 - Google Patents

图像处理方法、装置和电子设备 Download PDF

Info

Publication number
WO2022267466A1
WO2022267466A1 PCT/CN2022/073190 CN2022073190W WO2022267466A1 WO 2022267466 A1 WO2022267466 A1 WO 2022267466A1 CN 2022073190 W CN2022073190 W CN 2022073190W WO 2022267466 A1 WO2022267466 A1 WO 2022267466A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image block
view
field
block
Prior art date
Application number
PCT/CN2022/073190
Other languages
English (en)
French (fr)
Other versions
WO2022267466A9 (zh
Inventor
丁大钧
乔晓磊
肖斌
朱聪超
Original Assignee
荣耀终端有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 荣耀终端有限公司 filed Critical 荣耀终端有限公司
Priority to EP22826975.9A priority Critical patent/EP4242961A4/en
Priority to US18/266,546 priority patent/US20240046604A1/en
Publication of WO2022267466A1 publication Critical patent/WO2022267466A1/zh
Publication of WO2022267466A9 publication Critical patent/WO2022267466A9/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/54—Extraction of image or video features relating to texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present application belongs to the field of image processing, and in particular relates to an image processing method, device and electronic equipment.
  • the mobile phone is configured with two cameras, one is a main camera, and the other is a wide-angle camera or a telephoto camera.
  • the field of view of the wide-angle camera is larger than that of the main camera, which is suitable for close-up shooting
  • the field of view of the telephoto camera is smaller than that of the main camera, which is suitable for long-range shooting.
  • the fused image will have poor stereoscopic effect and poor quality because the field of view angles of the two cameras do not match. poor.
  • the two images obtained by a mobile phone using such a dual camera there are parts where the field of view overlaps, and there are also parts where the field of view does not overlap. If the two images are directly fused, then the part of the final captured image where the field of view angles overlap has high definition, and the part that does not overlap has low definition, so that the captured image will have inconsistencies in the definition of the central part and surrounding parts.
  • the problem is that there will be a fusion boundary on the image, which will affect the imaging effect.
  • the present application provides an image processing method, device and electronic equipment, by adding texture information in an image with a small field of view to an image with a large field of view, so as to enhance the details and achieve the purpose of improving the clarity of the image with a large field of view .
  • an image processing method comprising:
  • the multiple frames of original images are images taken for the same scene to be shot.
  • the multiple frames of original images include: small field of view images and large field of view images, and the field of view corresponding to the large field of view images includes small field of view
  • the field angle corresponding to the field angle image; both the small field angle image and the large field angle image are segmented to obtain at least one first image block corresponding to the small field angle image and at least one second image block corresponding to the large field angle image.
  • Image blocks, at least one first image block and at least one second image block have a mapping relationship; extract texture information in at least one first image block, and add the extracted texture information to a target image block to obtain a target image,
  • the target image block is: a second image block having a mapping relationship with at least one first image block.
  • An embodiment of the present application provides an image processing method, by acquiring multiple frames of original images, segmenting images with small field angles in the multiple frames of original images to obtain at least one corresponding first image block, and at the same time, segmenting the images in the multiple frames of original images
  • the large field of view image is segmented to obtain at least one corresponding second image block. Since the angle of view corresponding to the image with a large angle of view includes the angle of view corresponding to the image with a small angle of view, at least one first image block after segmentation has a mapping relationship with at least one second image block, and based on this, from a
  • the texture information is extracted from one or more first image blocks and added to the second image block having a mapping relationship with the first image block from which the texture information is extracted.
  • the image with a small field of view has higher definition and richer details than an image with a large field of view
  • the details of the second image block can be enhanced, Furthermore, it can play a role in improving the clarity and quality of images with large viewing angles.
  • both the small field of view image and the large field of view image are segmented to obtain at least one first image block corresponding to the small field of view image and at least one first image block corresponding to the large field of view image.
  • the second image block includes: using a deep learning network model to perform semantic segmentation on both the small field of view image and the large field of view image, and obtain at least one first image block corresponding to the small field of view image and the large field of view image respectively At least one second image block corresponding to the image; each first image block and each second image block have a corresponding category, and when at least one first image block is a plurality of first image blocks, the plurality of first image blocks Corresponding to different categories, when at least one second image block is a plurality of second image blocks, the plurality of second image blocks correspond to different categories; wherein, the mapping relationship is: M first image blocks and N second image blocks The categories of are the same, M and N are integers greater than or equal to 1 respectively, and the categories are used to
  • both the small field of view image and the large field of view image are segmented to obtain at least one first image block corresponding to the small field of view image and at least one first image block corresponding to the large field of view image.
  • the second image block includes: segmenting both the small field of view image and the large field of view image according to the color or hue, respectively obtaining at least one first image block corresponding to the small field of view image and at least one corresponding to the large field of view image A second image block; each first image block and each second image block have a corresponding color or tone, and when at least one first image block is a plurality of first image blocks, the plurality of first image blocks correspond to different color or hue, when at least one second image block is a plurality of second image blocks, the plurality of second image blocks correspond to different colors or hues; wherein, the mapping relationship is: P first image blocks and Q second image blocks The color or hue of the image blocks is the same, and P and Q are integers greater than or equal to 1, respectively.
  • the method before extracting the texture information in the first image block and adding the extracted texture information to the target image block, the method further includes: attaching the small field of view image to The first area in the large field of view image; wherein, the first area is the area corresponding to the small field of view image in the large field of view image, and the second area is the area other than the first area in the large field of view image , the target image block is: a second image block that has a mapping relationship with at least one first image block and is located in the second area, and a sub-image block in the second image block that has a mapping relationship with at least one first image block, The sub image blocks are located in the second area.
  • the image with a small field of view has a higher definition than the image with a large field of view. Therefore, it is only necessary to add the texture information extracted from the first image block to the second image block located in the second area and the sub-image block located in the second area in the second image block, so that Reduce the amount of calculation and improve processing efficiency.
  • the method before attaching the small field of view image to the first region of the large field of view image, the method further includes: using the watershed algorithm to determine the connectivity corresponding to at least one second image block domain; after the small field of view image is attached to the first area of the large field of view image, the method also includes: determining whether the connected domain corresponding to at least one first image block and the target image block is connected; if connected, then determine at least Image entropy of a first image block; according to the image entropy of at least one first image block, determine the addition method of adding texture information extracted from at least one first image block to the target image block; with the determined addition method Add the extracted texture information to the target image block.
  • the accuracy of subsequent texture information addition can be improved by determining whether the connected domains corresponding to the first image block and the target image block are connected, and the texture information in the first image block is only added to the connected connected regions. in the target image block in the domain.
  • the method further includes: determining at least one first image block and the target The similarity or peak signal-to-noise ratio between image blocks; when the similarity or peak signal-to-noise ratio satisfies the corresponding preset thresholds, then determine the image entropy of at least one first image block; according to the at least one first image block Image entropy, determining an adding manner of adding texture information extracted from at least one first image block to the target image block; adding the extracted texture information to the target image block in the determined adding manner.
  • the first image block and the target image block with stronger correlation can be screened out, and on this basis, the details of the first image block can be known by determining the image entropy Whether there is more or less information, thus, different ways of adding texture information to the target image block can be determined according to different image entropies.
  • the method further includes: if they are not connected, determining the similarity or peak signal-to-noise ratio between at least one first image block and the target image block; when the similarity or peak signal-to-noise ratio When the ratio meets the corresponding preset thresholds, then determine the image entropy of at least one first image block; according to the image entropy of at least one first image block, determine that the texture information extracted from at least one first image block is added to the target image Addition method of the block; add the extracted texture information to the target image block in the determined addition method.
  • the first image block may also be the same object as the target image block, so the similarity can be further judged to determine whether to Add texture information.
  • determining the method of adding texture information extracted from at least one first image block to the target image block includes: when at least one When the image entropy of the first image block is less than the image entropy threshold, the method of adding texture information extracted from at least one first image block to the target image block is an explicit addition method; when the image entropy of at least one first image block is greater than or equal to the image entropy threshold, the addition method of adding the texture information extracted from at least one first image block to the target image block is an implicit addition method; wherein, the addition method includes an explicit addition method and an implicit addition method, explicit The adding mode is used to indicate that the texture information is not changed when the texture information is added, and the implicit adding mode is used to indicate that the texture information is adaptively deformed when the texture information is added.
  • the texture information can be directly copied and pasted to the The target image block has less calculation and high efficiency; and when the image entropy of the first image block is greater than or equal to the image entropy threshold, it indicates that there are more detailed information in the first image block.
  • the texture information extracted from the first image block is adaptively deformed and then added to the target image block, so that the content and texture information in the original target image block are fused more naturally, increasing image details, and improving visual effects.
  • the deep learning network model is any one of FCN, SegNet, and DeepLab.
  • an image processing apparatus in a second aspect, includes a unit for performing each step in the above first aspect or any possible implementation manner of the first aspect.
  • an image processing device including: a receiving interface and a processor; the receiving interface is used to obtain multiple frames of original images from electronic equipment; the multiple frames of original images are images taken for the same scene to be shot, and multiple The frame original image includes: a small field of view image and a large field of view image, and the field of view corresponding to the large field of view image includes the field of view corresponding to the small field of view image; the processor is used to call the computer program stored in the memory, To execute the processing steps in the image processing method provided in the first aspect or any possible implementation manner of the first aspect.
  • an electronic device including a camera module, a processor, and a memory; the camera module is used to obtain multiple frames of original images; the multiple frames of original images are images taken for the same scene to be shot, and the multiple frames
  • the original image includes: a small field of view image and a large field of view image, and the field of view corresponding to the large field of view image includes the field of view corresponding to the small field of view image;
  • the memory is used to store a computer program that can run on the processor a processor, configured to execute the processing steps in the image processing method provided in the first aspect or any possible implementation manner of the first aspect.
  • the camera module includes a first camera and a second camera
  • the first camera is used to take pictures of the scene to be shot with a first angle of view
  • the second camera is used to Taking pictures at the second field of view
  • the first field of view is smaller than the second field of view
  • the first camera is used to obtain a small field of view image after the processor obtains the photographing instruction, and the field of view corresponding to the small field of view image is The first field of view
  • the second camera used to obtain a large field of view image after the processor obtains the photographing instruction, and the field of view corresponding to the large field of view image is the second field of view.
  • a chip including: a processor, configured to call and run a computer program from a memory, so that a device installed with the chip executes the chip as provided in the first aspect or any possible implementation manner of the first aspect. image processing method.
  • a computer-readable storage medium stores a computer program, and the computer program includes program instructions.
  • the program instructions When executed by a processor, the processor executes the first aspect or the first method.
  • the image processing method provided in any possible implementation manner of the aspect.
  • a computer program product includes a computer-readable storage medium storing a computer program, and the computer program enables the computer to execute the image provided in the first aspect or any possible implementation manner of the first aspect Approach.
  • the image processing method, device, and electronic equipment provided by the present application obtain at least one corresponding first image block by obtaining multiple frames of original images, and segment the images with small field angles in the multiple frames of original images.
  • the multi-frame original The image with a large field of view in the image is segmented to obtain at least one corresponding second image block. Since the angle of view corresponding to the image with a large angle of view includes the angle of view corresponding to the image with a small angle of view, at least one first image block after segmentation has a mapping relationship with at least one second image block, and based on this, from a
  • the texture information is extracted from one or more first image blocks and added to the second image block having a mapping relationship with the first image block from which the texture information is extracted.
  • the image with a small field of view has higher definition and richer details than an image with a large field of view
  • the details of the second image block can be enhanced, Furthermore, it can play a role in improving the clarity and quality of images with large viewing angles.
  • FIG. 1 is a schematic diagram of processing images captured by dual cameras provided in the prior art
  • FIG. 2 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
  • FIG. 3 is a hardware architecture diagram of an image processing device provided in an embodiment of the present application.
  • FIG. 4 is a schematic flow diagram of an image processing method provided in an embodiment of the present application.
  • FIG. 5 is a schematic flow diagram of another image processing method provided in the embodiment of the present application.
  • FIG. 6 is a schematic diagram of a texture information adding method provided by an embodiment of the present application.
  • FIG. 7 is a schematic flowchart of another image processing method provided in the embodiment of the present application.
  • FIG. 8 is a schematic diagram of determining a connected domain corresponding to a second image block provided by an embodiment of the present application.
  • FIG. 9 is a schematic flowchart of another image processing method provided in the embodiment of the present application.
  • FIG. 10 is a schematic flowchart of another image processing method provided in the embodiment of the present application.
  • FIG. 11 is a schematic flowchart of another image processing method provided in the embodiment of the present application.
  • FIG. 12 is a schematic flowchart of another image processing method provided in the embodiment of the present application.
  • FIG. 13 is a schematic flowchart of another image processing method provided by the embodiment of the present application.
  • FIG. 14 is a schematic flowchart of another image processing method provided in the embodiment of the present application.
  • FIG. 15 is a schematic structural diagram of an image processing device provided by an embodiment of the present application.
  • FIG. 16 is a schematic structural diagram of a chip provided by the embodiment of the application.
  • a relationship means that there may be three kinds of relationships, for example, A and/or B means: A exists alone, A and B exist simultaneously, and B exists alone.
  • plural refers to two or more than two.
  • first and second are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, a feature defined as “first” and “second” may explicitly or implicitly include one or more of these features. In the description of this embodiment, unless otherwise specified, “plurality” means two or more.
  • FOV Field of view
  • the camera can be divided into a main camera, a wide-angle camera, and a telephoto camera due to different field of view angles.
  • the field of view of the wide-angle camera is larger than that of the main camera, and the focal length is smaller, which is suitable for close-up shooting; while the field of view of the telephoto camera is smaller than that of the main camera, and the focal length is longer. Suitable for remote shooting.
  • Connected domain refers to: the interior of any simple closed curve belonging to a region A belongs to A, then A is called a connected domain, or the connected domain can also be expressed as: surrounded by any closed curve in A The resulting area contains only the points in A.
  • Watershed algorithm (watershed algorithm), the watershed algorithm is a mathematical morphology segmentation method based on topology theory, and it is also a commonly used image segmentation method.
  • the basic idea is to regard the image as a topological landform in geodesy , the gray value of each pixel in the image represents the altitude of the point, and the depression in the image is the local minimum in the image.
  • the catchment basin is the area affected by the depression in the terrain, and it is also the area surrounding the local minimum in the image.
  • these sunken spots are submerged as the water level rises, eventually forming dikes around them, and these dikes are used to prevent water from other catchment basins from mixing into their own basins.
  • dams are watersheds.
  • the mobile phone is configured with two cameras, one is a main camera, and the other is a wide-angle camera or a telephoto camera, or the two cameras are respectively a wide-angle camera and a telephoto camera.
  • the angle of view of the wide-angle camera is larger than that of the main camera, and the angle of view of the telephoto camera is smaller than that of the main camera. Then, the image taken by the main camera and the image taken by the wide-angle camera, or; the image taken by the main camera and the image taken by the telephoto camera are simply fused; Simple fusion.
  • FIG. 1 shows a schematic diagram of processing images captured by dual cameras in the prior art.
  • the first field of view image taken by the main camera is usually filled in the second field of view image taken by the wide-angle camera according to the size of the field of view, or the telephoto
  • the image of the first field of view captured by the camera is filled in the image of the second field of view captured by the main camera or the wide-angle camera.
  • the fused image will have poor stereoscopic effect and poor quality.
  • the two images obtained by a mobile phone using such a dual camera there are parts where the field of view overlaps, and there are also parts where the field of view does not overlap. If the two images are fused directly, the overlapping parts and non-overlapping parts of the final captured image may not be in alignment, and part of the content may be broken or deformed.
  • the overlapping part of the field of view may have high definition, and the non-overlapping part may have low definition, so that the captured image will have the problem of inconsistency in the definition of the central part and the surrounding part, that is, there will be a fusion boundary on the image, which will affect the imaging. Effect.
  • the embodiment of the present application provides an image processing method, aiming at the small field of view image and the large field of view image with overlapping parts of the field of view, by adding the texture information in the small field of view image to the large field of view In the corner image, the details are enhanced to achieve the purpose of improving the clarity of the image with a large field of view.
  • the image processing method provided in the embodiment of the present application may be applicable to various electronic devices, and correspondingly, the image processing apparatus provided in the embodiment of the present application may be electronic devices in various forms.
  • the electronic device may be various camera devices such as SLR cameras and card players, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (augmented reality, AR)/virtual reality (virtual reality) reality, VR) equipment, notebook computer, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook, personal digital assistant (personal digital assistant, PDA), etc., or other equipment or devices capable of image processing,
  • camera devices such as SLR cameras and card players, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (augmented reality, AR)/virtual reality (virtual reality) reality, VR) equipment, notebook computer, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook, personal digital assistant (personal digital assistant, PDA), etc., or other equipment or devices capable of image processing
  • the embodiment of the present application does not set any limitation on the specific type of the electronic device.
  • FIG. 2 shows a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application.
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like.
  • SIM subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.
  • the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • baseband processor baseband processor
  • neural network processor neural-network processing unit
  • the controller may be the nerve center and command center of the electronic device 100 .
  • the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is a cache memory.
  • the memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
  • the processor 110 may run the software code of the image processing method provided in the embodiment of the present application to capture an image with higher definition.
  • processor 110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input and output
  • subscriber identity module subscriber identity module
  • SIM subscriber identity module
  • USB universal serial bus
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .
  • MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc.
  • the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device 100 .
  • the processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device 100 .
  • the GPIO interface can be configured by software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on.
  • the GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like.
  • the USB interface 130 can be used to connect a charger to charge the electronic device 100 , and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.
  • the interface connection relationship between the modules shown in the embodiment of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 .
  • the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
  • the charging management module 140 is configured to receive a charging input from a charger.
  • the power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 .
  • the power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 .
  • the wireless communication function of the electronic device 100 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, a baseband processor, and the like.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.
  • Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
  • the antenna may be used in conjunction with a tuning switch.
  • the mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the electronic device 100 .
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like.
  • the mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation.
  • at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 .
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.
  • the wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite, etc. applied on the electronic device 100.
  • System global navigation satellite system, GNSS
  • frequency modulation frequency modulation, FM
  • near field communication technology near field communication, NFC
  • infrared technology infrared, IR
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
  • the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR techniques, etc.
  • GSM global system for mobile communications
  • GPRS general packet radio service
  • code division multiple access code division multiple access
  • CDMA broadband Code division multiple access
  • WCDMA wideband code division multiple access
  • time division code division multiple access time-division code division multiple access
  • TD-SCDMA time-division code division multiple access
  • the GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou navigation satellite system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • Beidou navigation satellite system beidou navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite based augmentation systems
  • the electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos and the like.
  • the display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc.
  • the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.
  • Camera 193 is used to capture images. It can be triggered by an application command to realize the camera function, such as capturing images of any scene.
  • a camera may include components such as an imaging lens, an optical filter, and an image sensor. The light emitted or reflected by the object enters the imaging lens, passes through the filter, and finally converges on the image sensor.
  • the image sensor is mainly used for converging and imaging the light emitted or reflected by all objects in the camera perspective (also called the scene to be shot, the target scene, or the scene image that the user expects to shoot); the filter is mainly used to It is used to filter out redundant light waves (such as light waves other than visible light, such as infrared) in the light; the image sensor is mainly used to perform photoelectric conversion on the received light signal, convert it into an electrical signal, and input it into the processor 130 for subsequent processing .
  • the camera 193 may be located at the front of the electronic device 100, or at the back of the electronic device 100, and the specific number and arrangement of the cameras may be set according to requirements, which are not limited in this application.
  • the electronic device 100 includes a front camera and a rear camera.
  • a front camera or a rear camera may include one or more cameras.
  • the image processing method provided in the embodiment of the present application may be used.
  • the camera is arranged on an external accessory of the electronic device 100, the external accessory is rotatably connected to the frame of the mobile phone, and the angle formed between the external accessory and the display screen 194 of the electronic device 100 is 0-360 degrees any angle between.
  • the electronic device 100 takes a selfie
  • the external accessory drives the camera to rotate to a position facing the user.
  • the mobile phone has multiple cameras, only some of the cameras may be set on the external accessories, and the rest of the cameras may be set on the electronic device 100 body, which is not limited in this embodiment of the present application.
  • the internal memory 121 may be used to store computer-executable program codes including instructions.
  • the internal memory 121 may include an area for storing programs and an area for storing data.
  • the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like.
  • the storage data area can store data created during the use of the electronic device 100 (such as audio data, phonebook, etc.) and the like.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
  • the internal memory 121 can also store the software code of the image processing method provided by the embodiment of the present application.
  • the processor 110 runs the software code, it executes the process steps of the image processing method to obtain an image with higher definition.
  • the internal memory 121 can also store captured images.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, so as to expand the storage capacity of the electronic device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving files such as music in an external memory card.
  • the software code of the image processing method provided in the embodiment of the present application can also be stored in an external memory, and the processor 110 can run the software code through the external memory interface 120 to execute the process steps of the image processing method to obtain a high-definition image.
  • Image Images captured by the electronic device 100 may also be stored in an external memory.
  • the user can designate whether to store the image in the internal memory 121 or the external memory.
  • the electronic device 100 when the electronic device 100 is currently connected to the external memory, if the electronic device 100 captures one frame of image, a prompt message may pop up to remind the user whether to store the image in the external memory or the internal memory; of course, there may be other specified ways , the embodiment of the present application does not impose any limitation on this; or, when the electronic device 100 detects that the amount of memory in the internal memory 121 is less than a preset amount, it may automatically store the image in the external memory.
  • the electronic device 100 can implement audio functions through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.
  • the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal.
  • pressure sensor 180A may be disposed on display screen 194 .
  • the gyro sensor 180B can be used to determine the motion posture of the electronic device 100 .
  • the angular velocity of the electronic device 100 around three axes ie, x, y and z axes
  • the gyro sensor 180B can be used for image stabilization.
  • the air pressure sensor 180C is used to measure air pressure.
  • the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
  • the magnetic sensor 180D includes a Hall sensor.
  • the electronic device 100 may detect opening and closing of the flip holster using the magnetic sensor 180D.
  • the electronic device 100 when the electronic device 100 is a clamshell machine, the electronic device 100 can detect opening and closing of the clamshell according to the magnetic sensor 180D.
  • features such as automatic unlocking of the flip cover are set.
  • the acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
  • the distance sensor 180F is used to measure the distance.
  • the electronic device 100 may measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.
  • Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes.
  • the light emitting diodes may be infrared light emitting diodes.
  • the electronic device 100 emits infrared light through the light emitting diode.
  • Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 .
  • the electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to make a call, so as to automatically turn off the screen to save power.
  • the proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.
  • the ambient light sensor 180L is used for sensing ambient light brightness.
  • the electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, take pictures with fingerprints, answer incoming calls with fingerprints, and the like.
  • the temperature sensor 180J is used to detect temperature.
  • the electronic device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the electronic device 100 may reduce the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection.
  • the electronic device 100 when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from being shut down abnormally due to the low temperature.
  • the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
  • the touch sensor 180K is also called “touch device”.
  • the touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to the touch operation can be provided through the display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the position of the display screen 194 .
  • the bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone.
  • the audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function.
  • the application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.
  • the keys 190 include a power key, a volume key and the like.
  • the key 190 may be a mechanical key. It can also be a touch button.
  • the electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100 .
  • the motor 191 can generate a vibrating reminder.
  • the motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.
  • touch operations applied to different applications may correspond to different vibration feedback effects.
  • the indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.
  • the SIM card interface 195 is used for connecting a SIM card.
  • the SIM card can be connected and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 .
  • the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • FIG. 3 shows a hardware architecture diagram of an image processing apparatus 200 provided by an embodiment of the present application.
  • the image processing device 200 may be, for example, a processor chip.
  • the hardware architecture diagram shown in FIG. 3 may be the processor 110 in FIG. 2 , and the image processing method provided in the embodiment of the present application may be applied on the processor chip.
  • the image processing apparatus 200 includes: at least one CPU, a memory, a microcontroller (microcontroller unit, MCU), a GPU, an NPU, a memory bus, a receiving interface, a sending interface, and the like.
  • the image processing device 200 may also include an AP, a decoder, a dedicated graphics processor, and the like.
  • the connectors include various interfaces, transmission lines or buses, etc. These interfaces are usually electrical communication interfaces, but they may also be mechanical interfaces or other forms The interface of this application does not impose any restrictions on it.
  • the CPU may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor.
  • the CPU may be a processor group composed of multiple processors, and the multiple processors are coupled to each other through one or more buses.
  • the connection interface can be the data input interface of the processor chip.
  • the receiving interface and the sending interface can be high definition multimedia interface (high definition multimedia interface, HDMI), V-By-One Interface, embedded display port (embedded display port, eDP), mobile industry processor interface (mobile industry processor interface, MIPI) display port (DP), etc.
  • the memory can refer to the above description of the internal memory 121.
  • the above-mentioned parts are integrated on the same chip.
  • the CPU, GPU, decoder, receiving interface, and sending interface are integrated on one chip, and each part inside the chip accesses an external memory through a bus.
  • a dedicated graphics processor can be a dedicated ISP.
  • the NPU can also be used as an independent processor chip.
  • the NPU is used to implement related operations of various neural networks or deep learning.
  • the image processing method provided in the embodiment of the present application may be implemented by a GPU or an NPU, or may be implemented by a dedicated graphics processor.
  • the chip involved in the embodiment of the present application is a system manufactured on the same semiconductor substrate by an integrated circuit process, also called a semiconductor chip, which may be an integrated circuit formed on the substrate by using an integrated circuit process. Assemblies whose outer layers are usually encapsulated by semiconductor encapsulation materials.
  • the integrated circuit may include various functional devices, and each type of functional device includes transistors such as logic gate circuits, metal oxide semiconductor (MOS) transistors, and diodes, and may also include other components such as capacitors, resistors, or inductors.
  • MOS metal oxide semiconductor
  • Each functional device can work independently or under the action of necessary driver software, and can realize various functions such as communication, calculation or storage.
  • FIG. 4 is a schematic flowchart of an image processing method shown in an embodiment of the present application. As shown in FIG. 4, the image processing method 10 includes: S10 to S30.
  • the multi-frame original images include: images with a small field of view and images with a large field of view.
  • the viewing angle corresponding to the image with the large viewing angle includes the viewing angle corresponding to the image with the small viewing angle.
  • the execution subject of the image processing method may be the electronic device 100 provided with the camera module as shown in FIG. 2 , or the image processing apparatus 200 shown in FIG. 3 .
  • the execution subject is an electronic device
  • multiple frames of original images are obtained through the cameras in the camera module. Specifically, several cameras or which camera is used to obtain multiple frames can be set and changed as required, and this embodiment of the application does not impose any restrictions on this .
  • the execution subject is an image processing device
  • multiple frames of original images can be obtained through the receiving interface, and the multiple frames of original images are captured by a camera module of an electronic device connected to the image processing device.
  • the aforementioned original image may also be called a RAW image.
  • the above-mentioned original image may be an original image directly captured, or may be an original image intercepted from a captured video.
  • the small field-of-view image and the large-field-of-view image may include one frame or multiple frames respectively, but at least the multi-frame original images acquired include one frame of small field of view image and one frame of large field of view. Field of view image.
  • the image with a small field of view and the image with a large field of view may or may not be obtained at the same time, but at least they should be images shot for the same scene to be shot in the same time period.
  • the same scene to be shot is shot within 2 seconds to obtain multiple frames of images with a small field of view and multiple frames of images with a large field of view.
  • the dimensions of multiple frames of original images may all be the same.
  • the sizes of the multiple frames of original images may also be partly the same, partly different; or completely different.
  • the embodiment of the present application does not impose any limitation on this.
  • multiple frames of original images may be acquired continuously, and the acquisition intervals may be the same or different.
  • multiple frames of original images may not be acquired continuously, and this embodiment of the present application does not impose any limitation on this.
  • the viewing angle corresponding to the large viewing angle image is larger than the viewing angle corresponding to the small viewing angle image, and the viewing angle corresponding to the large viewing angle image includes the viewing angle corresponding to the small viewing angle image, thus,
  • the image with a large field of view and the image with a small field of view have an overlapping portion of the field of view, and the overlapping portion of the field of view is the corresponding field of view of the image with a small field of view. That is to say, the image with a large field of view includes content in the image with a small field of view.
  • At least one first image block has a mapping relationship with at least one second image block, that is, one or more first image blocks have a mapping relationship with one or more second image blocks.
  • the number of the first image block and the number of the second image blocks respectively obtained by segmenting both the large field of view image and the small field of view image may be the same or may be different.
  • the number of second image blocks obtained by segmenting the large field of view image should be greater than or equal to the number of first image blocks obtained by segmenting the small field of view image.
  • the second image block obtained by segmenting the image with a large viewing angle should include at least one first image block obtained by segmenting an image with a small viewing angle.
  • mapping relationship there may be a quantitative mapping relationship between the first image block and the second image block, and there may also be a content mapping relationship, which may be a one-to-one mapping relationship, or a one-to-many, many-to-one mapping relationship. Or a many-to-many mapping relationship, the specific mapping relationship is determined by the segmentation method, and the segmentation method can be selected and modified according to needs, and this embodiment of the present application does not impose any restrictions on this.
  • the above S20 may include:
  • semantic segmentation semantic segmentation
  • the second image block is obtained.
  • each first image block and each second image block has a corresponding category, and after the semantic segmentation is performed on the small field of view image, when only one first image block is obtained, the first image block corresponds to a category; When multiple first image blocks are obtained, the multiple first image blocks correspond to different categories.
  • the second image block corresponds to a category; when multiple second image blocks are obtained, the multiple second image blocks correspond to different category.
  • mapping relationship is as follows: the categories of the M first image blocks and the N second image blocks are the same, and M and N are integers greater than or equal to 1 respectively.
  • semantics refers to the meaning of speech in speech recognition, but in the field of images, semantics refers to the content of images and the understanding of the meaning of images. Based on this, semantic segmentation refers to segmenting different objects in an image from the perspective of pixels.
  • the category is the content attribute of different image blocks in the image.
  • the category is the category or category corresponding to the different objects in the image.
  • the category can be preset and changed as needed, and this embodiment of the present application does not impose any limitation on this.
  • categories can be animals such as “cat” and “dog”, plants such as “tree” and “flower”, or objects such as “table” and "car”.
  • the image blocks corresponding to the 3 people can be segmented, and the category of each image block corresponds to the "person" category , and the segmented image blocks corresponding to the three bicycles, the category of each image block corresponds to the "bicycle" category.
  • the plurality of first image blocks may correspond to different categories for each first image, or multiple groups of first image blocks may correspond to different categories, and each group of first image blocks includes one or more first image blocks.
  • the plurality of second image blocks may correspond to different categories, or multiple groups of second image blocks may correspond to different categories, and each group of second image blocks includes one or more second image blocks.
  • the image with a large field of view and the image with a small field of view may contain multiple image blocks belonging to the same category
  • the field of view image corresponding to the large field of view image includes the field of view corresponding to the small field of view image, that is, the large field of view image contains the content of the small field of view image, so there may be: M first images
  • the category of the block is the same as that of the N second image blocks, and M and N are integers greater than or equal to 1 respectively.
  • the mapping relationship is: the category of one first image block is the same as the category of multiple second image blocks, or; the category of multiple first image blocks is the same as the category of one second image block, or; multiple The category of the first image block is the same as the category of the plurality of second image blocks.
  • the deep learning network model for semantic segmentation can be any one of fully convolutional network (FCN), SegNet, and DeepLab.
  • FCN fully convolutional network
  • SegNet SegNet
  • DeepLab DeepLab
  • the deep learning network model can also be other models, as long as it can realize the semantic segmentation function, which can be specifically set according to needs, and the embodiment of the present application does not impose any limitation.
  • the above S20 may also include:
  • both the small viewing angle image and the large viewing angle image are segmented to obtain at least one first image block corresponding to the small viewing angle image and at least one second image block corresponding to the large viewing angle image.
  • each first image block and each second image block have a corresponding color or hue, and after the small field of view image is segmented, when only one first image block is obtained, the first image block corresponds to a color or hue; when multiple first image blocks are obtained, the multiple first image blocks correspond to different colors or hues.
  • the second image block corresponds to a color or hue; when multiple second image blocks are obtained, the multiple second image blocks correspond to different color or hue.
  • the above mapping relationship is as follows: the P first image blocks and the Q second image blocks have the same color or hue, and P and Q are integers greater than or equal to 1, respectively.
  • the plurality of first image blocks may correspond to different colors or hues for each first image, or may correspond to different colors or hues for multiple groups of first image blocks, and each group of first image blocks includes one or A plurality of first image blocks.
  • the plurality of second image blocks may correspond to different colors or hues for each second image block, or multiple groups of second image blocks may correspond to different colors or hues, and each group of second image blocks includes one or more second image blocks Two image blocks.
  • both the large viewing angle image and the small viewing angle image may contain multiple image blocks belonging to the same color or hue
  • in the small viewing angle image there may be multiple first image blocks corresponding to the color or The hue is the same.
  • in the image with a large field of view there may be multiple second image blocks corresponding to the same color or hue.
  • P first image blocks and Q second image blocks have the same color, or, P first image blocks and Q second image blocks
  • the hues of the image blocks are the same, and P and Q are integers greater than or equal to 1, respectively.
  • the mapping relationship is: the color of one first image block is the same as the color of multiple second image blocks, or; the color of multiple first image blocks is the same as the color of one second image block, or; multiple The color of a first image block is the same as the color of a plurality of second image blocks.
  • the above S30 can also be expressed as: extracting texture information of at least some of the first image blocks, and adding the extracted texture information to the target image block to obtain the target image.
  • the target image block is: a second image block that has a mapping relationship with the first image block whose texture information is extracted. That is to say, for any first image block whose texture information is extracted, the target image block is: one or more second image blocks that have a mapping relationship with the first image block whose texture information is extracted.
  • the specific number of the second image blocks depends on the mapping relationship between the second image blocks and the first image blocks after division.
  • the target image is a large field of view image with texture information added.
  • the texture information in this application refers to the uneven grooves on the surface of the object, and also includes the color patterns on the smooth surface of the object, which are usually more called patterns.
  • the texture information can reflect the details of the object in the first image block.
  • the first image block segmented by the image with a small field of view is also clearer than the second image block segmented by the image with a large field of view. Therefore, after the texture information in the first image block can be extracted, it can be added to the second image block that has a mapping relationship with the first image block of the extracted texture information, so as to improve the quality of these second image blocks. clarity.
  • texture information of the second image block except for the texture information of the second image block, other information such as color, high dynamic range (high dynamic range, HDR), and brightness remain unchanged.
  • An embodiment of the present application provides an image processing method, by acquiring multiple frames of original images, segmenting images with small field angles in the multiple frames of original images to obtain at least one corresponding first image block, and at the same time, segmenting the images in the multiple frames of original images
  • the large field of view image is segmented to obtain at least one corresponding second image block. Since the viewing angle corresponding to the large viewing angle image includes the viewing angle corresponding to the small viewing angle image, at least one first image block after segmentation has a mapping relationship with at least one second image block. Based on this, from 1 Extract texture information from one or more first image blocks and add it to a second image block that has a mapping relationship with the first image block whose texture information is extracted.
  • the image with a small field of view has higher definition and richer details than an image with a large field of view
  • the details of the second image block can be enhanced, Furthermore, it can play a role in improving the clarity and quality of images with large viewing angles.
  • the method 10 may further include S40.
  • the first region is a region in the image with a large field of view corresponding to the image with a small field of view.
  • the second area is an area in the image with a large field of view except the first area.
  • the first area is the area in the image with the large angle of view that overlaps with the angle of view of the image with the small angle of view
  • the second area is the area that does not overlap with the angle of view of the image with the small angle of view.
  • the central area of the image with a large field of view overlaps with the field of view of the image with a small field of view
  • the central area is the first area
  • the area surrounding the first area is the second area.
  • the target image block in S30 will indicate: a second image block that has a mapping relationship with the first image block whose texture information is extracted and is located in the second area;
  • the sub-image block in the second image block in which the texture information is extracted from the first image block has a mapping relationship, and the sub-image block is located in the second area.
  • the second image block located in the first area should be excluded, and the second image block with The overlapping sub-image blocks of the first area, and only the second image block located in the second area, and adding texture to the sub-image blocks overlapping with the second area in the second image block.
  • the method 10 may further include: S21 to S23.
  • S21 Determine the similarity or peak signal to noise ratio (peak signal to noise ratio, PSNR) between at least one first image block and the target image block.
  • PSNR peak signal to noise ratio
  • a structural similarity (SSIM) algorithm may be used to determine the similarity between at least one first image block and the target image block.
  • the preset thresholds corresponding to the similarity and the preset thresholds corresponding to the peak signal-to-noise ratio can be set and changed as required, and this embodiment of the present application does not impose any limitation on this.
  • image entropy refers to a statistical form of image features, which reflects the average amount of information in an image.
  • the one-dimensional entropy of the image refers to the amount of information contained in the aggregation features of the gray distribution in the image. For example, let Pi represent the proportion of pixels with gray value i in the image, so the one-dimensional entropy can be calculated according to the following formula:
  • the one-dimensional entropy calculated according to the above formula is the image entropy.
  • the determined image entropy of the first image block is large, it indicates that the first image block has a large amount of information and more details; when the determined image entropy of the first image block is small, it indicates that the information of the first image block Small amount, less detail.
  • an image entropy threshold may be set, and the image entropy of the first image block is compared with the image entropy threshold to determine the amount of information of the first image block.
  • the size of the image entropy threshold can be set and modified as required, which is not limited in this embodiment of the present application.
  • the above S23 can also be expressed as: according to the image entropy of the first image block from which texture information is extracted, determine the method of adding texture information extracted from the first image block from which texture information is extracted to the target image block.
  • the texture information in at least one first image block is extracted, and the extracted texture information is added to the target image block in a determined adding manner to obtain a target image.
  • the above S23 may further include:
  • the addition mode of adding the texture information extracted from at least one first image block to the target image block is an explicit addition mode.
  • the adding method of adding the texture information extracted from the at least one first image block to the target image block is an implicit adding method.
  • the above S30 corresponds to: adding the extracted texture information to the target image block in an explicit adding manner.
  • the above S30 corresponds to: adding the extracted texture information to the target image block in the implicit adding manner.
  • the adding method includes an explicit adding method and an implicit adding method.
  • the explicit addition mode is used to indicate that the texture information is not changed when the texture information is added
  • the implicit addition mode is used to indicate that the texture information is adaptively deformed when the texture information is added.
  • the explicit adding method refers to: copying and pasting the texture information extracted from the first image block to the corresponding target image block. If the area of the second image block is relatively large, the texture information extracted from the first image block may be copied and pasted into the corresponding target image block multiple times in a certain order to fill the target image block.
  • the implicit addition method refers to: when copying and pasting the texture information extracted from the first image block to the corresponding target image block, perform adaptive deformation according to the shape and content of the target image block, for example: perform rotation , scaling, stretching, etc. At this time, if the area of the second image block is relatively large, the texture information extracted from the first image block can be copied and pasted to the corresponding target image block multiple times, and the processing performed each time can be different, so that The texture information added to the target image block is not exactly the same.
  • the texture information of the first image block is less than the image entropy threshold, it indicates that the detail information in the first image block is less, at this time, the texture information can be directly copied and pasted to the target image block in a simple manner, The amount of calculation is small and the efficiency is high; and when the image entropy of the first image block is greater than or equal to the image entropy threshold, it indicates that there are more detailed information in the first image block.
  • the target image block The visual effect is poor, and the small image blocks that look exactly the same are arranged rigidly together, causing the content to break sometimes, causing image distortion and other problems. Therefore, it is necessary to extract from the first image block in an implicit way.
  • the texture information of the target image is adaptively deformed and then added to the target image block, so that the content and texture information in the original target image block can be fused more naturally, increasing the details of the image, and improving the visual effect.
  • the image entropy of the first image block is less than the image entropy threshold, therefore, the texture information extracted from the first image block is directly copied and pasted into the target image block without Make any changes, that is, add texture information to the target image block in an explicit way.
  • the image entropy of the first image block is greater than the image entropy threshold, at this time, since the petal a in the target image block is blooming towards the left, and the petal b is blooming towards the right, then, when adding texture information , the texture information extracted from the first image block can be adaptively deformed according to the position and shape of petal a and added to petal a, and the texture information can be adaptively deformed according to the position and shape of petal b and then added to petal b , that is, add texture information to the target image block in an implicit way.
  • the method 10 may further include S50 .
  • the image with a large field of view may be divided into a plurality of second image blocks (such as K1-K10), and then, as shown in (b) in FIG. 8, Using the watershed algorithm, it can be determined that K1 corresponds to the connected domain L1, K2 ⁇ K3 corresponds to the connected domain L2, K5 corresponds to the connected domain L3, K6 corresponds to the connected domain L4, K7 corresponds to the connected domain L5, K8 corresponds to the connected domain L7, and K9 corresponds to the connected domain L7. K10 corresponds to the connected domain L8.
  • K1 corresponds to the connected domain L1
  • K2 ⁇ K3 corresponds to the connected domain L2
  • K5 corresponds to the connected domain L3
  • K6 corresponds to the connected domain L4
  • K7 corresponds to the connected domain L5
  • K8 corresponds to the connected domain L7
  • K9 corresponds to the connected domain L7.
  • K10 corresponds to the connected domain L8.
  • the method 10 further includes S60 and S70 .
  • the target image block is: the second image block having a mapping relationship with the first image block whose texture information is extracted
  • the connected domain corresponding to the target image block is the connected domain corresponding to the second image block having a mapping relationship.
  • the target image block is: the first image block that has a mapping relationship with the extracted texture information and is located in the second area.
  • Two image blocks, and sub-image blocks in the second image block that have a mapping relationship with the extracted texture information, the sub-image blocks are located in the second area.
  • the connected domain corresponding to the target image block is the connected domain corresponding to the second image block.
  • the target image block is: a sub-image block in the second image block having a mapping relationship with the extracted texture information
  • the sub-image block is located in the second area.
  • the connected domain corresponding to the target image block is the connected domain corresponding to the sub-image block.
  • the relative position between the first image block and the target image block is fixed. Therefore, it can be determined by pre-recording the coordinates Whether the connected domains corresponding to the first image block and the target image block are connected.
  • any algorithm such as scale-invariant feature transform (SIFT), SURF, ORB, etc. to perform key point detection on the first image block and the target image block; then calculate the homography ( homography) matrix, that is, convert the position of the key point in the world coordinate system to the position in the pixel coordinate system. Then, the relative position of the key point in the first image block and the target image block in the pixel coordinate system can be obtained through coordinate mapping, so that the connection between the first image block and the target image block can be judged according to the relative position Whether the domain is connected.
  • SIFT scale-invariant feature transform
  • SURF SURF
  • ORB reliable feature transform
  • first image block and the second image block correspond to the same category, for example, they are both leaves, they may be leaves of different trees. Therefore, in order to improve accuracy, it is possible to further judge the difference between the first image block and the target image. Whether the connected domain corresponding to the block is connected, when the first image block is connected to the connected domain corresponding to the target image block, the probability of being the leaves of the same tree is greater. Therefore, the accuracy of subsequent texture information addition can be improved by determining whether the connected domain corresponding to the first image block and the target image block are connected, and the texture information in the first image block is only added to the target in the connected connected domain. image block.
  • the image entropy of the at least one first image block it is determined how to add the texture information extracted from the at least one first image block to the target image block, and then use the determined explicit addition method or implicit
  • the adding method adds the extracted texture information to the target image block.
  • the method described in S21-S23 above can be used to continue to determine the similarity or peak signal-to-noise ratio between at least one first image block and the target image block, according to whether the similarity or peak signal-to-noise ratio is Satisfy the corresponding preset thresholds, and then further determine whether to calculate the image entropy of the first image block, when it is necessary to determine the image entropy of the first image block, then continue to judge according to the size of the determined image entropy of the first image block Texture information is added to the target image block in an explicit or implicit way.
  • Embodiment 1 an image processing method, as shown in FIG. 10 , the method includes the following S1010 to S1030.
  • the two frames of original images include: one frame of an image with a small field of view and one frame of an image with a large field of view, and the field of view corresponding to the image with a large field of view includes a field of view corresponding to the image with a small field of view.
  • the above S1020 can be expressed as: performing semantic segmentation on the image with a small field of view using a deep learning network model to obtain four first image blocks corresponding to the image with a small field of view, and each first image block has a corresponding category.
  • x1 is the sky
  • x2 is the trunk
  • x3 is the leaves
  • x4 is the grass.
  • Semantic segmentation is performed on the image with a large field of view using a deep learning network model, and six second image blocks corresponding to the image with a large field of view are obtained, and each second image block has a corresponding category.
  • y1 is the sky
  • y2 is the trunk
  • y3 is the leaves
  • y4 is the grass
  • y5 and y6 are the clouds.
  • mapping relationship between the first image block and the second image block is: x1 and y1 have the same type, x2 and y2 have the same category, x3 and y3 have the same category, x4 and y4 have the same category, and y5 and y6 have no The first image blocks of the same category.
  • the deep learning network model is any one of FCN, SegNet, and DeepLab.
  • the target image block is: one or more second image blocks of the same type as the first image block whose texture information is extracted.
  • the target image is: a large field of view image with texture information added.
  • the above S1030 can be expressed as: extract the texture information in x1, and add the extracted texture information to y1; extract the texture information in x2, and add the extracted texture information to y2; The texture information is extracted, and the extracted texture information is added to y3; the texture information in x4 is extracted, and the extracted texture information is added to y4.
  • Embodiment 2 an image processing method, as shown in FIG. 11 , the method includes the following steps S2010 to S2070.
  • S2010 Acquire 2 frames of original images, where the 2 frames of original images are images captured for the same scene to be captured.
  • the two frames of original images include: one frame of an image with a small field of view and one frame of an image with a large field of view, and the field of view corresponding to the image with a large field of view includes a field of view corresponding to the image with a small field of view.
  • the image with a small field of view is relatively clear, and the image with a large field of view is relatively blurred.
  • the above S2020 can be expressed as: performing semantic segmentation on the image with a small field of view using a deep learning network model to obtain four first image blocks corresponding to the image with a small field of view, and each first image block has a corresponding category.
  • x1 is the sky
  • x2 is the trunk
  • x3 is the leaves
  • x4 is the grass.
  • Semantic segmentation is performed on the image with a large field of view using a deep learning network model, and six second image blocks corresponding to the image with a large field of view are obtained, and each second image block has a corresponding category.
  • y1 is the sky
  • y2 is the trunk
  • y3 is the leaves
  • y4 is the grass
  • y5 and y6 are the clouds.
  • mapping relationship between the first image block and the second image block is: x1 and y1 have the same type, x2 and y2 have the same category, x3 and y3 have the same category, x4 and y4 have the same category, and y5 and y6 have no category the same first image block.
  • the deep learning network model is any one of FCN, SegNet, and DeepLab.
  • S2030 Determine the similarity or peak signal-to-noise ratio between each first image block and the target image block.
  • the above S2030 can be expressed as: determining the similarity or peak signal-to-noise ratio between x1 and y1; determining the similarity or peak signal-to-noise ratio between x2 and y2; determining the similarity or peak signal-to-noise ratio between x3 and y3 ; Determine the similarity or peak signal-to-noise ratio between x4 and y4.
  • the above S2040 can be expressed as: if the similarity between x1 and y1 meets the corresponding preset threshold, then determine the image entropy of x1; if the similarity between x2 and y2 is not less than the preset threshold, then do not continue processing; x3 If the similarity between x4 and y3 meets the corresponding preset threshold, the image entropy corresponding to x3 is determined, and the similarity between x4 and y4 also meets the corresponding preset threshold, then the image entropy corresponding to x4 is determined.
  • the process of determining the image entropy of each first image block according to the peak signal-to-noise ratio is similar and will not be repeated here.
  • the explicit addition mode is used to indicate that the texture information is not changed when the texture information is added; the implicit addition mode is used to indicate that the texture information is adaptively deformed when the texture information is added.
  • the target image is: an image with a large field of view added with texture information in an explicit and/or implicit way.
  • the above 2050 to S2070 can be expressed as: the above image entropy of x1 is less than the image entropy threshold, then the texture information extracted from x1 is added to y1 in an explicit way; the image entropy corresponding to the above x3 is greater than the image entropy threshold, then the texture information is implicitly
  • the texture information extracted from x3 is added to y3 in the formula addition method; the image entropy of x4 is equal to the image entropy threshold, and the texture information extracted from x4 is also added to y4 in the implicit addition method, thereby obtaining the target image.
  • Embodiment 3 an image processing method, as shown in FIG. 12 , the method includes the following steps S3010 to S3070.
  • the two frames of original images include: one frame of an image with a small field of view and one frame of an image with a large field of view, and the field of view corresponding to the image with a large field of view includes a field of view corresponding to the image with a small field of view.
  • the image with a small field of view is relatively clear, and the image with a large field of view is relatively blurred.
  • the above S3020 can be expressed as: performing semantic segmentation on the image with a small field of view using a deep learning network model to obtain 4 first image blocks corresponding to the image with a small field of view, and each first image block has a corresponding category.
  • x1 is the sky
  • x2 is the trunk
  • x3 is the leaves
  • x4 is the grass.
  • Semantic segmentation is performed on the image with a large field of view using a deep learning network model, and six second image blocks corresponding to the image with a large field of view are obtained, and each second image block has a corresponding category.
  • y1 is the sky
  • y2 is the trunk
  • y3 is the leaves
  • y4 is the grass
  • y5 and y6 are the clouds.
  • mapping relationship between the first image block and the second image block is: the categories of x1 and y1 are the same, the categories of x2 and y2 are the same, the categories of x3 and y3 are the same, the categories of x4 and y4 are the same, and the categories of y5 and y6 are the same. There are no first image blocks of the same category.
  • the deep learning network model is any one of FCN, SegNet, and DeepLab.
  • the first area is an area in the image with a large angle of view corresponding to the image with a small angle of view
  • the second area is an area in the image with a large angle of view except the first area
  • the target image block is: a second image block of the same type as the first image block and located in the second area, and a sub-image block in the second image block of the same type as the first image block, the sub-image block is located in second area.
  • S3040 Determine the similarity or peak signal-to-noise ratio between each first image block and the target image block.
  • the category of y1 is the same as that of x1, and the sub-image block in y1 located in the second area is y12; the category of y2 is the same as that of x2, but located in the first area, and is not calculated; the category of y3 is the same as that of x3, but located in In the first area, it is no longer calculated; the categories of y4 and x4 are the same, and the sub-image block in the second area in y4 is y42.
  • the above S3040 can be expressed as: determining the similarity or peak signal-to-noise ratio between x1 and y12; determining the similarity or peak signal-to-noise ratio between x4 and y42.
  • the above S3050 can be expressed as: the similarity between x1 and y12 meets the corresponding preset threshold, then determine the image entropy of x1; when the similarity between x4 and y42 meets the corresponding preset threshold, then determine the image entropy corresponding to x4 Image entropy.
  • the process of determining the image entropy of each first image block according to the peak signal-to-noise ratio is similar and will not be repeated here.
  • the explicit addition mode is used to indicate that the texture information is not changed when the texture information is added; the implicit addition mode is used to indicate that the texture information is adaptively deformed when the texture information is added.
  • the target image is: an image with a large field of view added with texture information in an explicit and/or implicit way.
  • the above S3060 to S3080 can be expressed as: the above image entropy of x1 is less than the image entropy threshold, then the texture information extracted from x1 is added to y12 in an explicit way; the above image entropy of x4 is equal to the image entropy threshold, and the texture information extracted from In addition mode, the texture information extracted from x4 is added to y42, thereby obtaining the target image.
  • Embodiment 4 an image processing method, as shown in FIG. 13 , the method includes the following steps S4010 to S4090.
  • the two frames of original images include: one frame of an image with a small field of view and one frame of an image with a large field of view, and the field of view corresponding to the image with a large field of view includes a field of view corresponding to the image with a small field of view.
  • the image with a small field of view is relatively clear, and the image with a large field of view is relatively blurred.
  • the above S4020 can be expressed as: segment the image with small viewing angle according to the color to obtain a plurality of first image blocks corresponding to the image with small viewing angle, and each first image block has a corresponding color.
  • w1 is brown
  • w2 is black
  • w3 is yellow
  • w4 is green, etc.
  • the image with a large viewing angle is segmented according to the color to obtain a plurality of second image blocks corresponding to the image with a large viewing angle, and each second image block has a corresponding color.
  • z1 is green
  • z2 and z3 are brown etc.
  • mapping relationship between the first image block and the second image block is: the colors of w1 and z2 and z3 are the same, the colors of w4 and z1 are the same, and so on.
  • z1 corresponds to the connected domain r1
  • z2 to z9 correspond to the connected domain r2 (the oblique area shown in FIG. 13 ), etc.
  • the first area is an area in the image with a large angle of view corresponding to the image with a small angle of view
  • the second area is an area in the image with a large angle of view except the first area
  • the target image block is: a second image block with the same color as the first image block and located in the second area, and a sub-image block in the second image block with the same color as the first image block, and the sub-image block is located in second area.
  • z2 and z3 having the same color as w1 are located in the second region, then z2 and z3 are target image blocks, and the connected domain corresponding to z2 and z3 is r2.
  • Part of z9 with the same color as w1 is located in the first area (P1 shown in Figure 13), and the other part is located in the second area (P2 shown in Figure 13), pasting the small field of view image to the large field of view
  • the target image block corresponding to w1 is the sub-image block located in the second area in z9. Since the connected domain corresponding to the sub-image block located in the second area in z9 is r2, it is necessary to judge the first image Whether block w1 and connected domain r2 are connected.
  • the explicit addition mode is used to indicate that the texture information is not changed when the texture information is added; the implicit addition mode is used to indicate that the texture information is adaptively deformed when the texture information is added.
  • the above S4070 to S4090 can be expressed as: since the first image block w1 is connected to the connected domain r2, when the image entropy of the above w1 is greater than the image entropy threshold, the texture information extracted from w1 is added to z2 and The image blocks located in the second area such as z3, and the sub-image blocks located in the second area in z9.
  • Embodiment 5 an image processing method, as shown in FIG. 14 , the method includes the following steps S5010 to S5100.
  • the two frames of original images include: one frame of an image with a small field of view and one frame of an image with a large field of view, and the field of view corresponding to the image with a large field of view includes a field of view corresponding to the image with a small field of view.
  • the image with a small field of view is relatively clear, and the image with a large field of view is relatively blurred.
  • the above S5020 can be expressed as: using the deep learning network model to semantically segment the image with a small field of view to obtain a plurality of first image blocks corresponding to the image with a small field of view, and each first image block has a corresponding category.
  • e1 is a tree trunk
  • e2 is a person
  • e3 is a leaf
  • e1 is a tree trunk
  • e2 is a person
  • e3 is a leaf
  • the deep learning network model is used to perform semantic segmentation on the image with a large field of view, and a plurality of second image blocks corresponding to the image with a large field of view are obtained, and each second image block has a corresponding category.
  • f1-f9 are tree trunks, etc.
  • the deep learning network model is any one of FCN, SegNet, and DeepLab.
  • f1 corresponds to the connected domain v1
  • f2-f9 correspond to the connected domain v2 (the oblique area shown in FIG. 14 ), etc.
  • the first area is an area in the image with a large angle of view corresponding to the image with a small angle of view
  • the second area is an area in the image with a large angle of view except the first area
  • the target image block is: a second image block of the same type as the first image block and located in the second area, and a sub-image block in the second image block of the same type as the first image block, the sub-image block is located in second area.
  • f2 and f3 of the same category as e1 are located in the second region, then f2 and f3 are target image blocks, and the connected domain corresponding to f2 and f3 is v2.
  • Part of f9 having the same category as e1 is located in the first area (Q1 as shown in Figure 14), and the other part is located in the second area (Q2 as shown in Figure 14), and the small field of view image is attached to the large area.
  • the target image block corresponding to e1 is the sub-image block located in the second area in f9. Since the connected domain corresponding to the part located in the second area in f9 is v2, it is necessary to judge the first image block Whether e1 and connected domain v2 are connected.
  • the explicit addition mode is used to indicate that the texture information is not changed when the texture information is added; the implicit addition mode is used to indicate that the texture information is adaptively deformed when the texture information is added.
  • the target image is: an image with a large field of view added with texture information in an explicit and/or implicit way.
  • the electronic equipment and the image processing apparatus include corresponding hardware structures or software modules for performing each function, or a combination of both.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software in combination with the units and algorithm steps of each example described in the embodiments disclosed herein. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.
  • the embodiment of the present application can divide the functional modules of the electronic equipment and the image processing device according to the above-mentioned method example, for example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function:
  • FIG. 15 is a schematic structural diagram of an image processing device provided by an embodiment of the present application.
  • the image processing apparatus 200 includes an acquisition module 210 and a processing module 220, and the processing module 220 may include a first processing module and a second processing module.
  • the image processing device can perform the following schemes:
  • the acquiring module 210 is configured to acquire multiple frames of original images.
  • the multiple frames of original images are images captured for the same scene to be photographed, and the multiple frames of original images include: a first field of view image and a second field of view image.
  • the viewing angle corresponding to the image with the large viewing angle includes the viewing angle corresponding to the image with the small viewing angle.
  • the first processing module is configured to segment both the small field of view image and the large field of view image, and respectively obtain at least one first image block corresponding to the small field of view image and at least one second image block corresponding to the large field of view image , at least one first image block has a mapping relationship with at least one second image block.
  • the second processing module is used to extract the texture information in the first image block, and add the extracted texture information to the target image block to obtain the target image, the target image block is: having a mapping relationship with at least one first image block The second image block of .
  • the first processing module is configured to use a deep learning network model to perform semantic segmentation on both the small field of view image and the large field of view image, and obtain at least one first image block corresponding to the small field of view image and the large field of view image respectively. At least one second image block corresponding to the field angle image.
  • Each first image block and each second image block has a corresponding category.
  • the plurality of first image blocks correspond to different categories.
  • At least one second image block When the block is a plurality of second image blocks, the plurality of second image blocks correspond to different categories.
  • mapping relationship is: the categories of M first image blocks and N second image blocks are the same, M and N are respectively integers greater than or equal to 1, and the categories are used to represent the first image block and the second image block. content attribute.
  • the first processing module is configured to segment both the small field of view image and the large field of view image according to color or hue, to obtain at least one first image block corresponding to the small field of view image and the large field of view image respectively At least one second image block corresponding to the image.
  • Each first image block and each second image block has a corresponding color or hue, and when at least one first image block is a plurality of first image blocks, the plurality of first image blocks correspond to different colors or hues, at least When one second image block is multiple second image blocks, the multiple second image blocks correspond to different colors or hues.
  • mapping relationship is: P first image blocks and Q second image blocks have the same color or hue, and P and Q are integers greater than or equal to 1 respectively.
  • the first processing module is further configured to paste the small field of view image to the first area of the large field of view image.
  • the first area is the area corresponding to the small field of view image in the large field of view image
  • the second area is the area in the large field of view image except the first area
  • the target image block is: at least one of the first
  • the image block has a mapping relationship and is located in the second image block of the second area
  • the sub-image block in the second image block having a mapping relationship with at least one first image block, the sub-image block is located in the second area.
  • the first processing module is further configured to determine the connected domain corresponding to the second image block by using the watershed algorithm. In addition, it is also used to determine whether the connected domains corresponding to the at least one first image block and the target image block are connected.
  • the first processing module is further configured to determine the image entropy of at least one first image block.
  • the second processing module is further configured to determine, according to the image entropy of the at least one first image block, how to add the texture information extracted from the at least one first image block to the target image block;
  • the first processing module is further configured to determine a similarity or peak signal-to-noise ratio between at least one first image block and the target image block.
  • the image entropy of at least one first image block is determined.
  • the second processing module is further configured to determine, according to the image entropy of the at least one first image block, how to add the texture information extracted from the at least one first image block to the target image block;
  • the second processing module is further configured to extract the texture information in the at least one first image block when the image entropy of the at least one first image block is less than the image entropy threshold, and explicitly add the The extracted texture information is added to the target image block.
  • the texture information in the at least one first image block is extracted, and the extracted texture information is added to the target image block in an implicit addition manner.
  • the explicit addition mode is used to indicate that the texture information is not changed when the texture information is added; the implicit addition mode is used to indicate that the texture information is adaptively deformed when the texture information is added.
  • the acquisition module 210 in FIG. 15 can be realized by the receiving interface in FIG. 3, and the processing module 220 in FIG. At least one of a processor, a microcontroller, and a neural network processor, which is not limited in this embodiment of the present application.
  • the embodiment of the present application also provides another image processing device, including: a receiving interface and a processor;
  • the receiving interface is used to obtain multiple frames of original images from the electronic device; the multiple frames of original images are images taken for the same scene to be shot, and the multiple frames of original images include: small field of view images and large field of view images, large field of view
  • the viewing angle corresponding to the image includes the viewing angle corresponding to the image with a small viewing angle.
  • the processor is used for invoking the computer program stored in the memory to execute the processing steps in the above-mentioned image processing method 10 .
  • the embodiment of the present application also provides another electronic device, including a camera module, a processor, and a memory.
  • the camera module is used to obtain multiple frames of original images; the multiple frames of original images are images taken for the same scene to be shot, and the multiple frames of original images include: small field of view images and large field of view images, and the large field of view images correspond to The field of view includes the field of view corresponding to the image with a small field of view.
  • Memory which stores computer programs that run on the processor.
  • the processor is configured to execute the processing steps in the above-mentioned image processing method 10 .
  • the camera module includes a first camera and a second camera, the first camera is used to take pictures of the scene to be shot with a first angle of view, and the second camera is used to take pictures of the scene to be shot with a second angle of view;
  • the first viewing angle is smaller than the second viewing angle.
  • the first camera is configured to acquire an image with a small field of view after the processor obtains a photographing instruction; the field of view corresponding to the image with a small field of view is a first field of view.
  • the second camera is configured to acquire a large field of view image after the processor obtains the photographing instruction; the field of view corresponding to the large field of view image is the second field of view.
  • the image is obtained by the image processor in the color camera and the black and white camera.
  • the image sensor may be, for example, a charge-coupled device (charge-coupled device, CCD), a complementary metal oxide semiconductor (complementary metal oxide semiconductor, CMOS) and the like.
  • the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores computer instructions; when the computer-readable storage medium runs on the image processing device, the image processing device executes the above method shown.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be accessed by a computer, or may be a data storage device including one or more servers, data centers, etc.
  • the available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium, or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), etc.
  • the embodiment of the present application also provides a computer program product including computer instructions, which, when run on an image processing device, enables the image processing device to execute the above method.
  • FIG. 16 is a schematic structural diagram of a chip provided by an embodiment of the present application.
  • the chip shown in FIG. 16 may be a general-purpose processor or a special-purpose processor.
  • the chip includes a processor 401 .
  • the processor 401 is used to support the image processing apparatus to execute the above technical solution.
  • the chip further includes a transceiver 402, and the transceiver 402 is configured to be controlled by the processor 401, and configured to support the communication device to execute the above-mentioned technical solution.
  • the chip shown in FIG. 16 may further include: a storage medium 403 .
  • the chip shown in Figure 16 can be implemented using the following circuits or devices: one or more field programmable gate arrays (field programmable gate array, FPGA), programmable logic device (programmable logic device, PLD) , controllers, state machines, gate logic, discrete hardware components, any other suitable circuitry, or any combination of circuitry capable of performing the various functions described throughout this application.
  • field programmable gate array field programmable gate array, FPGA
  • programmable logic device programmable logic device
  • controllers state machines, gate logic, discrete hardware components, any other suitable circuitry, or any combination of circuitry capable of performing the various functions described throughout this application.
  • the electronic equipment, image processing device, computer storage medium, computer program product, and chip provided by the above-mentioned embodiments of the present application are all used to execute the method provided above. Therefore, the beneficial effects that it can achieve can refer to the above-mentioned The beneficial effects corresponding to the method will not be repeated here.
  • sequence numbers of the above processes do not mean the order of execution, and the execution order of each process should be determined by its functions and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.
  • presetting and predefining can be realized by pre-saving corresponding codes, tables or other methods that can be used to indicate related information in devices (for example, including electronic devices) , the present application does not limit its specific implementation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

一种图像处理方法、装置和电子设备，涉及图像技术领域，图像处理方法包括：获取多帧原始图像，多帧原始图像包括：小视场角图像和大视场角图像；对小视场角图像和大视场角图像均进行分割，分别得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块，至少一个第一图像块与至少一个第二图像块具有映射关系；对至少一个第一图像块中的纹理信息进行提取，并将提取的纹理信息添加至目标图像块，得到目标图像，目标图像块为：与至少一个第一图像块具有映射关系的第二图像块。本申请通过将小视场角图像中的纹理信息添加到大视场角图像中，从而可以增强细节，实现提高大视场角图像的清晰度的目的。

Description

图像处理方法、装置和电子设备
本申请要求于2021年06月24日提交国家知识产权局、申请号为202110707978.1、申请名称为“图像处理方法、装置和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请属于图像处理领域，尤其涉及一种图像处理方法、装置和电子设备。
背景技术
随着电子设备的广泛使用，使用电子设备进行拍照已经成为人们生活中的一种日常行为方式。以电子设备为手机为例，现有技术中，为了提高拍照质量，业界提出了在手机上设置双摄像头，利用两个摄像头获取的图像信息之间的差异，进行图像信息的互补，由此来提升拍摄的图像质量。
但是实际上，目前配置有双摄像头的手机在拍摄图像时，只是将两个摄像头获取的图像进行简单的融合，而这种方式无法在各种场景下均拍摄出质量较高的图像。
示例性的，手机配置了两个摄像头，一个是主摄像头，另一个是广角摄像头或者是长焦摄像头。其中，广角摄像头的视场角相对于主摄像头的视场角较大，适合近景拍摄，长焦摄像头的视场角相对于主摄像头的视场角较小，适合远景拍摄。此时，若将主摄像头拍摄的图像和广角摄像头或者和长焦摄像头拍摄的图像进行简单融合，由于两个摄像头的视场角不匹配，将会导致融合得到的图像立体感较差，质量也较差。
例如，采用这种双摄像头的手机得到的两种图像中有视场角重合的部分，也有视场角不重合的部分。如果直接将两张图像进行融合，那么最终拍摄得到的图像中视场角重合的部分清晰度高，不重合的部分清晰度低，使得拍摄得到的图像会出现中心部分和四周部分的清晰度不一致的问题，即图像上会出现融合边界，影响成像效果。
因此，亟待一种新的图像处理方法，来有效提高获取的图像的清晰度。
发明内容
本申请提供一种图像处理方法、装置和电子设备，通过将小视场角图像中的纹理信息添加到大视场角图像中，从而可以增强细节，实现提高大视场角图像的清晰度的目的。
为达到上述目的，本申请采用如下技术方案：
第一方面，提供一种图像处理方法，该方法包括：
获取多帧原始图像，多帧原始图像为对相同的待拍摄场景拍摄的图像，多帧原始图像包括：小视场角图像和大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角；对小视场角图像和大视场角图像均进行分割，分别得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块，至少一个第一图像块与至少一个第二图像块具有映射关系；对至少一个第一图像块中的纹理 信息进行提取，并将提取的纹理信息添加至目标图像块，得到目标图像，目标图像块为：与至少一个第一图像块具有映射关系的第二图像块。
本申请实施例提供一种图像处理方法，通过获取多帧原始图像，对多帧原始图像中的小视场角图像进行分割，得到对应的至少一个第一图像块，同时，对多帧原始图像中的大视场角图像进行分割，得到对应的至少一个第二图像块。由于大视场角图像对应的视场角包括小视场角图像对应的视场角，所以，分割后的至少一个第一图像块与至少一个第二图像块具有映射关系，基于此，再从一个或多个第一图像块中提取纹理信息并添加至与被提取纹理信息的第一图像块具有映射关系的第二图像块中。由于小视场角图像相对于大视场角图像清晰度较高，细节更丰富，因此，将从第一图像块提取的纹理信息添加给第二图像块时，可以增强第二图像块的细节，进而可以起到提高大视场角图像的清晰度和质量的作用。
在第一方面一种可能的实现方式中，对小视场角图像和大视场角图像均进行分割，得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块，包括：利用深度学习网络模型，对小视场角图像和所述大视场角图像均进行语义分割，分别得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块；每个第一图像块和每个第二图像块均具有对应的类别，至少一个第一图像块为多个第一图像块时，多个第一图像块对应不同的类别，至少一个第二图像块为多个第二图像块时，多个第二图像块对应不同的类别；其中，映射关系为：M个第一图像块与N个第二图像块的类别相同，M和N分别为大于或者等于1的整数，类别用于表示第一图像块和第二图像块的内容属性。
在第一方面一种可能的实现方式中，对小视场角图像和大视场角图像均进行分割，得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块，包括：根据颜色或色调，对小视场角图像和大视场角图像均进行分割，分别得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块；每个第一图像块和每个第二图像块均具有对应的颜色或色调，至少一个第一图像块为多个第一图像块时，多个第一图像块对应不同的颜色或色调，至少一个第二图像块为多个第二图像块时，多个第二图像块对应不同的颜色或色调；其中，映射关系为：P个第一图像块和Q个第二图像块的颜色或色调相同，P和Q分别为大于或者等于1的整数。
在第一方面一种可能的实现方式中，在对第一图像块中的纹理信息进行提取，并将提取的纹理信息添加至目标图像块之前，该方法还包括：将小视场角图像贴到大视场角图像中的第一区域；其中，第一区域为大视场角图像中与小视场角图像对应的区域，第二区域为大视场角图像中除了第一区域之外的区域，目标图像块为：与至少一个第一图像块具有映射关系且位于第二区域的第二图像块，以及，与至少一个第一图像块具有映射关系的第二图像块中的子图像块，子图像块位于第二区域。在该实现方式中，小视场角图像相对于大视场角图像的清晰度较高，将小视场角图像贴到大视场角图像中的第一区域时，整体提高了大视场角图像中第一区域的清晰度，因此，后续只需要将第一图像块提取的纹理信息添加位于第二区域的第二图像块以及第二图像块中位于第二区域的子图像块中，从而可以减少计算量，提高处理效率。
在第一方面一种可能的实现方式中，在将小视场角图像贴到大视场角图像的第一区域之前，该方法还包括：利用分水岭算法，确定至少一个第二图像块对应的连通域；在将小视场角图像贴到大视场角图像的第一区域之后，该方法还包括：确定至少一个第一图像块与目标图像块对应的连通域是否连通；若连通，则确定至少一个第一图像块的图像熵；根据至少一个第一图像块的图像熵，确定将从至少一个第一图像块中提取出的纹理信息添加至目标图像块的添加方式；以确定出的添加方式将提取的纹理信息添加至目标图像块。在该实现方式中，可以通过确定第一图像块与目标图像块对应的连通域是否连通，来提高后续添加纹理信息的准确度，将第一图像块中的纹理信息仅添加给相连通的连通域中的目标图像块中。
在第一方面一种可能的实现方式中，当利用深度学习网络模型，对小视场角图像和大视场角图像均进行语义分割时，该方法还包括：确定至少一个第一图像块和目标图像块之间的相似度或峰值信噪比；当相似度或峰值信噪比满足各自对应的预设阈值时，则确定至少一个第一图像块的图像熵；根据至少一个第一图像块的图像熵，确定将从至少一个第一图像块提取的纹理信息添加至目标图像块的添加方式；以确定出的添加方式将提取的纹理信息添加至目标图像块。在该实现方式中，根据相似度或峰值信噪比，可以筛选出关联性更强的第一图像块和目标图像块，在此基础上，通过确定图像熵即可知道第一图像块的细节信息较多还是较少，由此，可以根据不同的图像熵确定出不同的对目标图像块添加纹理信息的添加方式。
在第一方面一种可能的实现方式中，该方法还包括：若非连通，则确定至少一个第一图像块和目标图像块之间的相似度或峰值信噪比；当相似度或峰值信噪比满足各自对应的预设阈值时，则确定至少一个第一图像块的图像熵；根据至少一个第一图像块的图像熵，确定将从至少一个第一图像块提取的纹理信息添加至目标图像块的添加方式；以确定出的添加方式将提取的纹理信息添加至目标图像块。在该实现方式中，虽然第一图像块与目标图像块对应的连通域非连通，但是，第一图像块也有可能与目标图像块是同一种对象，因此，可以进一步判断相似度来确定是否要进行纹理信息的添加。
在第一方面一种可能的实现方式中，根据至少一个第一图像块的图像熵，确定将从至少一个第一图像块提取的纹理信息添加至目标图像块的添加方式，包括：当至少一个第一图像块的图像熵小于图像熵阈值时，将从至少一个第一图像块提取的纹理信息添加至目标图像块的添加方式为显式添加方式；当至少一个第一图像块的图像熵大于或等于图像熵阈值时，将从至少一个第一图像块提取的纹理信息添加至目标图像块的添加方式为隐式添加方式；其中，添加方式包括显式添加方式和隐式添加方式，显式添加方式用于表示在添加纹理信息时对纹理信息不改变，隐式添加方式用于表示在添加纹理信息时对纹理信息进行适应性变形。在该实现方式中，当第一图像块的图像熵小于图像熵阈值时，表明第一图像块中的细节信息较少，此时，可以以显示添加的方式，直接将纹理信息复制并粘贴至目标图像块，计算量少，效率高；而当第一图像块的图像熵大于或者等于图像熵阈值时，表明第一图像块中的细节信息较多，此时，需以隐式添加的方式，将从第一图像块提取的纹理信息进行适应性变形后再添加给目标图像块，使得原来的目标图像块中的内容和纹理信息融合的更加自然服帖，增加图 像的细节，提高视觉效果。
在第一方面一种可能的实现方式中，深度学习网络模型为FCN、SegNet、DeepLab中的任意一种。
第二方面，提供了一种图像处理装置，该装置包括用于执行以上第一方面或第一方面的任意可能的实现方式中各个步骤的单元。
第三方面，提供了一种图像处理装置，包括：接收接口和处理器；接收接口用于从电子设备处获取多帧原始图像；多帧原始图像为对相同的待拍摄场景拍摄的图像，多帧原始图像包括：小视场角图像和大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角；处理器，用于调用存储器中存储的计算机程序，以执行如第一方面或第一方面的任意可能的实现方式中提供的图像处理方法中进行处理的步骤。
第四方面，提供了一种电子设备，包括摄像头模组、处理器和存储器；摄像头模组，用于获取多帧原始图像；多帧原始图像为对相同的待拍摄场景拍摄的图像，多帧原始图像包括：小视场角图像和大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角；存储器，用于存储可在处理器上运行的计算机程序；处理器，用于执行如第一方面或第一方面的任意可能的实现方式中提供的图像处理方法中进行处理的步骤。
在第四方面一种可能的实现方式中，摄像头模组包括第一摄像头和第二摄像头，第一摄像头用于对待拍摄场景以第一视场角进行拍照，第二摄像头用于对待拍摄场景以第二视场角进行拍照；第一视场角小于第二视场角；第一摄像头，用于在处理器获取拍照指令后，获取小视场角图像，小视场角图像对应的视场角为第一视场角；第二摄像头，用于在处理器获取拍照指令后，获取大视场角图像，大视场角图像对应的视场角为第二视场角。
第五方面，提供了一种芯片，包括：处理器，用于从存储器中调用并运行计算机程序，使得安装有芯片的设备执行执行如第一方面或第一方面的任意可能的实现方式中提供的图像处理方法。
第六方面，提供了一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，计算机程序包括程序指令，程序指令当被处理器执行时，使处理器执行如第一方面或第一方面的任意可能的实现方式中提供的图像处理方法。
第七方面，提供了一种计算机程序产品，计算机程序产品包括存储了计算机程序的计算机可读存储介质，计算机程序使得计算机执行如第一方面或第一方面的任意可能的实现方式中提供的图像处理方法。
本申请提供的图像处理方法、装置和电子设备，通过获取多帧原始图像，对多帧原始图像中的小视场角图像进行分割，得到对应的至少一个第一图像块，同时，对多帧原始图像中的大视场角图像进行分割，得到对应的至少一个第二图像块。由于大视场角图像对应的视场角包括小视场角图像对应的视场角，所以，分割后的至少一个第一图像块与至少一个第二图像块具有映射关系，基于此，再从一个或多个第一图像块中提取纹理信息并添加至与被提取纹理信息的第一图像块具有映射关系的第二图像块中。由于小视场角图像相对于大视场角图像清晰度较高，细节更丰富，因此，将从第 一图像块提取的纹理信息添加给第二图像块时，可以增强第二图像块的细节，进而可以起到提高大视场角图像的清晰度和质量的作用。
附图说明
图1为现有技术提供的一种对双摄像头拍摄的图像进行处理的示意图；
图2为本申请实施例提供的一种电子设备的结构示意图；
图3为本申请实施例提供的一种图像处理装置的硬件架构图；
图4为本申请实施例提供的一种图像处理方法的流程示意图；
图5为本申请实施例提供的另一种图像处理方法的流程示意图；
图6为本申请实施例提供的一种纹理信息添加方式的示意图；
图7为本申请实施例提供的又一种图像处理方法的流程示意图；
图8为本申请实施例提供的一种确定第二图像块对应的连通域的示意图；
图9为本申请实施例提供的又一种图像处理方法的流程示意图；
图10为本申请实施例提供的又一种图像处理方法的流程示意图；
图11为本申请实施例提供的又一种图像处理方法的流程示意图；
图12为本申请实施例提供的又一种图像处理方法的流程示意图；
图13为本申请实施例提供的又一种图像处理方法的流程示意图；
图14为本申请实施例提供的又一种图像处理方法的流程示意图；
图15为本申请实施例提供的一种图像处理装置的结构示意图；
图16为申请实施例提供的一种芯片的结构示意图。
具体实施方式
下面将结合附图，对本申请中的技术方案进行描述。
在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，在本申请实施例的描述中，“多个”是指两个或多于两个。
以下，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。
首先，对本申请实施例中的部分用语进行解释说明，以便于本领域技术人员理解。
1、视场角(field of view，FOV)，用于指示摄像头所能拍摄到的最大的角度范围。若待拍摄物体处于这个角度范围内，该待拍摄物体便会被摄像头捕捉到。若待拍摄物体处于这个角度范围之外，该待拍摄物体便不会被摄像头捕捉到。
通常，摄像头的视场角越大，则拍摄范围就越大，焦距就越短；而摄像头的视场角越小，则拍摄范围就越小，焦距就越长。因此，摄像头因视场角的不同可以被划分主摄像头、广角摄像头和长焦摄像头。其中，广角摄像头的视场角相对于主摄像头的视场角较大，焦距较小，适合近景拍摄；而长焦摄像头的视场角相对于主摄像头的视场角较小，焦距较长，适合远景拍摄。
2、连通域，指的是：属于一区域A内任一项简单闭合曲线的内部都属于A，则 称A为连通域，或者，连通域也可以表述为：A内任一封闭曲线所围成的区域内只含有A中的点。
3、分水岭算法(watershed algorithm)，分水岭算法是一种基于拓扑理论的数学形态学的分割方法，也是一种常用的图像分割的方法，其基本思想是把图像看做是测地学上的拓扑地貌，图像中每一点像素的灰度值表示该点的海拔高度，图像中的凹陷地点就是图像中的局部极小值。而汇水盆就是地形中凹陷地点影响的区域，也是图像中包围局部极小值的区域。通过注水，在水面上涨的过程中，这些凹陷的地点被浸没，最终在其周围形成堤坝，这些堤坝用于阻挡其他汇水盆里的水混到自己的盆中。当地形中所有的汇水盆都被堤坝包围后，停止注水，此时，这些堤坝即是分水岭。
以上是对本申请实施例所涉及名词的简单介绍，以下不再赘述。
随着电子设备的广泛使用，使用电子设备进行拍照已经成为人们生活中的一种日常行为方式。以手机为例，现有技术中，为了提高拍照质量，业界提出了在手机上设置双摄像头，利用两个摄像头获取的图像信息之间的差异，进行图像信息的互补，由此来提升拍摄的图像质量。
但是实际上，目前配置有双摄像头的手机在拍摄图像时，只是将两个摄像头获取的图像进行简单的融合，而这种方式无法在各种场景下均拍摄出质量较高的图像。
示例性的，手机配置了两个摄像头，一个是主摄像头，另一个是广角摄像头或者是长焦摄像头，或者，两个摄像头分别为广角摄像头和长焦摄像头。其中，广角摄像头的视场角相对于主摄像头的视场角较大，长焦摄像头的视场角相对于主摄像头的视场角较小。然后，将主摄像头拍摄的图像和广角摄像头拍摄的图像，或者；将主摄像头拍摄的图像和长焦摄像头拍摄的图像进行简单融合，或者；将广角摄像头拍摄的图像和长焦摄像头拍摄的图像进行简单融合。
图1示出了一种现有技术对双摄像头拍摄的图像进行处理的示意图。
如图1所示，在现有技术中，通常会根据视场角大小，将主摄像头拍摄的第一视场角图像填充在广角摄像头拍摄的第二视场角图像中，或者，将长焦摄像头拍摄的第一视场角图像填充在主摄像头或广角摄像头拍摄的第二视场角图像中。但是，在这种方式中，由于两个摄像头的视场角不匹配，将会导致融合得到的图像立体感较差，质量也较差。
例如，采用这种双摄像头的手机得到的两种图像中有视场角重合的部分，也有视场角不重合的部分。如果直接将两张图像进行融合，那么最终拍摄得到的图像中视场角重合的部分与不重合的部分可能对位对不上，部分内容产生断裂或畸形。此外，视场角重合的部分可能清晰度高，不重合的部分清晰度低，使得拍摄得到的图像会出现中心部分和四周部分的清晰度不一致的问题，即图像上会出现融合边界，影响成像效果。
有鉴于此，本申请实施例提供了一种图像处理方法，针对具有视场角重合部分的小视场角图像和大视场角图像，通过将小视场角图像中的纹理信息添加到大视场角图像中，增强细节，以实现提高大视场角图像的清晰度的目的。
本申请实施例提供的图像处理方法可以适用于各种电子设备，对应的，本申请实施例提供的图像处理装置可以为多种形态的电子设备。
在本申请的一些实施例中，该电子设备可以为单反相机、卡片机等各种摄像装置、手机、平板电脑、可穿戴设备、车载设备、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)等，或者可以为其他能够进行图像处理的设备或装置，对于电子设备的具体类型，本申请实施例不作任何限制。
下文以电子设备为手机为例，图2示出了本申请实施例提供的一种电子设备100的结构示意图。
电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
其中，控制器可以是电子设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
处理器110可以运行本申请实施例提供的图像处理方法的软件代码，拍摄得到清晰度较高的图像。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial  bus，USB)接口等。
MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。
USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口，Micro USB接口，USB Type C接口等。USB接口130可以用于连接充电器为电子设备100充电，也可以用于电子设备100与外围设备之间传输数据。也可以用于连接耳机，通过耳机播放音频。该接口还可以用于连接其他电子设备，例如AR设备等。
可以理解的是，本申请实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
充电管理模块140用于从充电器接收充电输入。
电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，内部存储器121，显示屏194，摄像头193，和无线通信模块160等供电。
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术 (infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。
在一些实施例中，电子设备100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications，GSM)，通用分组无线服务(general packet radio service，GPRS)，码分多址接入(code division multiple access，CDMA)，宽带码分多址(wideband code division multiple access，WCDMA)，时分码分多址(time-division code division multiple access，TD-SCDMA)，长期演进(long term evolution，LTE)，BT，GNSS，WLAN，NFC，FM，和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system，GPS)，全球导航卫星系统(global navigation satellite system，GLONASS)，北斗卫星导航系统(beidou navigation satellite system，BDS)，准天顶卫星系统(quasi-zenith satellite system，QZSS)和/或星基增强系统(satellite based augmentation systems，SBAS)。
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。
摄像头193用于捕获图像。可以通过应用程序指令触发开启，实现拍照功能，如拍摄获取任意场景的图像。摄像头可以包括成像镜头、滤光片、图像传感器等部件。物体发出或反射的光线进入成像镜头，通过滤光片，最终汇聚在图像传感器上。图像传感器主要是用于对拍照视角中的所有物体(也可称为待拍摄场景、目标场景，也可以理解为用户期待拍摄的场景图像)发出或反射的光汇聚成像；滤光片主要是用于将光线中的多余光波(例如除可见光外的光波，如红外)滤去；图像传感器主要是用于对接收到的光信号进行光电转换，转换成电信号，并输入处理器130进行后续处理。其中，摄像头193可以位于电子设备100的前面，也可以位于电子设备100的背面，摄像头的具体个数以及排布方式可以根据需求设置，本申请不做任何限制。
示例性的，电子设备100包括前置摄像头和后置摄像头。例如，前置摄像头或者后置摄像头，均可以包括1个或多个摄像头。以电子设备100具有3个后置摄像头为例，这样，电子设备100启动启动3个后置摄像头进行拍摄时，可以使用本申请实施例提供的图像处理方法。或者，摄像头设置于电子设备100的外置配件上，该外置配 件可旋转的连接于手机的边框，该外置配件与电子设备100的显示屏194之间所形成的角度为0-360度之间的任意角度。比如，当电子设备100自拍时，外置配件带动摄像头旋转到朝向用户的位置。当然，手机具有多个摄像头时，也可以只有部分摄像头设置在外置配件上，剩余的摄像头设置在电子设备100本体上，本申请实施例对此不进行任何限制。
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行电子设备100的各种功能应用以及数据处理。
内部存储器121还可以存储本申请实施例提供的图像处理方法的软件代码，当处理器110运行所述软件代码时，执行图像处理方法的流程步骤，得到清晰度较高的图像。
内部存储器121还可以存储拍摄得到的图像。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐等文件保存在外部存储卡中。
当然，本申请实施例提供的图像处理方法的软件代码也可以存储在外部存储器中，处理器110可以通过外部存储器接口120运行所述软件代码，执行图像处理方法的流程步骤，得到清晰度较高的图像。电子设备100拍摄得到的图像也可以存储在外部存储器中。
应理解，用户可以指定将图像存储在内部存储器121还是外部存储器中。比如，电子设备100当前与外部存储器相连接时，若电子设备100拍摄得到1帧图像时，可以弹出提示信息，以提示用户将图像存储在外部存储器还是内部存储器；当然，还可以有其他指定方式，本申请实施例对此不进行任何限制；或者，电子设备100检测到内部存储器121的内存量小于预设量时，可以自动将图像存储在外部存储器中。
电子设备100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。
压力传感器180A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中，压力传感器180A可以设置于显示屏194。
陀螺仪传感器180B可以用于确定电子设备100的运动姿态。在一些实施例中，可以通过陀螺仪传感器180B确定电子设备100围绕三个轴(即，x，y和z轴)的角速度。陀螺仪传感器180B可以用于拍摄防抖。
气压传感器180C用于测量气压。在一些实施例中，电子设备100通过气压传感器180C测得的气压值计算海拔高度，辅助定位和导航。
磁传感器180D包括霍尔传感器。电子设备100可以利用磁传感器180D检测翻盖 皮套的开合。在一些实施例中，当电子设备100是翻盖机时，电子设备100可以根据磁传感器180D检测翻盖的开合。进而根据检测到的皮套的开合状态或翻盖的开合状态，设置翻盖自动解锁等特性。
加速度传感器180E可检测电子设备100在各个方向上(一般为三轴)加速度的大小。当电子设备100静止时可检测出重力的大小及方向。还可以用于识别电子设备姿态，应用于横竖屏切换，计步器等应用。
距离传感器180F，用于测量距离。电子设备100可以通过红外或激光测量距离。在一些实施例中，拍摄场景，电子设备100可以利用距离传感器180F测距以实现快速对焦。
接近光传感器180G可以包括例如发光二极管(LED)和光检测器，例如光电二极管。发光二极管可以是红外发光二极管。电子设备100通过发光二极管向外发射红外光。电子设备100使用光电二极管检测来自附近物体的红外反射光。当检测到充分的反射光时，可以确定电子设备100附近有物体。当检测到不充分的反射光时，电子设备100可以确定电子设备100附近没有物体。电子设备100可以利用接近光传感器180G检测用户手持电子设备100贴近耳朵通话，以便自动熄灭屏幕达到省电的目的。接近光传感器180G也可用于皮套模式，口袋模式自动解锁与锁屏。
环境光传感器180L用于感知环境光亮度。电子设备100可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合，检测电子设备100是否在口袋里，以防误触。
指纹传感器180H用于采集指纹。电子设备100可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。
温度传感器180J用于检测温度。在一些实施例中，电子设备100利用温度传感器180J检测的温度，执行温度处理策略。例如，当温度传感器180J上报的温度超过阈值，电子设备100执行降低位于温度传感器180J附近的处理器的性能，以便降低功耗实施热保护。在另一些实施例中，当温度低于另一阈值时，电子设备100对电池142加热，以避免低温导致电子设备100异常关机。在其他一些实施例中，当温度低于又一阈值时，电子设备100对电池142的输出电压执行升压，以避免低温导致的异常关机。
触摸传感器180K，也称“触控器件”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于电子设备100的表面，与显示屏194所处的位置不同。
骨传导传感器180M可以获取振动信号。在一些实施例中，骨传导传感器180M可以获取人体声部振动骨块的振动信号。骨传导传感器180M也可以接触人体脉搏，接收血压跳动信号。在一些实施例中，骨传导传感器180M也可以设置于耳机中，结合成骨传导耳机。音频模块170可以基于所述骨传导传感器180M获取的声部振动骨块的振动信号，解析出语音信号，实现语音功能。应用处理器可以基于所述骨传导传 感器180M获取的血压跳动信号解析心率信息，实现心率检测功能。
按键190包括开机键，音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入，产生与电子设备100的用户设置以及功能控制有关的键信号输入。
马达191可以产生振动提示。马达191可以用于来电振动提示，也可以用于触摸振动反馈。例如，作用于不同应用(例如拍照，音频播放等)的触摸操作，可以对应不同的振动反馈效果。
指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。
SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195，或从SIM卡接口195拔出，实现和电子设备100的接触和分离。
可以理解的是，本申请实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
本申请实施例提供的图像处理方法，还可以适用于各种图像处理装置。图3示出了本申请实施例提供的一种图像处理装置200的硬件架构图。如图3所示，该图像处理装置200例如可以为处理器芯片。示例性的，图3所示的硬件架构图可以是图2中的处理器110，本申请实施例提供的图像处理方法可以应用在该处理器芯片上。
如图3所示，该图像处理装置200包括：至少一个CPU,存储器、微控制器(microcontroller unit，MCU)、GPU、NPU、内存总线、接收接口和发送接口等。除此之外，该图像处理装置200还可以包括AP、解码器以及专用的图形处理器等。
该图像处理装置200的上述各个部分通过连接器相耦合，示例性的，连接器包括各类接口、传输线或总线等，这些接口通常是电性通信接口，但是，也可能是机械接口或其他形式的接口，本申请实施例对此不做任何限制。
可选地，CPU可以是一个单核(single-CPU)处理器或多核(multi-CPU)处理器。
可选地，CPU可以是多个处理器构成的处理器组，多个处理器之间通过一个或多个总线彼此耦合。该连接接口可以为处理器芯片的数据输入的接口，在一种可选地情况下，该接收接口和发送接口可以是高清晰度多媒体接口(high definition multimedia interface，HDMI)、V-By-One接口、嵌入式显示端口(embedded display port，eDP)、移动产业处理器接口(mobile industry processor interface，MIPI)display port(DP)等，该存储器可以参考上述对内部存储器121部分的描述。在一种可能实现的方式中，上述各部分集成在同一个芯片上。在另一个可能实现的方式中，CPU、GPU、解码器、接收接口以及发送接口集成在一个芯片上，该芯片内部的各部分通过总线访问外部的存储器。专用图形处理器可以为专用ISP。
可选地，NPU也可以作为独立的处理器芯片。该NPU用于实现各种神经网络或者深度学习的相关运算。本申请实施例提供的图像处理方法可以由GPU或NPU实现，也可以由专门的图形处理器来实现。
应理解，在本申请实施例中涉及的芯片是以集成电路工艺制造在同一个半导体衬底上的系统，也叫半导体芯片，其可以是利用集成电路工艺制作在衬底上形成的集成电路的集合，其外层通常被半导体封装材料封装。所述集成电路可以包括各类功能器件，每一类功能器件包括逻辑门电路、金属氧化物半导体(metal oxide semiconductor，MOS)晶体管、二极管等晶体管，也可以包括电容、电阻或电感等其他部件。每个功能器件可以独立工作或者在必要的驱动软件的作用下工作，可以实现通信、运算或存储等各类功能。
下面结合说明书附图，对本申请实施例所提供的图像处理方法进行详细介绍。
图4为本申请实施例所示的一种图像处理方法的流程示意图。如图4所示，该图像处理方法10包括：S10至S30。
S10、获取多帧原始图像。多帧原始图像为对相同的待拍摄场景拍摄的图像。
多帧原始图像包括：小视场角图像和大视场角图像。大视场角图像对应的视场角包括小视场角图像对应的视场角。
该图像处理方法的执行主体可以是上述图2所示的设置有摄像头模组的电子设备100，还可以是上述图3所示的图像处理装置200。当执行主体是电子设备时，通过摄像头模组中的摄像头获取多帧原始图像，具体通过几个摄像头或者通过哪个摄像头获取，可以根据需要进行设置和更改，本申请实施例对此不进行任何限制。当执行主体是图像处理装置时，可以通过接收接口获取多帧原始图像，而该多帧原始图像为与图像处理装置连接的电子设备的摄像头模组所拍摄得到的。
上述原始图像也可称为RAW图。上述原始图像可以为直接拍摄得到的原始图像，也可以为对拍摄得到的视频截取的原始图像。
在获取的多帧原始图像中，小视场角图像和大视场角图像分别可以包括1帧，也可以包括多帧，但至少获取的多帧原始图像包括1帧小视场角图像和1帧大视场角图像。
应理解，小视场角图像和大视场角图像可以是同时拍摄得到的，也可以不是同时拍摄得到的，但至少应为同一时间段内对相同的待拍摄场景拍摄的图像。例如，在2秒内对相同的待拍摄场景进行拍摄，得到多帧小视场角图像和多帧大视场角图像。
可选地，多帧原始图像的尺寸可以全部相同。当然，多帧原始图像的尺寸也可以部分相同，部分不同；也可以完全不相同。本申请实施例对此不进行任何限制。
可选地，多帧原始图像可以是连续获取的，获取的间隔时间可以相同也可以不同。当然，多帧原始图像也可以不是连续获取的，本申请实施例对此不进行任何限制。
应理解，大视场角图像对应的视场角大于小视场角图像对应的视场角，并且，大视场角图像对应的视场角包括小视场角图像对应的视场角，由此，大视场角图像和小视场角图像具有视场角重合部分，该视场角重合部分即为小视场角图像对应的视场角。也就是说，大视场角图像包含小视场角图像中的内容。
还应理解，视场角越大，拍摄得到的图像细节信息越少，越不清晰，因此，大视场角图像相对于小视场角图像捕捉的细节信息较少，清晰度较低。
S20、对小视场角图像和大视场角图像均进行分割，分别得到小视场角图像对应的 至少一个第一图像块和大视场角图像对应的至少一个第二图像块。
其中，至少一个第一图像块与至少一个第二图像块具有映射关系，也即，一个或多个第一图像块与一个或多个第二图像块具有映射关系。
应理解，对大视场角图像和小视场角图像均进行分割，分别得到的第一图像块和第二图像块个数有可能相同也有可能不同。但由于大视场角图像对应的视场角较大，所以，对大视场角图像分割得到的第二图像块的个数应大于或者等于对小视场角图像分割得到的第一图像块的个数，而且，对大视场角图像分割得到的第二图像块应包含至少一个对小视场角图像分割得到的第一图像块。
应理解，第一图像块与第二图像块之间可以有数量上的映射关系，也可以有内容上的映射关系，可以是一一对应的映射关系，也可以是一对多、多对一或多对多的映射关系，具体映射关系由分割方式来决定，而分割方式可以根据需要进行选择和修改，本申请实施例对此不进行任何限制。
可选地，作为一种可能实现的方式，上述S20可以包括：
利用深度学习网络模型，对小视场角图像和大视场角图像均进行语义分割(semantic segmentation)，分别得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块。
其中，每个第一图像块和每个第二图像块均具有对应的类别，对小视场角图像进行语义分割后，当只得到一个第一图像块时，该第一图像块对应一个类别；当得到多个第一图像块时，该多个第一图像块对应不同的类别。
对大视场角图像进行语义分割后，当只得到一个第二图像块时，该第二图像块对应一个类别；当得到多个第二图像块时，该多个第二图像块对应不同的类别。
此处，上述映射关系为：M个第一图像块与N个第二图像块的类别相同，M和N分别为大于或者等于1的整数。
应理解，语义在语音识别中指的是语音的意思，但是，在图像领域中，语义指的是图像的内容，对图像意思的理解。基于此，语义分割指的是：从像素的角度分割出图像中的不同对象。
应理解，类别即为图像中的不同图像块的内容属性，换句话说，类别即为图像中的不同对象分别对应的品类或门类。类别可以根据需要进行预设和更改，本申请实施例对此不进行任何限制。例如，类别可以为“猫”、“狗”等动物，“树”、“花朵”等植物，也可以为“桌子”“汽车”等物体。
例如，一张图像中有3个人，每人骑了一辆自行车，则对该图像进行语义分割时，可以分割出3个人所对应的图像块，每个图像块的类别对应为“人”类别，而分割出的3辆自行车所对应的图像块，每个图像块的类别对应为“自行车”类别。
应理解，该多个第一图像块可以为每个第一图像对应不同的类别，或者，可以为多组第一图像块对应不同的类别，每组第一图像块包含一个或多个第一图像块。该多个第二图像块可以为每个第二图像块对应不同的类别，或者，可以多组第二图像块对应不同的类别，每组第二图像块包含一个或多个第二图像块。
应理解，由于大视场角图像和小视场角图像中都可能包含属于同一类别的多个图像块，所以，在小视场角图像中，可能有多个第一图像块对应的类别相同，同理，在 大视场角图像中，也可能有多个第二图像块对应的类别相同。又由于大视场角图像对应的视场角图像包括小视场角图像对应的视场角，也即，大视场角图像包含小视场角图像的内容，所以，可能存在：M个第一图像块与N个第二图像块的类别相同，M和N分别为大于或者等于1的整数。
例如，该映射关系为：1个第一图像块的类别与多个第二图像块的类别相同，或者；多个第一图像块的类别与1个第二图像块的类别相同，或者；多个第一图像块的类别与多个第二图像块的类别相同。
可选地，此处进行语义分割的深度学习网络模型可以为完全卷积网络(FCN)、SegNet、DeepLab中的任意一种。
当然，深度学习网络模型还可以为其他模型，只要能实现语义分割功能即可，具体可以根据需要进行设置，本申请实施例不进行任何限制。
可选地，作为另一种可能实现的方式，上述S20还可以包括：
根据颜色或色调，对小视场角图像和大视场角图像均进行分割，分别得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块。
其中，每个第一图像块和每个第二图像块均具有对应的颜色或色调，对小视场角图像进行分割后，当只得到一个第一图像块时，该第一图像块对应一个颜色或色调；当得到多个第一图像块时，该多个第一图像块对应不同的颜色或色调。
对大视场角图像进行分割后，当只得到一个第二图像块时，该第二图像块对应一个颜色或色调；当得到多个第二图像块时，该多个第二图像块对应不同的颜色或色调。
此处，上述映射关系为：P个第一图像块和Q个第二图像块的颜色或色调相同，P和Q分别为大于或者等于1的整数。
应理解，该多个第一图像块可以为每个第一图像对应不同的颜色或色调，或者，可以为多组第一图像块对应不同的颜色或色调，每组第一图像块包含一个或多个第一图像块。该多个第二图像块可以为每个第二图像块对应不同的颜色或色调，或者，可以多组第二图像块对应不同的颜色或色调，每组第二图像块包含一个或多个第二图像块。
应理解，由于大视场角图像和小视场角图像中都可能包含属于同一颜色或色调的多个图像块，所以，在小视场角图像中，可能有多个第一图像块对应的颜色或色调相同，同理，在大视场角图像中，也可能有多个第二图像块对应的颜色或色调相同。又由于大视场角图像包含小视场角图像的内容，所以，可能存在：P个第一图像块与Q个第二图像块的颜色相同，或者，P个第一图像块与Q个第二图像块的色调相同，P和Q分别为大于或者等于1的整数。
例如，该映射关系为：1个第一图像块的颜色与多个第二图像块的颜色相同，或者；多个第一图像块的颜色与1个第二图像块的颜色相同，或者；多个第一图像块的颜色与多个第二图像块的颜色相同。
S30、对至少一个第一图像块中纹理信息进行提取，并将提取的纹理信息添加至目标图像块，得到目标图像。
上述S30还可以表述为：对多个第一图像块中至少部分图像块的纹理信息进行提取，并将提取的纹理信息添加至目标图像块，得到目标图像。
其中，目标图像块为：与被提取纹理信息的第一图像块具有映射关系的第二图像块。也就是说，针对任意一个被提取纹理信息的第一图像块，目标图像块为：与被提取纹理信息的第一图像块具有映射关系的1个或多个第二图像块。第二图像块具体数量为多少，决定于分割后第二图像块与第一图像块的映射关系。
此外，目标图像为添加了纹理信息的大视场角图像。
应理解，本申请中的纹理信息指的是物体表面呈现凹凸不平的沟纹，同时也包括在物体的光滑表面上的彩色图案，通常更多地称之为花纹。纹理信息能反应第一图像块中物体的细节。
应理解，由于小视场角图像相对于大视场角图像的清晰度较高，所以，由小视场角图像分割的第一图像块也相对于大视场角图像分割的第二图像块的清晰度较高，由此，可以将第一图像块中的纹理信息提取后，添加至与该被提取纹理信息的第一图像块具有映射关系的第二图像块中，以提高这些第二图像块的清晰度。
还应理解，在从不同的第一图像块中提取出的纹理信息不同的情况下，将从不同的第一图像块中提取出的纹理信息添加至目标图像块时，可以使得不同的第二图像块增加的细节不同，在提高大视场角图像的细节丰富度的同时可以使得大视场角图像更逼真，立体效果更好。
此处，第二图像块除了纹理信息会改变，其他颜色、高动态范围(high dynamic range，HDR)、亮度等信息均保持不变。
本申请实施例提供一种图像处理方法，通过获取多帧原始图像，对多帧原始图像中的小视场角图像进行分割，得到对应的至少一个第一图像块，同时，对多帧原始图像中的大视场角图像进行分割，得到对应的至少一个第二图像块。由于大视场角图像对应的视场角包括小视场角图像对应的视场角，所以，分割后的至少一个第一图像块与至少一个第二图像块具有映射关系，基于此，再从1个或多个第一图像块中提取纹理信息并添加至与被提取纹理信息的第一图像块具有映射关系的第二图像块中。由于小视场角图像相对于大视场角图像清晰度较高，细节更丰富，因此，将从第一图像块提取的纹理信息添加给第二图像块时，可以增强第二图像块的细节，进而可以起到提高大视场角图像的清晰度和质量的作用。
可选地，如图5所示，在上述S20之后，S30之前，该方法10还可以包括S40。
S40、将小视场角图像贴到大视场角图像中的第一区域。
其中，第一区域为大视场角图像中与小视场角图像对应的区域。第二区域为大视场角图像中除了第一区域之外的区域。
应理解，第一区域即为大视场角图像中，与小视场角图像的视场角重合的区域，第二区域即为与小视场角图像的视场角未重合的区域。例如，大视场角图像的中心区域是与小视场角图像视场角重合的区域，则中心区域即为第一区域，环绕第一区域的区域即为第二区域。
此时，在执行S40的情况下，S30中的目标图像块将指示的是：与被提取纹理信息的第一图像块具有映射关系的且位于第二区域的第二图像块，以及，与被提取纹理信息的第一图像块具有映射关系的第二图像块中的子图像块，该子图像块是位于第二区域的。
应理解，小视场角图像相对于大视场角图像的清晰度较高，将小视场角图像贴到大视场角图像中的第一区域时，整体提高了大视场角图像中第一区域的清晰度，因此，后续只需要将第一图像块提取的纹理信息添加至位于第二区域的第二图像块以及第二图像块中位于第二区域的子图像块中，从而可以减少计算量，提高处理效率。
基于此，在添加纹理信息时，虽然，第二图像块与被提取纹理信息的第一图像块具有映射关系，但是，应该排除位于第一区域的第二图像块，以及第二图像块中与第一区域重叠的子图像块，而仅对位于第二区域的第二图像块，以及，对第二图像块中与第二区域重叠的子图像块进行纹理添加。
可选地，当利用深度学习网络模型，对小视场角图像和大视场角图像均进行语义分割时，该方法10还可以包括：S21至S23。
S21、确定至少一个第一图像块和目标图像块之间的相似度或峰值信噪比(peak signal to noise ratio,PSNR)。
其中，可以利用结构相似性(structural similarity，SSIM)算法，确定至少一个第一图像块和目标图像块之间的相似度。
S22、当相似度或峰值信噪比满足各自对应的预设阈值时，确定至少一个第一图像块的图像熵(image entropy)。
应理解，相似度对应的预设阈值、峰值信噪比对应的预设阈值均可以根据需要进行设置和更改，本申请实施例对此不进行任何限制。
应理解，图像熵指的是一种图像特征的统计形式，反映了图像中平均信息量的多少。其中，图像的一维熵指的是图像中灰度分布的聚集特征所包含的信息量。例如，设Pi表示图像中灰度值为i的像素所占的比例，因此，一维熵可以根据以下公式进行计算：
Figure PCTCN2022073190-appb-000001
应理解，根据上述公式计算得到的一维熵即为图像熵。当确定出的第一图像块的图像熵较大时，说明第一图像块的信息量大，细节较多；当确定出的第一图像块的图像熵较小时，说明第一图像块的信息量小，细节较少。由此，可以设定图像熵阈值，第一图像块的图像熵与图像熵阈值来对比，以判断第一图像块的信息量大小。
其中，图像熵阈值的大小可以根据需要进行设定和修改，本申请实施例对此不进行任何限制。
S23、根据至少一个第一图像块的图像熵，确定将从至少一个第一图像块中提取出的纹理信息添加至目标图像块的添加方式。
上述S23还可以表述为：根据被提取纹理信息的第一图像块的图像熵，确定从被提取纹理信息的第一图像块中提取出的纹理信息添加至目标图像块的添加方式。
则相应的，上述S30可以表述为：
对至少一个第一图像块中的纹理信息进行提取，并以确定出的添加方式将提取的纹理信息添加至目标图像块，得到目标图像。
基于此，上述S23进一步可以包括：
当至少一个第一图像块的图像熵小于图像熵阈值时，将从至少一个第一图像块提 取的纹理信息添加至目标图像块的添加方式为显式添加方式。
当至少一个第一图像块的图像熵大于或等于图像熵阈值时，将从至少一个第一图像块提取的纹理信息添加至目标图像块的添加方式为隐式添加方式。
当确定出的添加方式为显式添加方式时，上述S30对应为：以显式添加方式，将提取的纹理信息添加至目标图像块。
当确定出的添加方式为隐式添加方式时，上述S30对应为：以隐式添加方式，将提取的纹理信息添加至目标图像块。
其中，添加方式包括显式添加方式和隐式添加方式。显式添加方式用于指示添加纹理信息时对纹理信息不改变，隐式添加方式用于指示在添加纹理信息时对纹理信息进行适应性变形。
换句话说，显式添加方式指的是：将从第一图像块中提取的纹理信息，复制并粘贴至对应的目标图像块中。若第二图像块面积较大，可以多次将第一图像块中提取的纹理信息，按一定顺序复制并粘贴至对应的目标图像块中，填满目标图像块。
隐式添加方式指的是：将从第一图像块中提取的纹理信息，复制并粘贴至对应的目标图像块中时，根据目标图像块的形状和内容，进行适应性变形，例如：进行旋转、缩放、拉伸等处理。此时，若第二图像块面积较大，可以多次将第一图像块中提取的纹理信息复制并粘贴至对应的目标图像块中，并且，每次粘贴时进行的处理可以不相同，以使得添加至目标图像块中纹理信息不完全一样。
应理解，当第一图像块的图像熵小于图像熵阈值时，表明第一图像块中的细节信息较少，此时，可以以简单的方式，直接将纹理信息复制并粘贴至目标图像块，计算量少，效率高；而当第一图像块的图像熵大于或者等于图像熵阈值时，表明第一图像块中的细节信息较多，此时若进行显式添加，则会使得目标图像块的视觉效果较差，看起来像一模一样的小图像块僵硬的排列在一起，导致内容上时有断裂，造成图像失真等问题，因此，需以隐式添加的方式，将从第一图像块提取的纹理信息进行适应性变形后再添加给目标图像块，使得原来的目标图像块中的内容和纹理信息融合的更加自然服帖，增加图像的细节，提高视觉效果。
例如，如图6中的(a)所示，第一图像块的图像熵小于图像熵阈值，因此，将从第一图像块中提取的纹理信息，直接复制并粘贴至目标图像块中，不进行任何改变，也即，以显式添加方式将纹理信息添加至目标图像块。
如图6中的(b)所示，第一图像块的图像熵大于图像熵阈值，此时，由于目标图像块中的花瓣a朝左绽放，花瓣b朝右绽放，那么，在添加纹理信息时，可以将第一图像块中提取的纹理信息按照花瓣a的位置和形状进行适应性变形后添加至花瓣a中，将纹理信息按照花瓣b的位置和形状进行适应性变形后添加至花瓣b中，也即，以隐式添加方式将纹理信息添加至目标图像块。
可选地，如图7所示，在上述S40之前，该方法10还可以包括S50。
S50、利用分水岭算法，确定至少一个第二图像块对应的连通域。
当然，也可以利用其它算法，只要能确定出大视场角图像中的第二图像块所对应的连通域即可，本申请实施例对此不进行任何限制。
示例性的，如图8中的(a)所示，大视场角图像可以被分割为多个第二图像块(例 如K1～K10)，然后，如图8中的(b)所示，利用分水岭算法，可以确定K1对应连通域L1,K2～K3对应连通域L2，K5对应连通域L3，K6对应连通域L4，K7对应连通域L5，K8对应连通域L7，K9对应连通域L7，K10对应连通域L8。
在上述基础上，如图9所示，当确定出第二图像块对应的连通域后，该方法10还包括S60和S70。
S60、确定至少一个第一图像块与目标图像块对应的连通域是否连通。
应理解，当在该方法10中未包括将小视场角图像贴到大视场角图像上这一步骤时，目标图像块为：与被提取纹理信息的第一图像块具有映射关系的第二图像块，目标图像块对应的连通域即为该具有映射关系的第二图像块对应的连通域。
当在该方法10中包括将小视场角图像贴到大视场角图像上这一步骤时，目标图像块为：与被提取纹理信息的第一图像块具有映射关系且位于第二区域的第二图像块，以及与被提取纹理信息具有映射关系的第二图像块中的子图像块，子图像块位于第二区域。
当目标图像块为：与被提取纹理信息的第一图像块具有映射关系且位于第二区域的第二图像块时，目标图像块对应的连通域即为该第二图像块对应的连通域。
当目标图像块为：与被提取纹理信息具有映射关系的第二图像块中的子图像块，子图像块位于第二区域。目标图像块对应的连通域即为该子图像块对应的连通域。
例如，可以通过确定第一图像块与目标图像块对应的连通域相临近的边界处的像素值是否相同，来判断第一图像块与目标图像块对应的连通域是否连通。
例如，若拍摄小视场角图像和大视场角图像的两个摄像头位置是固定的，则第一图像块与目标图像块的相对位置是固定的，由此，可以通过预先记录坐标，来判断第一图像块与目标图像块对应的连通域是否连通。
例如，还可以先利用尺度不变特征变换(scale-invariant feature transform，SIFT)、SURF、ORB等任意一种算法，对第一图像块和目标图像块进行关键点检测；再计算单应性(homography)矩阵，即，将关键点在世界坐标系中的位置转换为像素坐标系中的位置。然后，即可通过坐标映射获取第一图像块和目标图像块中的关键点在像素坐标系中的相对位置，由此，可以根据该相对位置来判断第一图像块与目标图像块对应的连通域是否连通。
应理解，即使第一图像块与第二图像块对应的类别相同，例如，都是树叶，但是有可能是不同树木的树叶，因此，为了提高准确度，可以进一步判断第一图像块与目标图像块所对应的连通域是否连通，当第一图像块与目标图像块对应的连通域连通时，则为同一种树木的树叶的概率更大。因此，可以通过确定第一图像块与目标图像块对应的连通域是否连通，来提高后续添加纹理信息的准确度，将第一图像块中的纹理信息仅添加给相连通的连通域中的目标图像块中。
S70、若连通，则确定至少一个第一图像块的图像熵。
然后，继续根据至少一个第一图像块的图像熵，确定将从至少一个第一图像块中提取出的纹理信息添加至目标图像块的添加方式，再以确定出的显式添加方式或隐式添加方式将提取的纹理信息添加至目标图像块。
S80、若非连通，则可以利用如上述S21～S23所述的方法，继续确定至少一个第 一图像块与目标图像块之间的相似度或峰值信噪比，根据相似度或峰值信噪比是否满足各自对应的预设阈值，再进一步确定是否需要计算第一图像块的图像熵，当需要确定第一图像块的图像熵时，再根据确定的第一图像块的图像熵的大小，继续判断以显式添加方式或隐式添加方式将纹理信息添加至目标图像块。
结合以上，本申请还提供如下实施例：
实施例1，一种图像处理方法，如图10所示，该方法包括以下S1010至S1030。
S1010、获取2帧原始图像，该2帧原始图像为对相同的待拍摄场景拍摄的图像。
该2帧原始图像包括：1帧小视场角图像和1帧大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角。
应理解，小视场角图像较为清晰，大视场角图像较为模糊。
S1020、利用深度学习网络模型，对大视场角图像和小视场角图像均进行语义分割，分别得到小视场角图像对应的4个第一图像块和大视场角图像对应的6个第二图像块。
上述S1020可以表述为：对小视场角图像利用深度学习网络模型进行语义分割，得到小视场角图像对应的4个第一图像块，每个第一图像块相应具有一个类别。例如：x1为天空，x2为树干，x3为树叶，x4为草地。
对大视场角图像利用深度学习网络模型进行语义分割，得到大视场角图像对应的6个第二图像块，每个第二图像块相应具有一个类别。例如：y1为天空，y2为树干，y3为树叶，y4为草地，y5和y6为云朵。
其中，第一图像块和第二图像块具有的映射关系为：x1和y1的类型相同，x2和y2的类别相同，x3和y3的类别相同，x4和y4的类别相同，而y5和y6没有类别相同的第一图像块。
此处，深度学习网络模型为FCN、SegNet、DeepLab中的任意一种。
S1030、对每个第一图像块中的纹理信息进行提取，并将提取的纹理信息添加至目标图像块，得到目标图像。
目标图像块为：与被提取纹理信息的第一图像块类别相同的1个或多个第二图像块。
目标图像为：添加了纹理信息的大视场角图像。
上述S1030可以表述为：对x1中的纹理信息进行提取，并将提取的纹理信息添加至y1中；对x2中的纹理信息进行提取，并将提取的纹理信息添加至y2中；对x3中的纹理信息进行提取，并将提取的纹理信息添加至y3中；对x4中的纹理信息进行提取，并将提取的纹理信息添加至y4中。
此处，由于y5和y6没有类别相同的第一图像块，所以，对y5和y6不进行添加纹理信息的处理，但是，由于其他第二图像块都进行了纹理信息的添加，所以，大视场角图像整体上还是提高了清晰度和质量。
实施例2，一种图像处理方法，如图11所示，该方法包括以下S2010至S2070。
S2010、获取2帧原始图像，该2帧原始图像为对相同的待拍摄场景拍摄的图像。
该2帧原始图像包括：1帧小视场角图像和1帧大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角。
其中，小视场角图像较为清晰，大视场角图像较为模糊。
S2020、利用深度学习网络模型，对大视场角图像和小视场角图像均进行语义分割，分别得到小视场角图像对应的4个第一图像块和大视场角图像对应的6个第二图像块。
上述S2020可以表述为：对小视场角图像利用深度学习网络模型进行语义分割，得到小视场角图像对应的4个第一图像块，每个第一图像块相应具有一个类别。例如：x1为天空，x2为树干，x3为树叶，x4为草地。
对大视场角图像利用深度学习网络模型进行语义分割，得到大视场角图像对应的6个第二图像块，每个第二图像块相应具有一个类别。例如：y1为天空，y2为树干，y3为树叶，y4为草地，y5和y6为云朵。
其中，第一图像块和第二图像块具有的映射关系为：x1和y1的类型相同，x2和y2的类别相同，x3和y3的类别相同，x4和y4的类别相同，y5和y6没有类别相同的第一图像块。
此处，深度学习网络模型为FCN、SegNet、DeepLab中的任意一种。
S2030、确定每个第一图像块和目标图像块之间的相似度或峰值信噪比。
上述S2030可以表述为：确定x1和y1之间的相似度或峰值信噪比；确定x2和y2之间的相似度或峰值信噪比；确定x3和y3之间的相似度或峰值信噪比；确定x4和y4之间的相似度或峰值信噪比。
S2040、当相似度或峰值信噪比满足各自对应的预设阈值时，则确定对应的第一图像块的图像熵。
上述S2040可以表述为：上述x1和y1之间的相似度满足对应的预设阈值，则确定x1的图像熵；x2和y2之间的相似度不满于预设阈值，则不继续进行处理；x3和y3之间的相似度满足对应的预设阈值，则确定x3对应的图像熵，x4和y4之间的相似度也满足对应的预设阈值，则确定x4对应的图像熵。根据峰值信噪比确定每个第一图像块的图像熵的过程类似，在此不再赘述。
当相似度或峰值信噪比不满足各自对应的预设阈值时，说明不相似，则不用确定第一图像块的图像熵，结束流程。
S2050、当第一图像块的图像熵小于图像熵阈值时，对该第一图像块中的纹理信息进行提取，并以显式添加方式，将提取的纹理信息添加至目标图像块。
S2060、当第一图像块的图像熵大于或者等于图像熵阈值时，对该第一图像块中的纹理信息进行提取，并以隐式添加方式，将提取的纹理信息添加至目标图像块。
其中，显式添加方式用于指示在添加纹理信息时对纹理信息不改变；隐式添加方式用于指示在添加所述纹理信息时对纹理信息进行适应性变形。
S2070、根据S2050和S2060，得到目标图像。
目标图像为：以显式添加方式和/或隐式添加方式添加了纹理信息的大视场角图像。
上述2050至S2070可以表述为：上述x1的图像熵小于图像熵阈值，则以显式添加方式，将从x1提取的纹理信息添加至y1；上述x3对应的图像熵大于图像熵阈值，则以隐式添加方式，将从x3提取的纹理信息添加至y3；上述x4的图像熵等于图像熵 阈值，也以隐式添加方式，将从x4提取的纹理信息添加至y4，由此，得到目标图像。
实施例3，一种图像处理方法，如图12所示，该方法包括以下S3010至S3070。
S3010、获取2帧原始图像，该2帧原始图像为对相同的待拍摄场景拍摄的图像。
该2帧原始图像包括：1帧小视场角图像和1帧大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角。
其中，小视场角图像较为清晰，大视场角图像较为模糊。
S3020、利用深度学习网络模型，对大视场角图像和小视场角图像均进行语义分割，分别得到小视场角图像对应的4个第一图像块和大视场角图像对应的6个第二图像块。
上述S3020可以表述为：对小视场角图像利用深度学习网络模型进行语义分割，得到小视场角图像对应的4个第一图像块，每个第一图像块相应具有一个类别。例如：x1为天空，x2为树干，x3为树叶，x4为草地。
对大视场角图像利用深度学习网络模型进行语义分割，得到大视场角图像对应的6个第二图像块，每个第二图像块相应具有一个类别。例如：y1为天空，y2为树干，y3为树叶，y4为草地，y5和y6为云朵。
其中，第一图像块和第二图像块之间具有的映射关系为：x1和y1的类别相同，x2和y2的类别相同，x3和y3的类别相同，x4和y4的类别相同，y5和y6没有类别相同的第一图像块。
此处，深度学习网络模型为FCN、SegNet、DeepLab中的任意一种。
S3030、结合图12，将小视场角图像贴到大视场角图像中的第一区域。
其中，第一区域为大视场角图像中与小视场角图像对应的区域，第二区域为大视场角图像中除了第一区域之外的区域。
此时，目标图像块为：与第一图像块类别相同且位于第二区域的第二图像块，以及，与第一图像块类别相同的第二图像块中的子图像块，子图像块位于第二区域。
S3040、确定每个第一图像块和目标图像块之间的相似度或峰值信噪比。
应理解，y1与x1的类别相同，y1中位于第二区域的子图像块为y12；y2与x2的类别相同，但位于第一区域中，不再计算；y3与x3的类别相同，但位于第一区域中，也不再计算；y4和x4的类别相同，y4中位于第二区域的子图像块为y42。
由此，上述S3040可以表述为：确定x1和y12之间的相似度或峰值信噪比；确定x4和y42之间的相似度或峰值信噪比。
S3050、当相似度或峰值信噪比满足各自对应的预设阈值时，则确定第一图像块的图像熵。
上述S3050可以表述为：上述x1和y12之间的相似度满足对应的预设阈值，则确定x1的图像熵；x4和y42之间的相似度满足对应的预设阈值时，则确定x4对应的图像熵。根据峰值信噪比确定每个第一图像块的图像熵的过程类似，在此不再赘述。
当相似度或峰值信噪比不满足各自对应的预设阈值时，说明不相似，则不用确定第一图像块的图像熵，结束流程。
S3060、当第一图像块的图像熵小于图像熵阈值时，对该第一图像块中的纹理信息进行提取，并以显式添加方式，将提取的纹理信息添加至目标图像块。
S3070、当第一图像块的图像熵大于或者等于图像熵阈值时，对该第一图像块中的纹理信息进行提取，并以隐式添加方式，将提取的纹理信息添加至目标图像块。
其中，显式添加方式用于指示在添加纹理信息时对纹理信息不改变；隐式添加方式用于指示在添加所述纹理信息时对纹理信息进行适应性变形。
S3080、根据S3060和S3070，得到目标图像。
目标图像为：以显式添加方式和/或隐式添加方式添加了纹理信息的大视场角图像。
上述S3060至S3080可以表述为：上述x1的图像熵小于图像熵阈值，则以显式添加方式，将从x1提取的纹理信息添加至y12；上述x4的图像熵等于图像熵阈值，也以隐式添加方式，将从x4提取的纹理信息添加至y42，由此，得到目标图像。
实施例4，一种图像处理方法，如图13所示，该方法包括以下S4010至S4090。
S4010、获取2帧原始图像，该2帧原始图像为对相同的待拍摄场景拍摄的图像。
该2帧原始图像包括：1帧小视场角图像和1帧大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角。
其中，小视场角图像较为清晰，大视场角图像较为模糊。
S4020、根据颜色或色调，对大视场角图像和小视场角图像均进行分割，分别得到小视场角图像对应的多个第一图像块和大视场角图像对应的多个第二图像块。
上述S4020可以表述为：根据颜色对小视场角图像进行分割，得到小视场角图像对应的多个第一图像块，每个第一图像块相应具有一个颜色。例如：w1为棕色、w2为黑色、w3为黄色、w4为绿色等。
根据颜色对大视场角图像进行分割，得到大视场角图像对应的多个第二图像块，每个第二图像块相应具有一个颜色。例如：z1为绿色，z2和z3为棕色等。
其中，第一图像块与第二图像块之间具有的映射关系为：w1与z2、z3的颜色相同，w4与z1的颜色相同等。
S4030、利用分水岭算法，确定每个第二图像块对应的连通域。
结合图13，例如：z1对应连通域r1，z2～z9对应连通域r2(如图13中所示的斜线区域)等。
S4040、将小视场角图像贴到大视场角图像中的第一区域。
其中，第一区域为大视场角图像中与小视场角图像对应的区域，第二区域为大视场角图像中除了第一区域之外的区域。
此时，目标图像块为：与第一图像块颜色相同且位于第二区域的第二图像块，以及，与第一图像块颜色相同的第二图像块中的子图像块，子图像块位于第二区域。
S4050、确定每个第一图像块与目标图像块对应的连通域是否连通。
例如，与w1具有相同颜色的z2和z3等均位于第二区域，则z2和z3等为目标图像块，z2和z3等对应的连通域为r2。
与w1具有相同颜色的z9有一部分位于第一区域(如图13中所示的P1)，另一部分位于第二区域(如图13中所示的P2)，将小视场角图像贴到大视场角图像之后，w1对应的目标图像块则为z9中位于第二区域的子图像块，由于z9中位于第二区域的子图像块对应的连通域为r2，由此，需判断第一图像块w1和连通域r2是否连通。
S4060、若连通，则确定第一图像块的图像熵。若非连通，则结束流程。
确定是否连通的步骤与上述S60中的描述相同，在此不再赘述。
S4070、当第一图像块的图像熵小于图像熵阈值时，对第一图像块中的纹理信息进行提取，并以显式添加方式，将提取的纹理信息添加至目标图像块。
S4080、当第一图像块的图像熵大于或者等于图像熵阈值时，对第一图像块中的纹理信息进行提取，并以隐式添加方式，将提取的纹理信息添加至目标图像块。
其中，显式添加方式用于指示在添加纹理信息时对纹理信息不改变；隐式添加方式用于指示在添加所述纹理信息时对纹理信息进行适应性变形。
S4090、根据S4070和S4080，得到目标图像。
上述S4070至S4090可以表述为：由于第一图像块w1和连通域r2连通，当上述w1的图像熵大于图像熵阈值时，则以隐式添加方式，将从w1提取的纹理信息添加至z2和z3等位于第二区域的图像块，以及z9中位于第二区域的子图像块。
实施例5，一种图像处理方法，如图14所示，该方法包括以下S5010至S5100。
S5010、获取2帧原始图像，该2帧原始图像为对相同的待拍摄场景拍摄的图像。
该2帧原始图像包括：1帧小视场角图像和1帧大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角。
其中，小视场角图像较为清晰，大视场角图像较为模糊。
S5020、利用深度学习网络模型，对大视场角图像和小视场角图像均进行语义分割，分别得到小视场角图像对应的多个第一图像块和大视场角图像对应的多个第二图像块。
上述S5020可以表述为：利用深度学习网络模型对小视场角图像进行语义分割，得到小视场角图像对应的多个第一图像块，每个第一图像块相应具有一个类别。例如：e1为树干、e2为人、e3为树叶等。
利用深度学习网络模型对大视场角图像进行语义分割，得到大视场角图像对应的多个第二图像块，每个第二图像块相应具有一个类别。例如：f1～f9均为树干等。
其中，e1和f1～f9的类别都相同。
此处，深度学习网络模型为FCN、SegNet、DeepLab中的任意一种。
S5030、利用分水岭算法，确定每个第二图像块对应的连通域。
结合图14，例如：f1对应连通域v1，f2～f9对应连通域v2(如图14中所示的斜线区域)等。
S5040、将小视场角图像贴到大视场角图像中的第一区域。
其中，第一区域为大视场角图像中与小视场角图像对应的区域，第二区域为大视场角图像中除了第一区域之外的区域。
此时，目标图像块为：与第一图像块类别相同且位于第二区域的第二图像块，以及，与第一图像块类别相同的第二图像块中的子图像块，子图像块位于第二区域。
S5050、确定每个第一图像块与目标图像块对应的连通域是否连通。
例如，与e1具有相同类别的f2和f3等均位于第二区域，则f2和f3等为目标图像块，f2和f3等对应的连通域为v2。
与e1具有相同的类别的f9有一部分位于第一区域(如图14中所示的Q1)，另 一部分位于第二区域(如图14中所示的Q2)，将小视场角图像贴到大视场角图像之后，e1对应的目标图像块则为f9中位于第二区域的子图像块，由于f9中位于第二区域的部分对应的连通域为v2，由此，需判断第一图像块e1和连通域v2是否连通。
S5060、若连通，则确定第一图像块的图像熵。
确定是否连通的步骤与上述S60中的描述相同，在此不再赘述。
S5070、当第一图像块的图像熵小于图像熵阈值时，对第一图像块中的纹理信息进行提取，并以显式添加方式，将提取的纹理信息添加至目标图像块。
S5080、当第一图像块的图像熵大于或者等于图像熵阈值时，对第一图像块中的纹理信息进行提取，并以隐式添加方式，将提取的纹理信息添加至目标图像块。
其中，显式添加方式用于指示在添加纹理信息时对纹理信息不改变；隐式添加方式用于指示在添加所述纹理信息时对纹理信息进行适应性变形。
S5090、若非连通，则利用继续确定第一图像块和目标图像块之间的相似度或峰值信噪比，当相似度或峰值信噪比满足对应的预设阈值时，则确定第一图像块的图像熵，再以S5070至S5080的方式进行纹理添加；若相似度或峰值信噪比不满足对应的预设阈值时，则结束流程。
S5100、根据S5070至S5090，得到目标图像。
目标图像为：以显式添加方式和/或隐式添加方式添加了纹理信息的大视场角图像。
上述主要从电子设备或图像处理装置的角度对本申请实施例提供的方案进行了介绍。可以理解的是，电子设备和图像处理装置，为了实现上述功能，其包含了执行每一个功能相应的硬件结构或软件模块，或两者结合。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对电子设备和图像处理装置进行功能模块的划分，例如，可以对应每一个功能划分每一个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。下面以采用对应每一个功能划分每一个功能模块为例进行说明：
图15为本申请实施例提供的一种图像处理装置的结构示意图。如图15所示，该图像处理装置200包括获取模块210和处理模块220，处理模块220可以包括第一处理模块和第二处理模块。
该图像处理装置可以执行以下方案：
获取模块210，用于获取多帧原始图像。多帧原始图像为对相同的待拍摄场景拍摄的图像，多帧原始图像包括：第一视场角图像和第二视场角图像。大视场角图像对应的视场角包括小视场角图像对应的视场角。
第一处理模块，用于对小视场角图像和大视场角图像均进行分割，分别得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块，至少一个第一图像块与至少一个第二图像块具有映射关系。
第二处理模块，用于对第一图像块中的纹理信息进行提取，并将提取的纹理信息添加至目标图像块，得到目标图像，目标图像块为：与至少一个第一图像块具有映射关系的第二图像块。
可选地，第一处理模块，用于利用深度学习网络模型，对小视场角图像和大视场角图像均进行语义分割，分别得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块。
每个第一图像块和每个第二图像块均具有对应的类别，至少一个第一图像块为多个第一图像块时，多个第一图像块对应不同的类别，至少一个第二图像块为多个第二图像块时，多个第二图像块对应不同的类别。
其中，该映射关系为：M个第一图像块与N个第二图像块的类别相同，M和N分别为大于或者等于1的整数，类别用于表示第一图像块和第二图像块的内容属性。
可选地，第一处理模块，用于根据颜色或色调，对小视场角图像和大视场角图像均进行分割，分别得到小视场角图像对应的至少一个第一图像块和大视场角图像对应的至少一个第二图像块。
每个第一图像块和每个第二图像块均具有对应的颜色或色调，至少一个第一图像块为多个第一图像块时，多个第一图像块对应不同的颜色或色调，至少一个第二图像块为多个第二图像块时，多个第二图像块对应不同的颜色或色调。
其中，该映射关系为：P个第一图像块和Q个第二图像块的颜色或色调相同，P和Q分别为大于或者等于1的整数。
可选地，第一处理模块，还用于将小视场角图像贴到大视场角图像中的第一区域。
其中，第一区域为大视场角图像中与小视场角图像对应的区域，第二区域为大视场角图像中除了第一区域之外的区域，目标图像块为：与至少一个第一图像块具有映射关系且位于第二区域的所述第二图像块，以及，与至少一个第一图像块具有映射关系的第二图像块中的子图像块，子图像块位于第二区域。
可选地，第一处理模块，还用于利用分水岭算法，确定第二图像块对应的连通域。此外，还用于确定至少一个第一图像块与目标图像块对应的连通域是否连通。
若连通，则第一处理模块，还用于确定至少一个第一图像块的图像熵。
第二处理模块，还用于根据至少一个第一图像块的图像熵，确定将从至少一个第一图像块中提取出的纹理信息添加至目标图像块的添加方式；
并以确定出的添加方式将提取的纹理信息添加至目标图像块。
可选地，第一处理模块，还用于确定至少一个第一图像块和目标图像块之间的相似度或峰值信噪比。
当相似度或峰值信噪比满足对应的预设阈值时，则确定至少一个第一图像块的图像熵。
第二处理模块，还用于根据至少一个第一图像块的图像熵，确定将从至少一个第一图像块中提取出的纹理信息添加至目标图像块的添加方式；
并以确定出的添加方式将提取的纹理信息添加至目标图像块。
可选地，第二处理模块，还用于当至少一个第一图像块的图像熵小于图像熵阈值时，对至少一个第一图像块中的纹理信息进行提取，并以显式添加方式，将提取的纹理信息添加至目标图像块。
当至少一个第一图像块的图像熵大于或等于图像熵阈值时，对至少一个第一图像块中的纹理信息进行提取，并以隐式添加方式，将提取的纹理信息添加至目标图像块。
其中，显式添加方式用于指示在添加纹理信息时对纹理信息不改变；隐式添加方式用于指示在添加纹理信息时对纹理信息进行适应性变形。
作为一个示例，结合图3所示的图像处理装置，图15中的获取模块210可以由图3中的接收接口来实现，图15中的处理模块220可以由图3中的中央处理器、图形处理器、微控制器和神经网络处理器中的至少一项来实现，本申请实施例对此不进行任何限制。
本申请实施例还提供另一种图像处理装置，包括：接收接口和处理器；
接收接口用于从电子设备处获取多帧原始图像；多帧原始图像为对相同的待拍摄场景拍摄的图像，多帧原始图像包括：小视场角图像和大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角。
处理器，用于调用存储器中存储的计算机程序，以执行如上述所述的图像处理方法10中进行处理的步骤。
本申请实施例还提供另一种电子设备，包括摄像头模组、处理器和存储器。
摄像头模组，用于获取多帧原始图像；多帧原始图像为对相同的待拍摄场景拍摄的图像，多帧原始图像包括：小视场角图像和大视场角图像，大视场角图像对应的视场角包括小视场角图像对应的视场角。
存储器，用于存储可在处理器上运行的计算机程序。
处理器，用于执行如上述所述的图像处理方法10中进行处理的步骤。
可选地，摄像头模组包括第一摄像头和第二摄像头，第一摄像头用于对待拍摄场景以第一视场角进行拍照，第二摄像头用于对待拍摄场景以第二视场角进行拍照；第一视场角小于第二视场角。
第一摄像头，用于在处理器获取拍照指令后，获取小视场角图像；小视场角图像对应的视场角为第一视场角。
第二摄像头，用于在处理器获取所述拍照指令后，获取大视场角图像；大视场角图像对应的视场角为第二视场角。
严格来说，是通过彩色摄像头和黑白摄像头中的图像处理器来获取图像。其中，图像传感器例如可以为电荷耦合元件(charge-coupled device，CCD)、互补金属氧化物半导体(complementary metal oxide semiconductor，CMOS)等。
本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机指令；当所述计算机可读存储介质在图像处理装置上运行时，使得该图像处理装置执行如上所示的方法。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机 指令可以从一个网站站点、计算机、服务器或者数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可以用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)，光介质、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
本申请实施例还提供了一种包含计算机指令的计算机程序产品，当其在图像处理装置上运行时，使得图像处理装置可以执行如上所示的方法。
图16为本申请实施例提供的一种芯片的结构示意图。图16所示的芯片可以为通用处理器，也可以为专用处理器。该芯片包括处理器401。其中，处理器401用于支持图像处理装置执行如上所示的技术方案。
可选的，该芯片还包括收发器402，收发器402用于接受处理器401的控制，用于支持通信装置执行如上所示的技术方案。
可选的，图16所示的芯片还可以包括：存储介质403。
需要说明的是，图16所示的芯片可以使用下述电路或者器件来实现：一个或多个现场可编程门阵列(field programmable gate array，FPGA)、可编程逻辑器件(programmable logic device，PLD)、控制器、状态机、门逻辑、分立硬件部件、任何其他适合的电路、或者能够执行本申请通篇所描述的各种功能的电路的任意组合。
上述本申请实施例提供的电子设备、图像处理装置、计算机存储介质、计算机程序产品、芯片均用于执行上文所提供的方法，因此，其所能达到的有益效果可参考上文所提供的方法对应的有益效果，在此不再赘述。
应理解，上述只是为了帮助本领域技术人员更好地理解本申请实施例，而非要限制本申请实施例的范围。本领域技术人员根据所给出的上述示例，显然可以进行各种等价的修改或变化，例如，上述检测方法的各个实施例中某些步骤可以是不必须的，或者可以新加入某些步骤等。或者上述任意两种或者任意多种实施例的组合。这样的修改、变化或者组合后的方案也落入本申请实施例的范围内。
还应理解，上文对本申请实施例的描述着重于强调各个实施例之间的不同之处，未提到的相同或相似之处可以互相参考，为了简洁，这里不再赘述。
还应理解，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
还应理解，本申请实施例中，“预先设定”、“预先定义”可以通过在设备(例如，包括电子设备)中预先保存相应的代码、表格或其他可用于指示相关信息的方式来实现，本申请对于其具体的实现方式不做限定。
还应理解，本申请实施例中的方式、情况、类别以及实施例的划分仅是为了描述的方便，不应构成特别的限定，各种方式、类别、情况以及实施例中的特征在不矛盾的情况下可以相结合。
还应理解，在本申请的各个实施例中，如果没有特殊说明以及逻辑冲突，不同的实施例之间的术语和/或描述具有一致性、且可以相互引用，不同的实施例中的技术特 征根据其内在的逻辑关系可以组合形成新的实施例。
最后应说明的是：以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (14)

  1. 一种图像处理方法，其特征在于，所述方法包括：
    获取多帧原始图像，所述多帧原始图像为对相同的待拍摄场景拍摄的图像，所述多帧原始图像包括：小视场角图像和大视场角图像，所述大视场角图像对应的视场角包括小视场角图像对应的视场角；
    对所述小视场角图像和所述大视场角图像均进行分割，分别得到所述小视场角图像对应的至少一个第一图像块和所述大视场角图像对应的至少一个第二图像块，所述至少一个第一图像块与所述至少一个第二图像块具有映射关系；
    对所述至少一个第一图像块中的纹理信息进行提取，并将提取的所述纹理信息添加至目标图像块，得到目标图像，所述目标图像块为：与所述至少一个第一图像块具有映射关系的所述第二图像块。
  2. 根据权利要求1所述的方法，其特征在于，对所述小视场角图像和所述大视场角图像均进行分割，得到所述小视场角图像对应的至少一个第一图像块和所述大视场角图像对应的至少一个第二图像块，包括：
    利用深度学习网络模型，对所述小视场角图像和所述大视场角图像均进行语义分割，分别得到所述小视场角图像对应的所述至少一个第一图像块和所述大视场角图像对应的所述至少一个第二图像块；
    每个第一图像块和每个第二图像块均具有对应的类别，所述至少一个第一图像块为多个第一图像块时，所述多个第一图像块对应不同的类别，所述至少一个第二图像块为多个第二图像块时，所述多个第二图像块对应不同的类别；
    其中，所述映射关系为：M个所述第一图像块与N个所述第二图像块的类别相同，M和N分别为大于或者等于1的整数，所述类别用于表示所述第一图像块和所述第二图像块的内容属性。
  3. 根据权利要求1所述的方法，其特征在于，对所述小视场角图像和所述大视场角图像均进行分割，得到所述小视场角图像对应的至少一个第一图像块和所述大视场角图像对应的至少一个第二图像块，包括：
    根据颜色或色调，对所述小视场角图像和所述大视场角图像均进行分割，分别得到所述小视场角图像对应的所述至少一个第一图像块和所述大视场角图像对应的所述至少一个第二图像块；
    每个第一图像块和每个第二图像块均具有对应的颜色或色调，所述至少一个第一图像块为多个第一图像块时，所述多个第一图像块对应不同的颜色或色调，所述至少一个第二图像块为多个第二图像块时，所述多个第二图像块对应不同的颜色或色调；
    其中，所述映射关系为：P个所述第一图像块和Q个所述第二图像块的颜色或色调相同，P和Q分别为大于或者等于1的整数。
  4. 根据权利要求1至3中任一项所述的方法，其特征在于，在对所述第一图像块中的纹理信息进行提取，并将提取的所述纹理信息添加至目标图像块之前，所述方法还包括：
    将所述小视场角图像贴到所述大视场角图像中的第一区域；
    其中，所述第一区域为所述大视场角图像中与所述小视场角图像对应的区域，第 二区域为所述大视场角图像中除了所述第一区域之外的区域，所述目标图像块为：与所述至少一个第一图像块具有映射关系且位于所述第二区域的所述第二图像块，以及，与所述至少一个第一图像块具有映射关系的所述第二图像块中的子图像块，所述子图像块位于所述第二区域。
  5. 根据权利要求4所述的方法，其特征在于，在将所述小视场角图像贴到所述大视场角图像的第一区域之前，所述方法还包括：
    利用分水岭算法，确定所述至少一个第二图像块对应的连通域；
    在将所述小视场角图像贴到所述大视场角图像的第一区域之后，所述方法还包括：
    确定所述至少一个第一图像块与所述目标图像块对应的连通域是否连通；
    若连通，则确定所述至少一个第一图像块的图像熵；
    根据所述至少一个第一图像块的图像熵，确定将从所述至少一个第一图像块中提取出的纹理信息添加至所述目标图像块的添加方式；
    以确定出的添加方式将提取的所述纹理信息添加至所述目标图像块。
  6. 根据权利要求2或4所述的方法，其特征在于，当利用深度学习网络模型，对所述小视场角图像和所述大视场角图像均进行语义分割时，所述方法还包括：
    确定所述至少一个第一图像块和所述目标图像块之间的相似度或峰值信噪比；
    当所述相似度或所述峰值信噪比满足各自对应的预设阈值时，则确定所述至少一个第一图像块的图像熵；
    根据所述至少一个第一图像块的图像熵，确定将从所述至少一个第一图像块提取的纹理信息添加至所述目标图像块的添加方式；
    以确定出的添加方式将提取的所述纹理信息添加至所述目标图像块。
  7. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    若非连通，则确定所述至少一个第一图像块和所述目标图像块之间的相似度或峰值信噪比；
    当所述相似度或所述峰值信噪比满足各自对应的预设阈值时，则确定所述至少一个第一图像块的图像熵；
    根据所述至少一个第一图像块的图像熵，确定将从所述至少一个第一图像块提取的纹理信息添加至所述目标图像块的添加方式；
    以确定出的添加方式将提取的所述纹理信息添加至所述目标图像块。
  8. 根据权利要求5至7中任一项所述的方法，其特征在于，根据所述至少一个第一图像块的图像熵，确定将从所述至少一个第一图像块提取的纹理信息添加至所述目标图像块的添加方式，包括：
    当所述至少一个第一图像块的图像熵小于所述图像熵阈值时，将从所述至少一个第一图像块提取的纹理信息添加至所述目标图像块的添加方式为显式添加方式；
    当所述至少一个第一图像块的图像熵大于或等于图像熵阈值时，将从所述至少一个第一图像块提取的纹理信息添加至所述目标图像块的添加方式为隐式添加方式；
    其中，所述添加方式包括所述显式添加方式和所述隐式添加方式，所述显式添加方式用于表示在添加所述纹理信息时对所述纹理信息不改变，所述隐式添加方式用于表示在添加所述纹理信息时对所述纹理信息进行适应性变形。
  9. 根据权利要求2所述的方法，其特征在于，所述深度学习网络模型为FCN、SegNet、DeepLab中的任意一种。
  10. 一种图像处理装置，其特征在于，包括：接收接口和处理器；
    所述接收接口用于从电子设备处获取多帧原始图像；所述多帧原始图像为对相同的待拍摄场景拍摄的图像，所述多帧原始图像包括：小视场角图像和大视场角图像，所述大视场角图像对应的视场角包括小视场角图像对应的视场角；
    所述处理器，用于调用存储器中存储的计算机程序，以执行如权利要求1至9中任一项所述的图像处理方法中进行处理的步骤。
  11. 一种电子设备，其特征在于，包括摄像头模组、处理器和存储器；
    所述摄像头模组，用于获取多帧原始图像；所述多帧原始图像为对相同的待拍摄场景拍摄的图像，所述多帧原始图像包括：小视场角图像和大视场角图像，所述大视场角图像对应的视场角包括小视场角图像对应的视场角；
    所述存储器，用于存储可在所述处理器上运行的计算机程序；
    所述处理器，用于执行如权利要求1至9中任一项所述的图像处理方法中进行处理的步骤。
  12. 根据权利要求11所述的电子设备，其特征在于，所述摄像头模组包括第一摄像头和第二摄像头，所述第一摄像头用于对待拍摄场景以第一视场角进行拍照，所述第二摄像头用于对所述待拍摄场景以第二视场角进行拍照；所述第一视场角小于所述第二视场角；
    所述第一摄像头，用于在所述处理器获取拍照指令后，获取所述小视场角图像，所述小视场角图像对应的视场角为所述第一视场角；
    所述第二摄像头，用于在所述处理器获取所述拍照指令后，获取所述大视场角图像，所述大视场角图像对应的视场角为所述第二视场角。
  13. 一种芯片，其特征在于，包括：处理器，用于从存储器中调用并运行计算机程序，使得安装有所述芯片的设备执行如权利要求1至9中任一项所述的图像处理方法。
  14. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行时，使所述处理器执行如权利要求1至9中任一项所述的图像处理方法。
PCT/CN2022/073190 2021-06-24 2022-01-21 图像处理方法、装置和电子设备 WO2022267466A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP22826975.9A EP4242961A4 (en) 2021-06-24 2022-01-21 IMAGE PROCESSING METHOD AND DEVICE AND ELECTRONIC DEVICE
US18/266,546 US20240046604A1 (en) 2021-06-24 2022-01-21 Image processing method and apparatus, and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110707978.1A CN113570617B (zh) 2021-06-24 2021-06-24 图像处理方法、装置和电子设备
CN202110707978.1 2021-06-24

Publications (2)

Publication Number Publication Date
WO2022267466A1 true WO2022267466A1 (zh) 2022-12-29
WO2022267466A9 WO2022267466A9 (zh) 2023-01-26

Family

ID=78162670

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/073190 WO2022267466A1 (zh) 2021-06-24 2022-01-21 图像处理方法、装置和电子设备

Country Status (4)

Country Link
US (1) US20240046604A1 (zh)
EP (1) EP4242961A4 (zh)
CN (1) CN113570617B (zh)
WO (1) WO2022267466A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116363538A (zh) * 2023-06-01 2023-06-30 贵州交投高新科技有限公司 一种基于无人机的桥梁检测方法及系统
CN117975044A (zh) * 2024-02-20 2024-05-03 蚂蚁云创数字科技（北京）有限公司 基于特征空间的图像处理方法及装置

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113570617B (zh) * 2021-06-24 2022-08-23 荣耀终端有限公司 图像处理方法、装置和电子设备
CN114627000A (zh) * 2022-01-28 2022-06-14 华为技术有限公司 图像处理方法、装置和电子设备
CN115034967B (zh) * 2022-06-27 2025-02-07 北京奇艺世纪科技有限公司 图像处理方法、装置、电子设备及计算机可读存储介质
CN118784928A (zh) * 2023-04-03 2024-10-15 抖音视界有限公司 视频处理方法、装置、电子设备和存储介质
CN118972710A (zh) * 2024-10-14 2024-11-15 深圳御光新材料有限公司 一种基于无人机集群搭载的多屏幕显示方法与系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107087107A (zh) * 2017-05-05 2017-08-22 中国科学院计算技术研究所 基于双摄像头的图像处理装置及方法
US20180352165A1 (en) * 2017-06-05 2018-12-06 Samsung Electronics Co., Ltd. Device having cameras with different focal lengths and a method of implementing cameras with different focal lenghts
CN111147755A (zh) * 2020-01-02 2020-05-12 普联技术有限公司 双摄像头的变焦处理方法、装置及终端设备
CN112819738A (zh) * 2021-01-19 2021-05-18 合肥英睿系统技术有限公司 红外图像融合方法、装置、计算机设备和存储介质
CN113570617A (zh) * 2021-06-24 2021-10-29 荣耀终端有限公司 图像处理方法、装置和电子设备
CN113592751A (zh) * 2021-06-24 2021-11-02 荣耀终端有限公司 图像处理方法、装置和电子设备

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150021353A (ko) * 2013-08-20 2015-03-02 삼성테크윈 주식회사 영상 융합 시스템 및 그 방법
CN108401104B (zh) * 2018-02-08 2020-11-13 浙江大学 基于频带修复和超分辨的双焦相机数字变焦方法
CN110458932B (zh) * 2018-05-07 2023-08-22 阿里巴巴集团控股有限公司 图像处理方法、装置、系统、存储介质和图像扫描设备
JP2020009099A (ja) * 2018-07-06 2020-01-16 キヤノン株式会社 画像処理装置、画像処理方法、及びプログラム
US10817996B2 (en) * 2018-07-16 2020-10-27 Samsung Electronics Co., Ltd. Devices for and methods of combining content from multiple frames
CN109741289B (zh) * 2019-01-25 2021-12-21 京东方科技集团股份有限公司 一种图像融合方法和vr设备
CN112087580B (zh) * 2019-06-14 2022-03-18 Oppo广东移动通信有限公司 图像采集方法和装置、电子设备、计算机可读存储介质
WO2020258297A1 (zh) * 2019-06-28 2020-12-30 深圳市大疆创新科技有限公司 图像语义分割方法、可移动平台及存储介质
CN110430359B (zh) * 2019-07-31 2021-07-09 北京迈格威科技有限公司 拍摄辅助方法、装置、计算机设备和存储介质
CN111294517B (zh) * 2020-03-03 2021-12-17 荣耀终端有限公司 一种图像处理的方法及移动终端
CN112837321B (zh) * 2021-02-09 2022-10-11 北京航空航天大学 一种基于光场的场景语义分割系统及方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107087107A (zh) * 2017-05-05 2017-08-22 中国科学院计算技术研究所 基于双摄像头的图像处理装置及方法
US20180352165A1 (en) * 2017-06-05 2018-12-06 Samsung Electronics Co., Ltd. Device having cameras with different focal lengths and a method of implementing cameras with different focal lenghts
CN111147755A (zh) * 2020-01-02 2020-05-12 普联技术有限公司 双摄像头的变焦处理方法、装置及终端设备
CN112819738A (zh) * 2021-01-19 2021-05-18 合肥英睿系统技术有限公司 红外图像融合方法、装置、计算机设备和存储介质
CN113570617A (zh) * 2021-06-24 2021-10-29 荣耀终端有限公司 图像处理方法、装置和电子设备
CN113592751A (zh) * 2021-06-24 2021-11-02 荣耀终端有限公司 图像处理方法、装置和电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4242961A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116363538A (zh) * 2023-06-01 2023-06-30 贵州交投高新科技有限公司 一种基于无人机的桥梁检测方法及系统
CN116363538B (zh) * 2023-06-01 2023-08-01 贵州交投高新科技有限公司 一种基于无人机的桥梁检测方法及系统
CN117975044A (zh) * 2024-02-20 2024-05-03 蚂蚁云创数字科技（北京）有限公司 基于特征空间的图像处理方法及装置

Also Published As

Publication number Publication date
WO2022267466A9 (zh) 2023-01-26
US20240046604A1 (en) 2024-02-08
CN113570617B (zh) 2022-08-23
CN113570617A (zh) 2021-10-29
EP4242961A1 (en) 2023-09-13
EP4242961A4 (en) 2024-08-28

Similar Documents

Publication Publication Date Title
WO2022267466A1 (zh) 图像处理方法、装置和电子设备
CN109544618B (zh) 一种获取深度信息的方法及电子设备
CN114092364B (zh) 图像处理方法及其相关设备
WO2022100685A1 (zh) 一种绘制命令处理方法及其相关设备
WO2023279862A1 (zh) 图像处理方法、装置和电子设备
CN112087649B (zh) 一种设备搜寻方法以及电子设备
CN115567630B (zh) 一种电子设备的管理方法、电子设备及可读存储介质
WO2022206589A1 (zh) 一种图像处理方法以及相关设备
WO2021238351A1 (zh) 一种图像校正方法与电子设备
WO2023284715A1 (zh) 一种物体重建方法以及相关设备
CN115601244A (zh) 图像处理方法、装置和电子设备
WO2021057626A1 (zh) 图像处理方法、装置、设备及计算机存储介质
CN114880251B (zh) 存储单元的访问方法、访问装置和终端设备
CN112700377A (zh) 图像泛光处理方法及装置、存储介质
CN113592751B (zh) 图像处理方法、装置和电子设备
CN114283195B (zh) 生成动态图像的方法、电子设备及可读存储介质
WO2022135144A1 (zh) 自适应显示方法、电子设备及存储介质
WO2022033344A1 (zh) 视频防抖方法、终端设备和计算机可读存储介质
CN116048769A (zh) 内存回收方法、装置和终端设备
CN118474522B (zh) 拍照方法、终端设备、芯片及存储介质
CN116074624B (zh) 一种对焦方法和装置
CN115640414B (zh) 图像的显示方法及电子设备
CN113129636B (zh) 车辆信息的推送方法、装置、用户账号服务器和用户设备
CN119520974A (zh) 拍摄方法、电子设备
CN116055870A (zh) 一种拍照方法和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22826975

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18266546

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2022826975

Country of ref document: EP

Effective date: 20230608

NENP Non-entry into the national phase

Ref country code: DE