CN114037608B - A method for image registration - Google Patents
A method for image registration Download PDFInfo
- Publication number
- CN114037608B CN114037608B CN202111219273.1A CN202111219273A CN114037608B CN 114037608 B CN114037608 B CN 114037608B CN 202111219273 A CN202111219273 A CN 202111219273A CN 114037608 B CN114037608 B CN 114037608B
- Authority
- CN
- China
- Prior art keywords
- image
- pixel
- fpga
- coordinate transformation
- registered
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 230000009466 transformation Effects 0.000 claims abstract description 125
- 238000012545 processing Methods 0.000 claims abstract description 29
- 238000004590 computer program Methods 0.000 claims description 11
- 238000011524 similarity measure Methods 0.000 claims description 9
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 2
- 238000004364 calculation method Methods 0.000 description 11
- 238000006243 chemical reaction Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 230000003139 buffering effect Effects 0.000 description 7
- 238000000605 extraction Methods 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 230000004927 fusion Effects 0.000 description 5
- 102100031584 Cell division cycle-associated 7-like protein Human genes 0.000 description 4
- 101000777638 Homo sapiens Cell division cycle-associated 7-like protein Proteins 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000004075 alteration Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000009966 trimming Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0007—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
本申请公开了一种图像配准方法及装置，用以在FPGA上实现满足配准效果和处理速度要求的图像配准方法。该方法包括：通过FPGA获取目标图像的第一坐标变换表；第一坐标变换表包括N个元素；其中，一个元素为目标图像中一个像素点在待配准图像中的坐标位置；通过FPGA根据第n个元素的横坐标获取待配准图像中的四个像素点；第n个元素是第一坐标变换表中的任一元素，n依次取遍1到N之间的任意整数；通过FPGA根据四个像素点，计算待配准图像中第n个元素的横坐标所对应的像素点的第一像素值；通过FPGA根据第n个元素的纵坐标，从N个第一像素值中四个第一像素值；通过FPGA根据四个第一像素值和元素的纵坐标，计算第二像素值。
The present application discloses an image registration method and device, which is used to implement an image registration method that meets the registration effect and processing speed requirements on an FPGA. The method includes: obtaining a first coordinate transformation table of a target image through an FPGA; the first coordinate transformation table includes N elements; wherein one element is the coordinate position of a pixel point in the target image in the image to be registered; obtaining four pixel points in the image to be registered through the FPGA according to the horizontal coordinate of the nth element; the nth element is any element in the first coordinate transformation table, and n is any integer between 1 and N in turn; calculating the first pixel value of the pixel point corresponding to the horizontal coordinate of the nth element in the image to be registered through the FPGA according to the four pixel points; obtaining four first pixel values from the N first pixel values according to the vertical coordinate of the nth element through the FPGA; calculating the second pixel value according to the four first pixel values and the vertical coordinate of the element through the FPGA.
Description
Technical Field
The application relates to the technical field of image processing, in particular to an image registration method.
Background
Image registration is the process of matching and overlaying two or more images acquired at different times, with different imaging devices, or under different conditions. Wherein, different conditions include different climates, illuminance, shooting positions, angles and the like. Existing image registration methods fall into two broad categories, including region-based registration methods and feature-based registration methods. The region-based registration method mainly uses gray information of regions independent of image features, and the feature-based registration method is to extract fixed features from images and complete registration through matching among the features. Many of these image registration methods are computationally complex and only suitable for running on a central processing unit (Central Processing Unit, CPU). However, the capability of the CPU to process large-scale data is not strong, and the CPU is not suitable for occasions of displaying large-resolution real-time video images.
The field programmable gate array (Field Programmable GATE ARRAY, FPGA) has strong large-scale data processing capability and high operation speed, so that the field programmable gate array has great advantages in the aspect of image processing, and the Field Programmable Gate Array (FPGA) is commonly used in a binocular camera to realize the image processing. The image registration is used as an intermediate link of image processing and is needed to be realized on the FPGA to be matched with other image processing algorithms, but the existing registration algorithm is generally complex, and the difficulty of realizing the image registration on the FPGA is quite high, so that an image registration method suitable for being realized on the FPGA is needed.
Disclosure of Invention
The embodiment of the application provides an image registration method which is used for realizing the image registration method which can meet the requirements of registration effect and processing speed and is suitable for being realized on an FPGA.
In a first aspect, an embodiment of the present application provides an image registration method, including:
Acquiring a first coordinate transformation table of a target image through a Field Programmable Gate Array (FPGA); the first coordinate transformation table comprises N elements, wherein N is a positive integer; one element is the coordinate position of one pixel point in the target image in the image to be registered; the target image and the image to be registered are images aiming at the same object, and the generation conditions of the target image and the image to be registered are different;
Acquiring four pixel points in the image to be registered according to an nth element through the FPGA; the nth element is any element in the first coordinate transformation table, and N is any integer between 1 and N in sequence; four pixel points in the image to be registered are pixel points in the same row in the image to be registered; the row of the four pixel points is determined according to the ordinate of the nth element;
calculating a first pixel value of a pixel point corresponding to an nth element in the image to be registered according to the four pixel points by the FPGA, and sequentially taking N to be any integer between 1 and N;
Acquiring four first pixel values corresponding to the nth element from the N first pixel values through the FPGA; the column numbers of the four first pixel values are the same as the column number of the nth element, the row numbers of the four first pixel values are adjacent, and the row number of the first pixel value in the four first pixel values is the same as the row number of the nth element;
And calculating a second pixel value by the FPGA according to the four first pixel values and the ordinate of the nth element.
Based on the scheme, after the coordinate transformation table of the target image and the image to be registered is obtained, the registration of the two images is realized by adopting a horizontal-first-then-vertical mode to perform bicubic interpolation, so that the calculated amount can be reduced, the registration of the images can be realized through an FPGA, and the method is simple and easy to realize.
In one possible implementation manner, the acquiring, by the FPGA, the first coordinate transformation table of the target image includes:
respectively extracting the characteristics of the image to be registered and the target image through a CPU (Central processing Unit) to obtain a first characteristic point set of the image to be registered and a second characteristic point set of the target image;
determining a characteristic point pair set according to the similarity measurement by the CPU; the characteristic point pair set comprises a plurality of characteristic point pairs, and one characteristic point pair comprises a first characteristic point in the first characteristic point set and a second characteristic point in the second characteristic point set; the similarity measure of the first feature point and the second feature point is greater than or equal to a first threshold;
and determining the first coordinate transformation table according to the characteristic point pair set by the CPU.
Based on the scheme, the requirements of steps such as feature extraction, feature matching and the like on the real-time performance in the real-time registration of the images are not high compared with the coordinate transformation and interpolation, but complex functions and iterative operation are required to be calculated, so that the operations of feature extraction and feature matching can be realized by utilizing a CPU, and the problem that a complex registration algorithm cannot be satisfied by using an FPGA alone is avoided.
In one possible implementation manner, determining the first coordinate transformation table according to the feature point pair set includes:
determining a second coordinate transformation table by the CPU according to the characteristic point pair set;
The CPU performs k times downsampling on the second coordinate transformation table to obtain a third coordinate transformation table; the number of elements in the second coordinate transformation table is k times the number of elements in the third coordinate transformation table; k is an integer greater than 1;
Performing k times up-sampling on the third coordinate transformation table through the FPGA to obtain the first coordinate transformation table; the number of elements in the first coordinate transformation table is the same as the number of elements in the second coordinate transformation table.
Based on the scheme, when the CPU sends the coordinate conversion table to the FPGA, the coordinate conversion table is firstly downsampled in the CPU, so that the data quantity to be sent can be reduced, then the downsampled table is sent to the FPGA, the FPGA firstly upsamples after obtaining the table and restores to the original size of the table for reuse, so that the sending time can be reduced, the storage resource occupation of the FPGA can be reduced, and meanwhile, the actual registration requirement can be met.
In a possible implementation manner, the acquiring, by the FPGA, four pixel points in the image to be registered according to the abscissa of the nth element includes:
Acquiring four pixel points from four RAMs according to the abscissa of the nth element by the FPGA; wherein, each RAM obtains a pixel point; the four RAMs are respectively stored with partial pixel points of the image to be registered; the number of pixel points stored in the four RAMs is the same.
In one possible implementation manner, the 4 m-th column of pixels in the image to be registered is stored in the first RAM, the 4m+1-th column of pixels is stored in the second RAM, the 4m+2-th column of pixels is stored in the third RAM, the 4m+3-th column of pixels is stored in the fourth RAM, and m is an integer greater than or equal to 0.
Based on the scheme, the RAM can only take out one data at a time, so that the requirement of taking four data at a time in a bicubic interpolation algorithm can be met by storing pixel points in four RAMs according to columns to finish horizontal interpolation.
In one possible implementation manner, after the calculating, by the FPGA, the first pixel value corresponding to the abscissa of the nth element according to the four pixel points, the method further includes:
caching the obtained N first pixel values into a fifth RAM;
The obtaining, by the FPGA, four first pixel values of the N first pixel values according to the ordinate of the nth element includes:
and acquiring four first pixel values in the N first pixel values from the fifth RAM according to the ordinate of the nth element by the FPGA.
Based on the scheme, the result of interpolation in the horizontal direction is stored in the fifth RAM, so that vertical interpolation can be performed on the result of the horizontal interpolation, the calculated amount of the FPGA can be reduced, and the occupation of the FPGA resources is reduced.
In one possible implementation manner, after the calculating, by the FPGA, the second pixel value corresponding to the ordinate of the nth element according to the four first pixel values, the method further includes:
Performing clock domain crossing operation on the N second pixel values through a first-in first-out memory FIFO; the clock frequency of the N second pixel values subjected to the cross-clock domain operation is the same as that of the DDR;
And storing the N second pixel values subjected to the cross-clock domain operation in DDR.
Based on the scheme, the image registration result in the RAM can be stored in the DDR by firstly storing the interpolation result in the FIFO to perform the operation of crossing clock domains.
In a second aspect, an embodiment of the present application provides an image registration apparatus, including a storage unit and a processing unit:
The storage unit is used for storing a computer program or instructions;
The processing unit is configured to perform the following operations:
Acquiring a first coordinate transformation table of a target image through a Field Programmable Gate Array (FPGA); the first coordinate transformation table comprises N elements, wherein N is a positive integer; one element is the coordinate position of one pixel point in the target image in the image to be registered; the target image and the image to be registered are images aiming at the same object, and the generation conditions of the target image and the image to be registered are different;
Acquiring four pixel points in the image to be registered according to an nth element through the FPGA; the nth element is any element in the first coordinate transformation table, and N is any integer between 1 and N in sequence; four pixel points in the image to be registered are pixel points in the same row in the image to be registered; the row of the four pixel points is determined according to the ordinate of the nth element;
calculating a first pixel value of a pixel point corresponding to an nth element in the image to be registered according to the four pixel points by the FPGA, and sequentially taking N to be any integer between 1 and N;
Acquiring four first pixel values corresponding to the nth element from the N first pixel values through the FPGA; the column numbers of the four first pixel values are the same as the column number of the nth element, the row numbers of the four first pixel values are adjacent, and the row number of the first pixel value in the four first pixel values is the same as the row number of the nth element;
And calculating a second pixel value by the FPGA according to the four first pixel values and the ordinate of the nth element.
In one possible implementation manner, the acquiring, by the FPGA, the first coordinate transformation table of the target image includes:
respectively extracting the characteristics of the image to be registered and the target image through a CPU (Central processing Unit) to obtain a first characteristic point set of the image to be registered and a second characteristic point set of the target image;
determining a characteristic point pair set according to the similarity measurement by the CPU; the characteristic point pair set comprises a plurality of characteristic point pairs, and one characteristic point pair comprises a first characteristic point in the first characteristic point set and a second characteristic point in the second characteristic point set; the similarity measure of the first feature point and the second feature point is greater than or equal to a first threshold;
and determining the first coordinate transformation table according to the characteristic point pair set by the CPU.
In one possible implementation manner, determining the first coordinate transformation table according to the feature point pair set includes:
determining a second coordinate transformation table by the CPU according to the characteristic point pair set;
The CPU performs k times downsampling on the second coordinate transformation table to obtain a third coordinate transformation table; the number of elements in the second coordinate transformation table is k times the number of elements in the third coordinate transformation table; k is an integer greater than 1;
Performing k times up-sampling on the third coordinate transformation table through the FPGA to obtain the first coordinate transformation table; the number of elements in the first coordinate transformation table is the same as the number of elements in the second coordinate transformation table.
In a possible implementation manner, the acquiring, by the FPGA, four pixel points in the image to be registered according to the abscissa of the nth element includes:
Acquiring four pixel points from four RAMs according to the abscissa of the nth element by the FPGA; wherein, each RAM obtains a pixel point; the four RAMs are respectively stored with partial pixel points of the image to be registered; the number of pixel points stored in the four RAMs is the same.
In one possible implementation manner, the 4 m-th column of pixels in the image to be registered is stored in the first RAM, the 4m+1-th column of pixels is stored in the second RAM, the 4m+2-th column of pixels is stored in the third RAM, the 4m+3-th column of pixels is stored in the fourth RAM, and m is an integer greater than or equal to 0.
In one possible implementation manner, after the calculating, by the FPGA, the first pixel value corresponding to the abscissa of the nth element according to the four pixel points, the method further includes:
caching the obtained N first pixel values into a fifth RAM;
The obtaining, by the FPGA, four first pixel values of the N first pixel values according to the ordinate of the nth element includes:
and acquiring four first pixel values in the N first pixel values from the fifth RAM according to the ordinate of the nth element by the FPGA.
In one possible implementation manner, after the calculating, by the FPGA, the second pixel value corresponding to the ordinate of the nth element according to the four first pixel values, the method further includes:
Performing clock domain crossing operation on the N second pixel values through a first-in first-out memory FIFO; the clock frequency of the N second pixel values subjected to the cross-clock domain operation is the same as that of the DDR;
And storing the N second pixel values subjected to the cross-clock domain operation in DDR.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;
the memory is used for storing a computer program or instructions;
The processor is configured to execute a computer program or instructions in the memory, such that the method according to any of the above aspects is performed.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, perform the steps of the method of any of the above aspects.
In addition, the advantages of the second aspect to the fourth aspect may be referred to as the advantages of the first aspect, and will not be described here.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application.
Fig. 1 is a schematic structural diagram of an image capturing apparatus according to an embodiment of the present application;
FIG. 2 is one of exemplary flowcharts of an image registration method provided by an embodiment of the present application;
Fig. 3 is a schematic diagram of downsampling a coordinate transformation table according to an embodiment of the present application;
FIG. 4 is a schematic diagram of bicubic interpolation according to an embodiment of the present application;
FIG. 5 is a simplified function and bicubic function curve comparison schematic diagram provided by an embodiment of the present application;
FIG. 6 is a block diagram of a horizontal interpolation result for line buffering according to an embodiment of the present application;
FIG. 7 is one of exemplary flowcharts of an image registration method provided by an embodiment of the present application;
fig. 8 is a schematic structural diagram of an image registration apparatus according to an embodiment of the present application;
Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the technical solutions of the present application, but not all embodiments. All other embodiments, based on the embodiments described in the present document, which can be obtained by a person skilled in the art without any creative effort, are within the scope of protection of the technical solutions of the present application.
The terms "first" and "second" in embodiments of the application are used to distinguish between different objects and are not used to describe a particular sequence. Furthermore, the term "include" and any variations thereof is intended to cover non-exclusive protection. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus. The term "plurality" in the present application may mean at least two, for example, two, three or more, and embodiments of the present application are not limited.
In addition, the term "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. The character "/" herein generally indicates that the associated object is an "or" relationship unless otherwise specified.
In recent years, the multisource fusion technology has rapid development, has important functions in the field of monitoring security, infrared and visible light fusion, and an infrared image is not limited by conditions such as weather illumination and the like, can be applied to scenes such as poor visible conditions and obvious target thermal characteristics, and is a powerful supplement to the visible light image, so that a binocular camera based on infrared and visible light image fusion plays an important role in the field of monitoring security. However, in the binocular camera with the infrared and visible light image fusion, the two cameras are different in installation position, so that the same shot picture is different, and then the two images shot by the two cameras are required to be aligned in picture alignment. The capability of the CPU to process large-scale data is not strong, and is not suitable for the occasion of displaying large-resolution real-time video images, so how to design an image registration method suitable for being implemented on an FPGA is needed to be solved.
In view of this, the embodiment of the application provides an image registration method. In the method, firstly, the operations of extracting the characteristics, matching the characteristics and estimating the transformation model of the target image are completed on a CPU, after a coordinate transformation table is obtained, the CPU performs downsampling on the coordinate transformation table and sends the downsampling result to the FPGA. And then the FPGA resamples the target image according to the coordinate transformation table to realize coordinate transformation and interpolation. Thereby achieving the purpose of realizing image registration on the FPGA.
The technical scheme provided by the embodiment of the application can be applied to an image registration device, such as a binocular camera based on infrared and visible light image fusion. Referring to fig. 1, a structure of an image registration apparatus according to an embodiment of the present application is provided. The device 100 includes a CPU101, a bus 102 and an FPGA103. The CPU101 has strong capability of scheduling, circulating and logic judging complex instructions, data communication between the CPU101 and the FPGA103 can be realized through the bus 102, and the FPGA103 has the characteristic of strong capability of processing large-scale data and high operation speed.
Referring to fig. 2, an exemplary flowchart of an image registration method according to an embodiment of the present application may include the following operations:
s201, the device acquires a first coordinate transformation table of the target image through the FPGA.
The first coordinate transformation table may be a set of positions of each pixel point in the target image in the image to be registered. Before the device obtains a first coordinate transformation table of a target image through the FPGA, a second coordinate transformation table of the target image is calculated on the CPU according to a certain image feature extraction and feature matching method. The second coordinate transformation table comprises the position of each pixel point of the target image in the image to be registered. The target image and the image to be registered are images for the same object, and the generation conditions of the target image and the image to be registered are different. The generating conditions may include conditions such as photographing time, imaging device, photographing angle, photographing position, or different weather. In addition, the target image and the image to be registered may also be composite images, and the present application is not limited herein.
The device calculates the second coordinate conversion table of the target image on the CPU as follows:
The device respectively performs feature extraction on the image to be registered and the target image on the CPU to obtain a first feature point set of the image to be registered and a second feature point set of the target image, then performs similarity measurement on feature points in the first feature point set and the second feature point set, and determines a feature point pair set. The feature point pair set comprises a plurality of feature point pairs, one feature point pair comprises a first feature point in the first feature point set and a second feature point in the second feature point set, and when the similarity measure of the first feature point and the second feature point is greater than or equal to a first threshold value, the feature point pair is determined. The apparatus determines a second coordinate transformation table of the target image from the set of feature point pairs on the CPU.
The first threshold value may be determined according to an empirical value, and the present application is not particularly limited. Optionally, the feature point pair includes a first feature point and a second feature point, and the similarity measure between the first feature point and the other feature points except the second feature point is the largest.
It should be noted that, the method for acquiring the coordinate transformation table in the embodiment of the present application is merely exemplary, and those skilled in the art may acquire the coordinate transformation table of the target image and the image to be registered in other manners.
Based on the scheme, the requirements of steps such as feature extraction, feature matching and the like on the real-time performance in the real-time registration of the images are not high compared with the coordinate transformation and interpolation, but complex functions and iterative operation are required to be calculated, so that the operations of feature extraction and feature matching can be realized by utilizing a CPU, and the problem that a complex registration algorithm cannot be satisfied by using an FPGA alone is avoided.
The device calculates a second coordinate transformation table of the target image on the CPU, then performs k times downsampling on the second coordinate transformation table to obtain a third coordinate transformation table, and then sends the third coordinate transformation table to the FPGA. Where k is an integer greater than 1.
And after the device receives the third coordinate transformation table through the FPGA, up-sampling the third coordinate transformation table by k times, and recovering the third coordinate transformation table to be the same as the second coordinate transformation table in size. Here, k may be determined based on an empirical value, and the present application is not particularly limited.
Referring to fig. 3, a schematic diagram of downsampling a coordinate transformation table according to an embodiment of the present application is provided. Fig. 3 illustrates an example of 4-fold downsampling of a coordinate conversion table of 32x32 resolution. The shaded part in the graph is the point reserved after downsampling, the point not shaded is the point lost after downsampling, after 4 times downsampling is carried out on a coordinate conversion table with the resolution of 32x32, the size of the coordinate conversion table is changed to 8x8, the upsampling process is opposite to downsampling, the upsampling is the point lost after downsampling is reduced, and a bilinear interpolation method can be adopted in the upsampling interpolation calculation.
Based on the scheme, when the CPU sends the coordinate conversion table to the FPGA, the coordinate conversion table is firstly downsampled in the CPU, so that the data quantity to be sent can be reduced, then the downsampled table is sent to the FPGA, the FPGA firstly upsamples after obtaining the table and restores to the original size of the table for reuse, so that the sending time can be reduced, the storage resource occupation of the FPGA can be reduced, and meanwhile, the actual registration requirement can be met.
S202, the device acquires four pixel points in an image to be registered according to an nth element through an FPGA; the nth element is any element in the first coordinate transformation table, and N is sequentially taken through any integer between 1 and N.
Specifically, when the device acquires four pixel points according to the nth element through the FPGA, the device may first acquire, according to the integer part of the ordinate of the nth element, the pixel points in the row of the integer part of the ordinate in the image to be registered, and then select, as the four pixel points of the horizontal interpolation, the four pixel points with the smallest interpolation with the abscissa of the nth element in the row. For example, when the nth element coordinate is (2.3,1.5), the first row of pixels is determined to be used according to the ordinate 1.5, and then the 1 st, 2 nd, 3 rd and 4 th columns of the first row are determined to be used according to the abscissa 2.3, and the four pixel points are used as four pixel points of the horizontal interpolation.
S203, the device calculates a first pixel value of a pixel point corresponding to an nth element in the image to be registered according to the four pixel points through the FPGA.
In the embodiment of the present application, when the pixel value of the image to be registered is calculated through the first coordinate transformation table, since the coordinates in the coordinate transformation table are fractional, that is, the pixel point corresponding to the coordinates may be located between a few pixel points in the image to be registered, the value of the pixel point corresponding to the coordinates needs to be calculated through interpolation, that is, the value of the pixel point surrounding the coordinates is calculated. The embodiment of the application adopts a bicubic interpolation mode to carry out interpolation calculation.
Referring to fig. 4, a bicubic interpolation schematic diagram is provided in an embodiment of the present application. The bicubic interpolation is to calculate the pixel value of the point to be interpolated by using 16 pixel points within a 4x4 range around the point to be interpolated, mainly by finding a factor to represent the influence factor of the pixel value of the 16 pixel points to be interpolated, so as to calculate the pixel value of the point to be interpolated according to the influence factor, wherein the calculation formula is shown in formula 1:
In the above formula, (x, y) represents coordinates of a point to be interpolated, (x i,yi) represents coordinates of 16 points around the point to be interpolated, where i e (0, 1,2, 3), j e (0, 1,2, 3), f (x, y) represents a pixel value at the coordinates (x, y), h (x) represents a weight coefficient at a distance x from the point to be interpolated, it is calculated by using a bicubic function, h (x) =sin (pi x)/(pi x), and the function is simplified as shown in formula 2 to obtain:
In the embodiment of the application, i epsilon (0, 1,2, 3) is assumed, and i epsilon (1, 2,3, 4) can be also adopted in the specific implementation. Similarly, in the embodiment of the present application, j's epsilon (0, 1,2, 3) may be j's epsilon (1, 2,3, 4) during implementation, and the present application is not limited in particular.
Referring to fig. 5, a simplified functional and bicubic functional curve comparison schematic is provided for an embodiment of the present application. Wherein the solid line is a simplified functional image, and the dotted line is a double cubic functional image.
And then, the simplified function is fixed to obtain a fixed-point function:
in the above formula, h (x) represents a weight coefficient at which the distance between the point to be interpolated and the point to be interpolated is x, and m represents the width of the decimal part after the localization.
In general, when bicubic interpolation is performed, 16 pixel points within a range of 4x4 around a point to be interpolated are utilized to calculate a pixel value of the point to be interpolated, 36 multiplication operations are needed to calculate an interpolation result, and the calculated amount is relatively large.
When interpolation in the horizontal direction is carried out, 4 pixel points in the horizontal direction are taken, and a calculation formula is as follows:
Wherein: x represents the abscissa of the point to be interpolated, x i represents the abscissa of the i-th point of the 4 points around the point to be interpolated, where i e (0, 1,2, 3), f (x i) represents the pixel value of the image to be registered at the abscissa x i, and g (x) represents the horizontally interpolated pixel value at the abscissa x. Where h (x-x i) may represent the weight of the ith pixel value, which may be obtained by equation (3) above.
It should be understood that, of the 4 pixels in the image to be registered, the first pixel may be understood as the point with the smallest value on the abscissa, and the fourth pixel may be understood as the point with the largest value on the abscissa. In the above formulas (1) to (3), the first pixel may be regarded as i=0, the second pixel may be regarded as i=1, the third pixel may be regarded as i=2, and the fourth pixel may be regarded as i=3.
When the device realizes interpolation calculation on the FPGA, firstly, all pixel point Data of an image to be registered are read from a Double Data Rate synchronous dynamic random access memory (Double Data Rate SynchronousDynamicRandomAccessMemory, DDR) and written into a random access memory (Random Access Memory, RAM) RAM0, wherein the size of the RAM0 is 2 times of the number of the pixels of the image to be registered. Thus, the ping-pong operation can be performed when the RAM0 is read and written.
Wherein, the step of ping-pong operation is: in a first buffering period, buffering an input data stream to a first data buffering module; in the second buffering period, the input data stream is buffered to the second data buffering module through the switching of the input data selecting unit, and meanwhile, the 1 st period data buffered by the first data buffering module is sent to the data stream operation processing module for operation processing through the selection of the input data selecting unit; and in the third buffer period, the input data stream is buffered to the first data buffer module through the re-switching of the input data selection unit, and meanwhile, the data in the second period buffered by the second data buffer module is transmitted to the data stream operation processing module for operation processing through the switching of the input data selection unit, so that the cycle is circulated. The data can be transmitted to the data stream operation processing module for operation and processing through the switching of the input data selection unit and the output data selection unit according to the beat and the mutual matching through the ping-pong operation, and the buffer space can be saved.
And then reading all pixel point data of the image to be registered from the RAM0, and writing the pixel point data into 4 RAMs of the RAMs 1_0, the RAMs 1_1, the RAMs 1_2 and the RAMs 1_3 according to the positions of the pixel point data in the image to be registered, wherein the sizes of the four RAMs are one fourth of the size of the RAMs 0, the 4 m-th row of pixel points are stored in the RAMs 1_0, the 4m+1-th row of pixel points are stored in the RAMs 1_1, the 4m+2-th row of pixel points are stored in the RAMs 1_2, the 4m+3-th row of pixel points are stored in the RAMs 1_3, and m is an integer greater than or equal to 0. Therefore, the storage pixel points can ensure that 4 adjacent numbers in each row of data are respectively stored in 4 RAMs, and the required 4 pixel points can be obtained at one time when interpolation in the horizontal direction is carried out. And then the required pixel point is fetched from the RAM1_0, RAM1_1, RAM1_2 and RAM1_3 according to the coordinate values of the first coordinate conversion table.
Based on the scheme, the RAM can only take out one data at a time, so that the requirement of taking four data at a time in a bicubic interpolation algorithm can be met by storing pixel points in four RAMs according to columns to finish horizontal interpolation.
The device performs interpolation calculation in the horizontal direction on the FPGA according to the four pixel points taken out in S202, and after obtaining the first pixel value (interpolation calculation result), stores the interpolation calculation result in the RAM2 according to the line. The size of the RAM2 is three times the number of one row of pixels in the image to be registered, and is used for caching three rows of pixels.
S204, the device acquires four first pixel values corresponding to the nth element from the N first pixel values through the FPGA.
When interpolation in the vertical direction is performed, four first pixel values corresponding to the nth element may be fetched from the RAM 2. The column numbers of the four first pixel values are the same as the column number of the nth element, and the row numbers of the four first pixel values are adjacent. Since the first pixel values are calculated according to the order of the elements, the N first pixel values calculated in S203 are stored correspondingly according to the positions of the elements, and the row number and the column number where the first pixel values are located represent the relative positions of the first pixel values in the N first pixel values. For example, if a certain first pixel value is located in the 2 nd row and the 3 rd column among the N first pixel values, the row number of the first pixel value is2 and the column number of the first pixel value is 3. Assuming that the nth element is located in the 2 nd column and the 3 rd row of the first coordinate transformation table, the four pixel values are the first pixel value of the 2 nd column and the 3 rd row of the N first pixel values, the first pixel value of the 2 nd column and the 4 th row of the N first pixel values, the first pixel value of the 2 nd column and the 5 th row of the N first pixel values, and the first pixel value of the 2 nd column and the 6 th row of the N first pixel values, respectively.
S205, the device calculates a second pixel value through the FPGA according to the four first pixel values and the ordinate of the element in the first coordinate transformation table corresponding to the four first pixel values.
The second pixel value may be considered as an interpolation result obtained by performing vertical interpolation on the four first pixel values. Specifically, when vertical interpolation is performed, the calculation formula is: Wherein: y represents the ordinate of the point to be interpolated, g (y i) represents the i-th first pixel value of the four first pixel values, where i e (0, 1,2, 3), and p (y) represents the final pixel value at the ordinate y.
Wherein h (y-y i) is obtained by the above formula (3). Since the line numbers of the four pixel values are adjacent, the distance between the two pixel values can be regarded as 1, so after the four first pixel values are acquired, h (x-x i) can be calculated from the fractional part of the ordinate of the nth element. Where h (x-x i) can be calculated by the above formula (3), but (x-x i) substituted into the formula is calculated from the fractional part of the ordinate of the nth element, where i can represent the i-th first pixel value of the above four first pixel values. Wherein, the first pixel value of the four first pixel values can be regarded as i=0, the second pixel value can be regarded as i=1, and so on. Specifically, (x-x 0) = |n-th element's ordinate fractional part +1|, second (x-x 1) = |n-th element's ordinate fractional part|, third (x-x 2) = |1-n-th element's ordinate fractional part|, fourth (x-x 3) = |2-n-th element's ordinate fractional part|.
Based on the scheme, when interpolation is carried out in the FPGA, horizontal interpolation is carried out on the image to be registered, and then interpolation in the vertical direction is carried out on the basis of the horizontal interpolation, so that the use of resources, particularly the use of multipliers, can be reduced, and the image registration quality can be ensured.
In one possible implementation, the horizontal interpolation and the vertical interpolation may also be performed in parallel. Specifically, the device may perform the above S202 and S203 through the FPGA to obtain the horizontal interpolation result (the first pixel value) of the first three rows of elements in the first coordinate transformation table. After the device performs the above S202 and S203 to obtain the horizontal interpolation result of the first column element of the fourth row in the first coordinate transformation table through the FPGA, the device may perform the above S204 and S205 through the FPGA to obtain the second pixel value of the first element in the first coordinate transformation table. At this time, the apparatus may obtain the first pixel value of the remaining element in the first coordinate transformation table through the FPGAs in S202 and S203 described above, and may obtain the second pixel value of the remaining element in the first coordinate transformation table by simultaneously performing S204 and S205 described above through the FPGAs.
Referring to fig. 6, a block diagram of a horizontal interpolation result line buffer is provided in an embodiment of the present application. The data, i.e. the horizontal interpolation result, is input from the left end and output from the right end. The line cache 0, the line cache 1 and the line cache 2 respectively correspond to the three-line horizontal interpolation results of the cache, the output of the line cache 0 can be input into the line cache 1, the output of the line cache 1 can be input into the line cache 2, and the three line caches are all included in the RAM 2.
For example, the device takes the first line horizontal interpolation result and inputs it into line buffer 0. When the second horizontal interpolation result is obtained, the first horizontal interpolation result is output from the line cache 0 and input into the line cache 1, and the second horizontal interpolation result is stored in the line cache 0. When the third line horizontal interpolation result is obtained, the first line horizontal interpolation result is output from the line cache 1 and input into the line cache 2, the second line horizontal interpolation result is output from the line cache 0 and input into the line cache 1, and the third line interpolation result is input into the line cache 0. Then when the device calculates the horizontal interpolation result of the first row and the first column, the horizontal interpolation result of the first row and the first column in the row cache 2, the horizontal interpolation result of the first row and the first column in the row cache 1, the horizontal interpolation result of the first row and the first column in the row cache 0 and the horizontal interpolation result of the first column of the fourth row just calculated are obtained from the output end, and the four horizontal interpolation results are sequentially output from the first column to the last column, and are the four first pixel values for vertical interpolation. And performing vertical interpolation calculation according to the four first pixel values to obtain a vertical interpolation result, namely a second pixel value. The method of calculating the second pixel value may be described with reference to S205.
In one possible implementation, after the result of the above-mentioned vertical interpolation is obtained, registration of the images to be registered is achieved. The device can perform cross-clock domain operation on the image registration result through a first-in first-out memory (First Input First Output, FIFO) so that the clock frequency is the same as that of the DDR, and then the image registration result is read from the FIFO and stored in the DDR.
In addition, when the vertical interpolation is performed, the first pixel value may be insufficient when the vertical interpolation of the elements in the first row and the last two rows in the first coordinate conversion table is calculated. Therefore, when the device performs the vertical interpolation result of the first line element through the FPGA, the edge-trimming operation may be performed on the first pixel value of the first line element, where the line number of the first pixel value of the first line element may be regarded as 2, and the line number of the first pixel value obtained by performing the edge-trimming operation may be regarded as 1. And when the device calculates the vertical interpolation result of the last two rows of elements through the FPGA, performing edge interpolation operation under the first pixel values of the last two rows to finish the vertical interpolation of the last two rows of elements in the first coordinate transformation table. The method is not particularly limited, and the vertical interpolation of the first row of elements and the last two rows of elements in the first coordinate transformation table can be completed.
The technical scheme provided by the application is described below through specific embodiments. Referring to fig. 6, one of exemplary flowcharts of an image registration method according to an embodiment of the present application is provided.
S701, the device performs feature extraction and feature matching on the CPU to obtain a second coordinate transformation table of the target image.
The method for obtaining the second coordinate transformation table on the CPU by the device may be referred to as related description in the method embodiment shown in fig. 2, and will not be described herein.
S702, the device performs downsampling on the second coordinate transformation table on the CPU to obtain a third coordinate transformation table, and sends the third coordinate transformation table to the FPGA.
The method for the device to downsample the second coordinate transformation table on the CPU may refer to the related description in the method embodiment shown in fig. 2, and will not be described herein.
S703, the device performs up-sampling on the third coordinate transformation table on the FPGA to obtain a first coordinate transformation table.
The method for up-sampling the third coordinate transformation table by the device on the FPGA may refer to the related description in the method embodiment shown in fig. 2, which is not described herein.
S704, the device reads all pixel point data of the image to be registered from the DDR through the FPGA and writes the pixel point data into the RAM0.
S705, the device reads all pixel data of the image to be registered in RAM0 through the FPGA and writes into the RAM1_0, the RAM1_1, the RAM1_2, and the RAM1_3 in columns.
S706, the apparatus reads the pixel points from the RAM1_0, RAM1_1, RAM1_2, and RAM1_3 according to the coordinate values on the FPGA and calculates the horizontal interpolation.
The apparatus reads the pixel points from the RAM1_0, RAM1_1, RAM1_2, and RAM1_3 on the FPGA according to the coordinate values on the first coordinate conversion table obtained in S703 and calculates the horizontal interpolation. The method of interpolation in the horizontal direction may be referred to as related description in the method embodiment shown in fig. 2, and will not be described herein.
S707, the device stores the horizontal interpolation result into the RAM2 through the FPGA, and caches three rows of pixel points.
S708, the device reads out data from the RAM2 through the FPGA and performs vertical interpolation.
The method of interpolation in the vertical direction can be referred to as related description in the method embodiment shown in fig. 2, and will not be described herein.
S709, the device stores the interpolation result in the vertical direction into the FIFO for clock domain crossing through the FPGA.
After the interpolation calculation of S708 is completed, the interpolation result in the vertical direction is stored in the FIFO to perform the cross-clock domain operation, so that the clock frequency is the same as the clock frequency of the DDR, so that the image registration result is written into the DDR subsequently.
S710, the device reads out the image registration result from the FIFO through the FPGA and stores the image registration result into the DDR.
And storing the registration result into the DDR so as to facilitate the reading of the device.
Based on the same concept of the above method, referring to fig. 8, for an image registration apparatus 800 provided in an embodiment of the present application, the apparatus 800 is capable of performing each step in the above method, and in order to avoid repetition, details will not be described herein. The apparatus 800 comprises a storage unit 801 and a processing unit 802.
In one scenario:
The storage unit is used for storing a computer program or instructions;
The processing unit is configured to perform the following operations:
Acquiring a first coordinate transformation table of a target image through a Field Programmable Gate Array (FPGA); the first coordinate transformation table comprises N elements, wherein N is a positive integer; one element is the coordinate position of one pixel point in the target image in the image to be registered; the target image and the image to be registered are images aiming at the same object, and the generation conditions of the target image and the image to be registered are different;
Acquiring four pixel points in the image to be registered according to an nth element through the FPGA; the nth element is any element in the first coordinate transformation table, and N is any integer between 1 and N in sequence; four pixel points in the image to be registered are pixel points in the same row in the image to be registered; the row of the four pixel points is determined according to the ordinate of the nth element;
calculating a first pixel value of a pixel point corresponding to an nth element in the image to be registered according to the four pixel points by the FPGA, and sequentially taking N to be any integer between 1 and N;
Acquiring four first pixel values corresponding to the nth element from the N first pixel values through the FPGA; the column numbers of the four first pixel values are the same as the column number of the nth element, the row numbers of the four first pixel values are adjacent, and the row number of the first pixel value in the four first pixel values is the same as the row number of the nth element;
And calculating a second pixel value by the FPGA according to the four first pixel values and the ordinate of the nth element.
In one possible implementation manner, the acquiring, by the FPGA, the first coordinate transformation table of the target image includes:
respectively extracting the characteristics of the image to be registered and the target image through a CPU (Central processing Unit) to obtain a first characteristic point set of the image to be registered and a second characteristic point set of the target image;
determining a characteristic point pair set according to the similarity measurement by the CPU; the characteristic point pair set comprises a plurality of characteristic point pairs, and one characteristic point pair comprises a first characteristic point in the first characteristic point set and a second characteristic point in the second characteristic point set; the similarity measure of the first feature point and the second feature point is greater than or equal to a first threshold;
and determining the first coordinate transformation table according to the characteristic point pair set by the CPU.
In one possible implementation manner, determining the first coordinate transformation table according to the feature point pair set includes:
determining a second coordinate transformation table by the CPU according to the characteristic point pair set;
The CPU performs k times downsampling on the second coordinate transformation table to obtain a third coordinate transformation table; the number of elements in the second coordinate transformation table is k times the number of elements in the third coordinate transformation table; k is an integer greater than 1;
Performing k times up-sampling on the third coordinate transformation table through the FPGA to obtain the first coordinate transformation table; the number of elements in the first coordinate transformation table is the same as the number of elements in the second coordinate transformation table.
In a possible implementation manner, the acquiring, by the FPGA, four pixel points in the image to be registered according to the abscissa of the nth element includes:
Acquiring four pixel points from four RAMs according to the abscissa of the nth element by the FPGA; wherein, each RAM obtains a pixel point; the four RAMs are respectively stored with partial pixel points of the image to be registered; the number of pixel points stored in the four RAMs is the same.
In one possible implementation manner, the 4 m-th column of pixels in the image to be registered is stored in the first RAM, the 4m+1-th column of pixels is stored in the second RAM, the 4m+2-th column of pixels is stored in the third RAM, the 4m+3-th column of pixels is stored in the fourth RAM, and m is an integer greater than or equal to 0.
In a possible implementation manner, after the calculating, by the FPGA, a first pixel value corresponding to an abscissa of the nth element according to the four pixel points, the method further includes:
caching the obtained N first pixel values into a fifth RAM;
The obtaining, by the FPGA, four first pixel values of the N first pixel values according to the ordinate of the nth element includes:
and acquiring four first pixel values in the N first pixel values from the fifth RAM according to the ordinate of the nth element by the FPGA.
In one possible implementation manner, after the processing unit calculates, by using the FPGA, a second pixel value corresponding to an ordinate of the nth element according to the four first pixel values, the processing unit further includes:
Performing clock domain crossing operation on the N second pixel values through a first-in first-out memory FIFO; the clock frequency of the N second pixel values subjected to the cross-clock domain operation is the same as that of the DDR;
And storing the N second pixel values subjected to the cross-clock domain operation in DDR.
Based on the same concept of the above method, referring to fig. 9, an electronic device is provided in an embodiment of the present application, where the electronic device includes a processor 901 and a memory 902. The memory 902 is used for storing computer-executable instructions, and the processor 901 executes the computer-executable instructions in the memory to perform the operational steps of the method in any one of the possible implementations of the method described above using hardware resources in the controller. The processor 901 may be used to perform the operations of the processing unit 802, and the memory 902 may be used to perform the operations of the storage unit 801.
The embodiments of the present application also provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of any of the methods described above.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.
While specific embodiments of the application have been described above, it will be appreciated by those skilled in the art that these are by way of example only, and the scope of the application is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the principles and spirit of the application, but such changes and modifications fall within the scope of the application. While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (16)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111219273.1A CN114037608B (en) | 2021-10-20 | 2021-10-20 | A method for image registration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111219273.1A CN114037608B (en) | 2021-10-20 | 2021-10-20 | A method for image registration |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114037608A CN114037608A (en) | 2022-02-11 |
| CN114037608B true CN114037608B (en) | 2024-11-15 |
Family
ID=80142019
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111219273.1A Active CN114037608B (en) | 2021-10-20 | 2021-10-20 | A method for image registration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114037608B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102547067A (en) * | 2011-12-31 | 2012-07-04 | 中山大学 | Improved bicubic interpolation video scaling method |
| CN108769583A (en) * | 2018-05-24 | 2018-11-06 | 上海大学 | A kind of superfine electric scope high definition interpolating module and method based on FPGA |
| CN113506212A (en) * | 2021-05-21 | 2021-10-15 | 大连海事大学 | Improved POCS-based hyperspectral image super-resolution reconstruction method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4465844B2 (en) * | 2000-09-26 | 2010-05-26 | ソニー株式会社 | Image processing apparatus and method, and recording medium |
| CN102436652B (en) * | 2011-08-31 | 2014-08-27 | 航天恒星科技有限公司 | Automatic registering method of multisource remote sensing images |
| CN103903236B (en) * | 2014-03-10 | 2016-08-31 | 北京信息科技大学 | The method and apparatus of face image super-resolution rebuilding |
| CN110310309B (en) * | 2019-07-09 | 2021-08-31 | 中国电子科技集团公司第十三研究所 | Image registration method, image registration device and terminal |
| CN112862866B (en) * | 2021-04-13 | 2024-08-20 | 湖北工业大学 | Image registration method and system based on sparrow search algorithm and computing equipment |
-
2021
- 2021-10-20 CN CN202111219273.1A patent/CN114037608B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102547067A (en) * | 2011-12-31 | 2012-07-04 | 中山大学 | Improved bicubic interpolation video scaling method |
| CN108769583A (en) * | 2018-05-24 | 2018-11-06 | 上海大学 | A kind of superfine electric scope high definition interpolating module and method based on FPGA |
| CN113506212A (en) * | 2021-05-21 | 2021-10-15 | 大连海事大学 | Improved POCS-based hyperspectral image super-resolution reconstruction method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114037608A (en) | 2022-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210174471A1 (en) | Image Stitching Method, Electronic Apparatus, and Storage Medium | |
| JP6789402B2 (en) | Method of determining the appearance of an object in an image, equipment, equipment and storage medium | |
| US12148123B2 (en) | Multi-stage multi-reference bootstrapping for video super-resolution | |
| CN108073857B (en) | Dynamic visual sensor DVS event processing method and device | |
| CN108875900B (en) | Video image processing method and device, neural network training method, storage medium | |
| WO2019042404A1 (en) | Image processing method, terminal, and storage medium | |
| US9824486B2 (en) | High resolution free-view interpolation of planar structure | |
| WO2019238114A1 (en) | Three-dimensional dynamic model reconstruction method, apparatus and device, and storage medium | |
| US10621446B2 (en) | Handling perspective magnification in optical flow processing | |
| WO2018082185A1 (en) | Image processing method and device | |
| CN111507333B (en) | Image correction method and device, electronic equipment and storage medium | |
| WO2014045026A1 (en) | Systems and methods for reducing noise in video streams | |
| CN109743562B (en) | Matching cost calculation circuit structure based on Census algorithm and working method thereof | |
| CN113076685A (en) | Training method of image reconstruction model, image reconstruction method and device thereof | |
| KR102697687B1 (en) | Method of merging images and data processing device performing the same | |
| WO2021017589A1 (en) | Image fusion method based on gradient domain mapping | |
| WO2020238484A1 (en) | Method and apparatus for detecting object in image, and vehicle and robot | |
| EP4292059A1 (en) | Multiview neural human prediction using implicit differentiable renderer for facial expression, body pose shape and clothes performance capture | |
| WO2024002211A1 (en) | Image processing method and related apparatus | |
| CN117612153A (en) | Three-dimensional target recognition and positioning method based on image and point cloud information completion | |
| CN117173070A (en) | Image processing fusion method and system based on FPGA | |
| CN113362338A (en) | Rail segmentation method, device, computer equipment and rail segmentation processing system | |
| CN116704200A (en) | Image feature extraction and image noise reduction method and related device | |
| CN115564639A (en) | Background blurring method and device, computer equipment and storage medium | |
| CN114037608B (en) | A method for image registration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |