[go: up one dir, main page]

CN108090865B - On-orbit real-time streaming processing method and system for optical satellite remote sensing images - Google Patents

On-orbit real-time streaming processing method and system for optical satellite remote sensing images Download PDF

Info

Publication number
CN108090865B
CN108090865B CN201711354960.8A CN201711354960A CN108090865B CN 108090865 B CN108090865 B CN 108090865B CN 201711354960 A CN201711354960 A CN 201711354960A CN 108090865 B CN108090865 B CN 108090865B
Authority
CN
China
Prior art keywords
processing
data
time
module
real
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711354960.8A
Other languages
Chinese (zh)
Other versions
CN108090865A (en
Inventor
王密
张致齐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Land Sea Space Yantai Information Technology Co ltd
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN201711354960.8A priority Critical patent/CN108090865B/en
Publication of CN108090865A publication Critical patent/CN108090865A/en
Application granted granted Critical
Publication of CN108090865B publication Critical patent/CN108090865B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/20Processor architectures; Processor configuration, e.g. pipelining
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5044Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5017Task decomposition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10032Satellite or aerial image; Remote sensing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Image Input (AREA)

Abstract

本发明提供一种光学卫星遥感影像在轨实时流式处理方法及系统,包括构建单处理模组的处理流水线,分别测定数据流入、数据处理、数据流出的速度,记录相应耗时;估计要达到实时处理性能所需要的处理模组数N,制定并配置数据动态分包策略,根据不同处理算法需要实现配置,包括初步划分为连续的标准景,根据需要支持标准景内部进一步划分为连续的数据段,根据需要支持数据段内更进一步地划分为并列的格网、超像素或像素;根据硬件能力确定负载均衡策略,部署定制化流式处理架构以实现光学卫星遥感影像在轨实时处理。该方法可以充分利用在轨处理模组的处理能力与处理算法本身的特性,最优化在轨数据处理流程,提高遥感信息获取效率和实时性。

Figure 201711354960

The invention provides a method and system for in-orbit real-time streaming processing of optical satellite remote sensing images, including constructing a processing pipeline of a single processing module, respectively measuring the speed of data inflow, data processing and data outflow, and recording the corresponding time-consuming; The number of processing modules N required for real-time processing performance, formulate and configure data dynamic subcontracting strategies, and implement configurations according to different processing algorithms, including initial division into continuous standard scenes, and support for further division of standard scenes into continuous data as needed The data segment can be further divided into parallel grids, superpixels or pixels according to the needs; the load balancing strategy is determined according to the hardware capabilities, and the customized streaming processing architecture is deployed to realize the real-time processing of optical satellite remote sensing images in orbit. The method can make full use of the processing capability of the on-orbit processing module and the characteristics of the processing algorithm itself, optimize the on-orbit data processing flow, and improve the efficiency and real-time performance of remote sensing information acquisition.

Figure 201711354960

Description

Optical satellite remote sensing image on-orbit real-time streaming processing method and system
Technical Field
The invention belongs to the field of high-performance processing of space remote sensing images, and relates to an on-orbit real-time processing method and system of multi-processing module optical satellite remote sensing images based on intelligent data subpackaging and load balancing.
Background
The traditional optical remote sensing satellite information acquisition link comprises four main links of satellite acquisition data-satellite data downloading-ground centralized processing-information extraction, and with the rapid development of a high-resolution earth observation satellite, explosively increased remote sensing data brings huge pressure to the data downloading link and a ground processing system. In general, the time delay from satellite data generation to ground effective information acquisition is tens of minutes, and the information acquisition timeliness is low. In order to fully utilize the limited transmission bandwidth and the satellite transit time window and shorten the information acquisition delay, a secondary information acquisition link of satellite acquisition and processing data and effective information downloading and distribution needs to be established, namely, massive original data needs to be processed in an on-orbit real-time manner so as to directly generate and download effective information. Because the on-orbit processing platform has volume and power consumption limitations and limited performance, on one hand, a plurality of groups of embedded processing platforms with low power consumption and high performance need to be adopted to cooperatively process data to realize on-orbit real-time processing; on the other hand, it is necessary to fully parallelize the computation and fully exploit the performance of the computation storage means.
Disclosure of Invention
The invention aims to solve the problem of providing a streaming processing technical scheme suitable for in-orbit real-time processing of optical satellite remote sensing images aiming at a plurality of groups of low-power-consumption embedded in-orbit processing modules.
The technical scheme of the invention provides an on-orbit real-time streaming processing method for optical satellite remote sensing images, which comprises the following steps:
step 1, constructing a processing assembly line of a single processing module, respectively measuring the speed of data inflow, data processing and data outflow, and recording corresponding time consumption;
step 2, estimating the number N of processing modules needed to achieve real-time processing performance as follows,
Figure BDA0001510924600000011
wherein, TIN,TProcAnd TOUTRespectively measuring the time consumed by data inflow, data processing and data outflow in the step 1;
step 3, formulating and configuring a data dynamic subpackage strategy according to the result of the step 2, realizing configuration according to different processing algorithm requirements, and including preliminarily dividing into continuous standard scenes, supporting the interior of the standard scenes to be further divided into continuous data segments according to requirements, and further dividing into parallel grids, super pixels or pixels in the data segments according to requirements;
step 4, determining a load balancing strategy according to the hardware capacity;
step 5, deploying a customized streaming processing framework to realize on-orbit real-time processing of the optical satellite remote sensing image, wherein the on-orbit real-time processing comprises deploying N processing modules based on the processing module number N obtained in the step 2; and (4) performing real-time subpackaging and distribution on continuously inflowing input according to the data subpackaging strategy formulated in the step (3) and the load balancing strategy determined in the step (4) so as to realize parallel processing among a plurality of processing modules and realize on-orbit real-time processing of the optical satellite remote sensing images.
Moreover, the configuration in step 3 is implemented according to different processing algorithm requirements, including but not limited to the strategies of dividing according to a standard scene, dividing according to a grid, dividing according to a super-pixel, and dividing according to a pixel.
And, the load balancing policy in step 4 includes, but is not limited to, a policy of sequentially and evenly distributing data packets and a policy of weighting and distributing data packets according to different processing capabilities of modules according to different hardware capabilities and processing algorithm requirements.
And the strategy of weighted distribution according to the processing capacities of different modules is to set weights according to the processing capacities of the processing modules, and the modules with strong processing capacities and large weights are distributed with more to-be-processed data packets.
Moreover, the processing capability weights of the single processing module1 to the single processing module N are respectively W1,W2,…,WNThen, there are:
Figure BDA0001510924600000021
generating C data packets in unit time, the number of data packets processed by each processing module is Wi·。
The invention also provides an on-orbit real-time streaming processing system for the optical satellite remote sensing image, which comprises the following modules:
the first module is used for constructing a processing assembly line of the single processing module, respectively measuring the speed of data inflow, data processing and data outflow and recording corresponding consumed time;
a second module for estimating a number of processing modules N required to achieve real-time processing performance as follows,
Figure BDA0001510924600000022
wherein, TIN,TProcAnd TOUTRespectively measuring the time consumed by data inflow, data processing and data outflow in the step 1;
the third module is used for formulating and configuring a data dynamic subpackage strategy according to the result obtained by the second module, realizing configuration according to different processing algorithm requirements, and comprises the steps of preliminarily dividing the data into continuous standard scenes, further dividing the interior of the standard scenes into continuous data sections according to requirements, and further dividing the interior of the data sections into parallel grids, super pixels or pixels according to requirements;
a fourth module for determining a load balancing policy according to the hardware capability;
the fifth module is used for deploying a customized streaming processing architecture to realize on-orbit real-time processing of the optical satellite remote sensing image, and comprises a processing module number N obtained based on the second module and N processing modules; and performing real-time subpackaging and distribution on continuously inflowing input according to a data subpackaging strategy formulated by the third module and a load balancing strategy determined by the fourth module so as to realize parallel processing among a plurality of processing modules and realize on-orbit real-time processing of the optical satellite remote sensing images.
Moreover, the third module implements configurations according to different processing algorithm requirements, including but not limited to strategies of dividing according to a standard scene, dividing according to a grid, dividing according to super-pixels, and dividing according to pixels.
And the load balancing strategy in the fourth module includes, but is not limited to, strategies of sequentially and evenly distributing data packets and weighting and distributing according to different module processing capacities according to different hardware capacities and processing algorithm requirements.
And the strategy of weighted distribution according to the processing capacities of different modules is to set weights according to the processing capacities of the processing modules, and the modules with strong processing capacities and large weights are distributed with more to-be-processed data packets.
Moreover, the processing capability weights of the single processing module1 to the single processing module N are respectively W1,W2,…,WNThen, there are:
Figure BDA0001510924600000031
generating C data packets per unit timeThen the number of data packets processed by each processing module is Wi·。
The streaming processing framework suitable for the on-orbit processing of the optical satellite remote sensing image can fully utilize the processing capacity of the on-orbit processing module and the characteristics of the processing algorithm, optimize the on-orbit data processing flow, realize the on-orbit real-time processing of the optical satellite remote sensing image of 'data inflow-data processing-data outflow', and improve the acquisition efficiency and the real-time performance of remote sensing information. The processing algorithm applicable to the invention comprises and is not limited to optical satellite sensor correction, target detection, cloud detection and the like, and has important market value.
Drawings
FIG. 1 is a schematic flow chart of an embodiment of the present invention;
FIG. 2 is a schematic diagram of a data processing flow according to an embodiment of the present invention;
fig. 3 is a schematic diagram of data dynamic packetization according to an embodiment of the present invention.
Detailed Description
The technical scheme of the invention is explained in detail in the following by combining the drawings and the embodiment.
The invention provides an in-orbit real-time streaming processing architecture of an optical satellite remote sensing image, which comprises the steps of determining the time consumed by data inflow, data processing and data outflow in a single processing module aiming at the characteristics of an in-orbit processing algorithm and the characteristics of processing hardware of a specific optical satellite remote sensing image; further calculating the number of processing modules required for achieving the real-time processing performance; and combining the characteristics of an on-orbit processing algorithm to formulate and configure a data dynamic subpackage strategy so as to generate a customized streaming architecture combining software and hardware; and deploying a customized streaming framework to realize on-orbit real-time processing of the optical satellite remote sensing image.
As shown in fig. 1, an in-orbit real-time streaming processing method for optical satellite remote sensing images provided in an embodiment includes the following steps:
step 1, constructing a processing assembly line of a single processing module, respectively measuring the speed of data inflow, data processing and data outflow, and recording corresponding time consumption.
Here, the processing module refers to a basic data processing unit including functional modules for processing, storing, communicating, and the like, and includes but is not limited to a processing board and an SoC system using an FPGA and a DSP as core devices. Based on the processing module, the incoming data can be processed through programming, and the processing result is transmitted out. In the processing process, for a specific data block, three main steps of data inflow, data processing and data outflow must be executed in a strict sequence; for a plurality of data blocks, the three steps can be processed by constructing a pipeline stack, that is, for data A, B, C which flows in successively, data B can be processed while data C flows in, and the processing result of data a is output. Due to the inconsistency of the time consumption of the three steps of data inflow, data processing and data outflow, the execution efficiency of the whole process depends on the step (usually data processing) which takes the longest time.
The invention realizes on-track parallel real-time processing by reasonably sub-packaging the input data and uniformly distributing the input data to the multi-processing modules. Aiming at specific processing algorithms such as optical satellite sensor correction, target detection, cloud detection and the like, the ratio of the processing rate of the target algorithm to the data input and output rate needs to be measured first, and then the minimum number of processing modules needed to be used is obtained.
Combining the hardware characteristics of the processing module, the target algorithm needs to be efficiently mapped to the processing module, and the average values of the consumed time of data inflow, data processing and data outflow are obtained by executing a plurality of continuous tests on the single-processing module pipeline and are respectively marked as TIN,TProcAnd TOUT
And 2, calculating the number of the processing modules required for achieving the real-time processing performance, namely the total number of the single processing modules. The ratio of the data processing speed to the data inflow/outflow speed (the slowest of the two) can be calculated and rounded up. The invention further proposes a preferred scheme, and the number of the processing modules is N, and the value is obtained by the following formula:
Figure BDA0001510924600000041
wherein T isIN,TProcAnd TOUTAre respectively asAnd executing a plurality of groups of tests on the single processing module to obtain the average value of the consumed time of data inflow, data processing and data outflow.
The step is independent mathematical calculation based on the result of the step 1, and is independent of processing environments such as specific software and hardware.
And 3, formulating and configuring a data dynamic subpackage strategy according to the result of the step 2, wherein the strategy comprises the strategies of dividing according to a standard scene, dividing according to a grid, dividing according to a super pixel, dividing according to a pixel and the like.
In the field of optical satellite remote sensing image data processing, a square or nearly square image product is output according to the imaging width of a satellite, and the image product is called as a standard scene. As shown in fig. 2, the left side is the image data to be processed, which are sequentially transmitted (flowed) from top to bottom, and are initially divided into continuous standard scenes; inside the standard scene, the standard scene can be further divided into continuous data segments (segment 0, segment 1, segment 2, segment 3, etc.); further, the data segments may be divided into parallel grids, superpixels, or pixels. The right side is a parallel multi-processing module, and the processing module is internally composed of a control core (such as an embedded CPU, a DSP and the like) suitable for processing scalar operations such as control, branching, circulation, iteration and the like and a parallel core (such as an embedded GPU, an FPGA and the like) suitable for processing single instruction multiple data vector operations. In fig. 2, the left divided data segment 0, 1, 2, 3 is distributed to modules 0, 1, 2, 3(Module0, Module1, Module2, Module3) for processing, and the data segment is further divided into grids and processed with the control core and the parallel core.
The method adopts different sub-packaging strategies for different types of optical remote sensing image processing algorithms, sub-packages the inflow data according to the whole scene granularity for the algorithm needing the whole scene processing, such as the correction of an optical satellite sensor, and distributes the data to different processing modules one by one for processing; for algorithms such as target identification and cloud detection, a standard scene needs to be further divided into a plurality of approximately equal data segments, the data segments are sequentially distributed to different data processing modules section by section for processing, the data segments are further divided into grids, superpixels or pixels in the processing modules, and a control core and a parallel core are called to perform collaborative parallel computing according to the main operation type of a specific processing algorithm (different operation cores are suitable for processing different types of operations).
Based on the processing module number N required for realizing real-time calculation obtained in the step 2, the specific data subpackaging strategy is formulated by combining the characteristics of different types of processing algorithms. The sub-packaging strategy is closely related to the characteristics of the processing algorithm, and is not related to the configuration and the computing capacity of a single processing module. For example, for an algorithm needing to consider the relation between data segments, the data packet is distributed to N processing modules for processing by taking the whole scene as a unit; for the algorithm capable of processing independently among the data segments, the data packets are distributed to N processing modules by taking the data segments as units, and the data segments are further divided into grids which are distributed to different cores in a single module for processing.
And 4, determining a load balancing strategy according to the hardware capacity.
The load balancing strategy comprises the strategies of sequentially and evenly distributing data packets and weighting and distributing according to different module processing capacities according to different hardware capacities and processing algorithm requirements without limitation.
The embodiment sets the weight according to the processing capacity of the processing module, and the module group with strong processing capacity and large weight is allocated with more data packets to be processed, so that each processing module is fully utilized, the overall execution efficiency is improved, and node overload or node starvation is avoided. The processing capability weights of the single processing module1 to the single processing module N are respectively W1,W2,…,WNThen, there are:
Figure BDA0001510924600000051
generating C data packets in unit time, the number of data packets processed by each processing module is Wi·。
And 5, deploying a customized streaming processing framework to realize on-orbit real-time processing of the optical satellite remote sensing image. As shown in fig. 3, based on the number N of processing modules obtained in step 2, N processing modules (1-N) are deployed; the data distribution module performs real-time sub-packaging and distribution on continuously flowing input according to the data sub-packaging strategy formulated in the step 3 and the load balancing strategy determined in the step 4 so as to realize parallel processing among a plurality of processing modules; inside the processing module, scalar operation and vector operation are cooperatively processed by the control core and the parallel core according to the characteristics of the processing algorithm, so that the on-orbit real-time processing of the optical satellite remote sensing image is realized.
Taking a time line for realizing real-time cloud detection processing production of the optical satellite remote sensing image by applying the method, from the original point, the camera spends 0.403 seconds to generate data of 'data 0 segment 0', and the data segment is distributed to the module0 for processing and consumes 1.439 seconds; while the camera continues to spend 0.403 seconds generating "data 0 segment 1" data, which is assigned to module1 for processing, taking 1.501 seconds; and so on. Therefore, through a reasonable data sub-packaging and balancing strategy, the data can be immediately processed after being generated, and the whole system does not have the phenomena of waiting and data accumulation. And 1.417 seconds after the data generation is finished, the data processing is finished, and the real-time cloud detection processing of the optical satellite remote sensing image is realized.
In specific implementation, the method provided by the invention can realize automatic operation flow based on software technology, and can also realize a corresponding system in a modularized mode.
The embodiment of the invention also provides an on-orbit real-time streaming processing system for the optical satellite remote sensing image, which comprises the following modules:
the first module is used for constructing a processing assembly line of the single processing module, respectively measuring the speed of data inflow, data processing and data outflow and recording corresponding consumed time;
a second module for estimating a number of processing modules N required to achieve real-time processing performance as follows,
Figure BDA0001510924600000061
wherein, TIN,TProcAnd TOUTRespectively measuring the time consumed by data inflow, data processing and data outflow in the step 1;
the third module is used for formulating and configuring a data dynamic subpackage strategy according to the result obtained by the second module, realizing configuration according to different processing algorithm requirements, and comprises the steps of preliminarily dividing the data into continuous standard scenes, further dividing the interior of the standard scenes into continuous data sections according to requirements, and further dividing the interior of the data sections into parallel grids, super pixels or pixels according to requirements;
a fourth module for determining a load balancing policy according to the hardware capability;
the fifth module is used for deploying a customized streaming processing architecture to realize on-orbit real-time processing of the optical satellite remote sensing image, and comprises a processing module number N obtained based on the second module and N processing modules; and performing real-time subpackaging and distribution on continuously inflowing input according to a data subpackaging strategy formulated by the third module and a load balancing strategy determined by the fourth module so as to realize parallel processing among a plurality of processing modules and realize on-orbit real-time processing of the optical satellite remote sensing images.
The specific implementation of each module can refer to the corresponding step, and the detailed description of the invention is omitted.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.

Claims (8)

1.一种光学卫星遥感影像在轨实时流式处理方法,其特征在于,包括以下步骤:1. an optical satellite remote sensing image on-orbit real-time stream processing method, is characterized in that, comprises the following steps: 步骤1,构建单处理模组的处理流水线,分别测定数据流入、数据处理、数据流出的速度,记录相应耗时;Step 1, build a processing pipeline of a single processing module, measure the speed of data inflow, data processing, and data outflow respectively, and record the corresponding time-consuming; 步骤2,估计要达到实时处理性能所需要的处理模组数N如下式,Step 2, estimate the number of processing modules N required to achieve real-time processing performance as follows:
Figure FDA0003015164500000011
Figure FDA0003015164500000011
其中,TIN,TProc和TOUT分别为步骤1中测定的数据流入、数据处理、数据流出耗时;Wherein, T IN , T Proc and T OUT are respectively the data inflow, data processing, and data outflow time-consuming measured in step 1; 步骤3,根据步骤2结果制定并配置数据动态分包策略,根据不同处理算法需要实现配置,包括初步划分为连续的标准景,根据需要支持标准景内部进一步划分为连续的数据段,根据需要支持数据段内更进一步地划分为并列的格网、超像素或像素;Step 3: According to the result of step 2, formulate and configure the data dynamic subcontracting strategy, and realize the configuration according to the needs of different processing algorithms, including initial division into continuous standard scenes, and further division of the standard scene into continuous data segments according to needs, and support according to needs. The data segment is further divided into juxtaposed grids, superpixels or pixels; 步骤4,根据硬件能力确定负载均衡策略;Step 4: Determine the load balancing strategy according to the hardware capability; 设单处理模组1到单处理模组N的处理能力权重分别为W1,W2,...,WN,则有:Assuming that the processing capability weights of the single processing module 1 to the single processing module N are W 1 , W 2 , . . . , W N respectively, there are:
Figure FDA0003015164500000012
Figure FDA0003015164500000012
单位时间内产生C个数据包,则每个处理模组处理的数据包数为Wi·C;If C data packets are generated per unit time, the number of data packets processed by each processing module is W i ·C; 步骤5,部署定制化流式处理架构以实现光学卫星遥感影像在轨实时处理,包括基于步骤2求得的处理模组数N,部署N个处理模组;按照步骤3制定的数据分包策略和步骤4确定的负载均衡策略,对持续流入的输入进行实时分包与分发,以实现多个处理模组之间的并行处理,实现光学卫星遥感影像的在轨实时处理。Step 5, deploying a customized streaming processing architecture to realize real-time processing of optical satellite remote sensing images on-orbit, including deploying N processing modules based on the number of processing modules N obtained in Step 2; according to the data subcontracting strategy formulated in Step 3 With the load balancing strategy determined in step 4, real-time subcontracting and distribution of the continuously inflowing input is performed to realize parallel processing among multiple processing modules, and realize on-orbit real-time processing of optical satellite remote sensing images.
2.根据权利要求1所述光学卫星遥感影像在轨实时流式处理方法,其特征在于:步骤3中根据不同处理算法需要实现配置,包括且不限于按照标准景划分、按照格网划分、按照超像素划分、按照像素划分的策略。2. according to the described optical satellite remote sensing image in-orbit real-time streaming processing method of claim 1, it is characterized in that: in step 3, need to realize configuration according to different processing algorithms, including but not limited to according to standard scene division, according to grid division, according to Superpixel division, according to the pixel division strategy. 3.根据权利要求1或2所述光学卫星遥感影像在轨实时流式处理方法,其特征在于:步骤4中的负载均衡策略,根据不同硬件能力和处理算法需要,包括且不限于数据包依次平均分发、根据不同模组处理能力加权分发的策略。3. according to the described optical satellite remote sensing image in-orbit real-time streaming method of claim 1 or 2, it is characterized in that: the load balancing strategy in step 4, according to different hardware capabilities and processing algorithm needs, including but not limited to data packets sequentially A strategy of average distribution and weighted distribution according to the processing capabilities of different modules. 4.根据权利要求3所述光学卫星遥感影像在轨实时流式处理方法,其特征在于:所述根据不同模组处理能力加权分发的策略,是按照处理模组处理能力设置权重,处理能力强、权重大的模组分配更多的待处理数据包。4. The method for on-orbit real-time streaming processing of optical satellite remote sensing images according to claim 3, wherein: the described strategy of weighted distribution according to the processing capabilities of different modules is to set weights according to the processing capabilities of the processing modules, and the processing capabilities are strong , Modules with heavier weights allocate more packets to be processed. 5.一种光学卫星遥感影像在轨实时流式处理系统,其特征在于,包括以下模块:5. An on-orbit real-time stream processing system for optical satellite remote sensing images, comprising the following modules: 第一模块,用于构建单处理模组的处理流水线,分别测定数据流入、数据处理、数据流出的速度,记录相应耗时;The first module is used to construct a processing pipeline of a single processing module, respectively measure the speed of data inflow, data processing, and data outflow, and record the corresponding time-consuming; 第二模块,用于估计要达到实时处理性能所需要的处理模组数N如下式,The second module is used to estimate the number of processing modules N required to achieve real-time processing performance as follows:
Figure FDA0003015164500000021
Figure FDA0003015164500000021
其中,TIN,TProc和TOUT分别为第一模块中测定的数据流入、数据处理、数据流出耗时;Wherein, T IN , T Proc and T OUT are respectively the data inflow, data processing, and data outflow time-consuming measured in the first module; 第三模块,用于根据第二模块所得结果制定并配置数据动态分包策略,根据不同处理算法需要实现配置,包括初步划分为连续的标准景,根据需要支持标准景内部进一步划分为连续的数据段,根据需要支持数据段内更进一步地划分为并列的格网、超像素或像素;The third module is used to formulate and configure the data dynamic subcontracting strategy according to the results obtained by the second module, and realize the configuration according to the needs of different processing algorithms, including initial division into continuous standard scenes, and support for further division of the standard scene into continuous data as needed segments, supporting further division of data segments into juxtaposed grids, superpixels, or pixels as needed; 第四模块,用于根据硬件能力确定负载均衡策略;The fourth module is used to determine the load balancing strategy according to the hardware capability; 设单处理模组1到单处理模组N的处理能力权重分别为W1,W2,...,WN,则有:Assuming that the processing capability weights of the single processing module 1 to the single processing module N are W 1 , W 2 , . . . , W N respectively, there are:
Figure FDA0003015164500000022
Figure FDA0003015164500000022
单位时间内产生C个数据包,则每个处理模组处理的数据包数为Wi·C;If C data packets are generated per unit time, the number of data packets processed by each processing module is W i ·C; 第五模块,用于部署定制化流式处理架构以实现光学卫星遥感影像在轨实时处理,包括基于第二模块求得的处理模组数N,部署N个处理模组;按照第三模块制定的数据分包策略和第四模块确定的负载均衡策略,对持续流入的输入进行实时分包与分发,以实现多个处理模组之间的并行处理,实现光学卫星遥感影像的在轨实时处理。The fifth module is used to deploy a customized streaming processing architecture to realize real-time processing of optical satellite remote sensing images on-orbit, including deploying N processing modules based on the number N of processing modules obtained from the second module; formulating according to the third module The data subcontracting strategy and the load balancing strategy determined by the fourth module perform real-time subcontracting and distribution of the continuously inflowing input to realize parallel processing among multiple processing modules and realize on-orbit real-time processing of optical satellite remote sensing images. .
6.根据权利要求5所述光学卫星遥感影像在轨实时流式处理系统,其特征在于:第三模块中根据不同处理算法需要实现配置,包括且不限于按照标准景划分、按照格网划分、按照超像素划分、按照像素划分的策略。6. The in-orbit real-time streaming processing system of optical satellite remote sensing images according to claim 5, is characterized in that: the third module needs to realize configuration according to different processing algorithms, including but not limited to dividing according to standard scene, dividing according to grid, The strategy of dividing by superpixels and dividing by pixels. 7.根据权利要求5或6所述光学卫星遥感影像在轨实时流式处理系统,其特征在于:第四模块中的负载均衡策略,根据不同硬件能力和处理算法需要,包括且不限于数据包依次平均分发、根据不同模组处理能力加权分发的策略。7. The on-orbit real-time streaming processing system of optical satellite remote sensing images according to claim 5 or 6, wherein: the load balancing strategy in the fourth module, according to different hardware capabilities and processing algorithm needs, includes but is not limited to data packets A strategy of uniform distribution in turn and weighted distribution according to the processing capabilities of different modules. 8.根据权利要求7所述光学卫星遥感影像在轨实时流式处理系统,其特征在于:所述根据不同模组处理能力加权分发的策略,是按照处理模组处理能力设置权重,处理能力强、权重大的模组分配更多的待处理数据包。8. The on-orbit real-time streaming processing system of optical satellite remote sensing images according to claim 7, characterized in that: the strategy for weighted distribution according to the processing capabilities of different modules is to set weights according to the processing capabilities of the processing modules, and the processing capabilities are strong. , Modules with heavier weights allocate more packets to be processed.
CN201711354960.8A 2017-12-15 2017-12-15 On-orbit real-time streaming processing method and system for optical satellite remote sensing images Active CN108090865B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711354960.8A CN108090865B (en) 2017-12-15 2017-12-15 On-orbit real-time streaming processing method and system for optical satellite remote sensing images

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711354960.8A CN108090865B (en) 2017-12-15 2017-12-15 On-orbit real-time streaming processing method and system for optical satellite remote sensing images

Publications (2)

Publication Number Publication Date
CN108090865A CN108090865A (en) 2018-05-29
CN108090865B true CN108090865B (en) 2021-06-15

Family

ID=62176588

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711354960.8A Active CN108090865B (en) 2017-12-15 2017-12-15 On-orbit real-time streaming processing method and system for optical satellite remote sensing images

Country Status (1)

Country Link
CN (1) CN108090865B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112165572A (en) * 2020-09-10 2021-01-01 Oppo(重庆)智能科技有限公司 Image processing method, device, terminal and storage medium
CN113283412B (en) * 2021-07-26 2021-09-21 广州乐盈信息科技股份有限公司 On-orbit real-time streaming processing method for optical satellite remote sensing image

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006115283A (en) * 2004-10-15 2006-04-27 Mitsubishi Electric Corp Data compressor and data expanding device
JP2007293558A (en) * 2006-04-25 2007-11-08 Hitachi Ltd Target recognition program and target recognition apparatus
CN102521687A (en) * 2011-12-01 2012-06-27 中国资源卫星应用中心 Miniaturized universal platform for preprocessing remote-sensing satellite data
CN103020620A (en) * 2012-12-14 2013-04-03 武汉大学 Remote-sensing image ortho-rectification method based on central processing unit (CPU) and graphics processing unit (GPU) coprocessing
CN103796018A (en) * 2014-01-14 2014-05-14 齐齐哈尔大学 Remote-sensing image real-time compression and progressive transmission system
CN103809969A (en) * 2014-01-15 2014-05-21 中国公路工程咨询集团有限公司 Remote-sensing image data parallel resampling method based on pre-fragmentation in cloud environment
CN106909644A (en) * 2017-02-21 2017-06-30 中国科学院电子学研究所 A kind of multistage tissue and indexing means towards mass remote sensing image

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013020143A1 (en) * 2011-08-04 2013-02-07 University Of Southern California Image-based crack quantification
CN105430378A (en) * 2015-11-26 2016-03-23 航天恒星科技有限公司 An image quality evaluation system and method
CN106846292B (en) * 2016-12-14 2019-08-27 中国资源卫星应用中心 A Method for On-orbit Detection of Invalid Pixels
CN107391258B (en) * 2017-07-20 2020-09-22 武汉大学 Software and hardware integrated portable remote sensing image real-time processing system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006115283A (en) * 2004-10-15 2006-04-27 Mitsubishi Electric Corp Data compressor and data expanding device
JP2007293558A (en) * 2006-04-25 2007-11-08 Hitachi Ltd Target recognition program and target recognition apparatus
CN102521687A (en) * 2011-12-01 2012-06-27 中国资源卫星应用中心 Miniaturized universal platform for preprocessing remote-sensing satellite data
CN103020620A (en) * 2012-12-14 2013-04-03 武汉大学 Remote-sensing image ortho-rectification method based on central processing unit (CPU) and graphics processing unit (GPU) coprocessing
CN103796018A (en) * 2014-01-14 2014-05-14 齐齐哈尔大学 Remote-sensing image real-time compression and progressive transmission system
CN103809969A (en) * 2014-01-15 2014-05-21 中国公路工程咨询集团有限公司 Remote-sensing image data parallel resampling method based on pre-fragmentation in cloud environment
CN106909644A (en) * 2017-02-21 2017-06-30 中国科学院电子学研究所 A kind of multistage tissue and indexing means towards mass remote sensing image

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ZY-3卫星轨道拟合与预报精度分析;向夏芸 等;《测绘通报》;20151231(第1期);第8-14页 *
大型遥感影像数据库的空间无缝数据组织;王密 等;《武汉大学学报信息科学版》;20011031;第26卷(第5期);第419-424页 *

Also Published As

Publication number Publication date
CN108090865A (en) 2018-05-29

Similar Documents

Publication Publication Date Title
KR102492477B1 (en) Matrix multiplier
CN107301456B (en) Implementation method of multi-core acceleration of deep neural network based on vector processor
US9099866B2 (en) Apparatus, methods and systems for parallel power flow calculation and power system simulation
CN108875958A (en) Use the primary tensor processor of outer product unit
KR20180052063A (en) Convolution neural network system and operation method thereof
KR102610842B1 (en) Processing element and operating method thereof in neural network
CN111931917A (en) Forward computing implementation method and device, storage medium and electronic device
Parthasarathy et al. Defer: Distributed edge inference for deep neural networks
KR20220015813A (en) Method and apparatus for performing deep learning operations.
Schmidt et al. A generic VHDL template for 2D stencil code applications on FPGAs
CN113010845B (en) Computing device, method and related product for performing matrix multiplication
CN112199636A (en) Fast convolution method and device suitable for microprocessor
CN113486298A (en) Model compression method and matrix multiplication module based on Transformer neural network
CN113592066B (en) Hardware acceleration method, device, equipment and storage medium
CN108090865B (en) On-orbit real-time streaming processing method and system for optical satellite remote sensing images
CN111861793A (en) Distribution and utilization electric service distribution method and device based on cloud edge cooperative computing architecture
CN108537334A (en) A kind of acceleration array design methodology for CNN convolutional layer operations
CN116962176A (en) Data processing method, device and system of distributed cluster and storage medium
CN108304926A (en) A kind of pond computing device and method suitable for neural network
CN111831354A (en) Data precision configuration method, device, chip array, equipment and medium
TW202316325A (en) Parallel depth-wise processing architectures for neural networks
CN108875957B (en) Primary tensor processor and the system for using primary tensor processor
CN109190755A (en) Matrix conversion device and method towards neural network
CN113052299A (en) Neural network memory computing device based on lower communication bound and acceleration method
US11682099B2 (en) Hardware accelerator for integral image computation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20240108

Address after: No. 120, Haixiang Middle Road, Fengcheng Street, Haiyang City, Yantai City, Shandong Province, 265100

Patentee after: Land sea space (Yantai) Information Technology Co.,Ltd.

Address before: 430072 Hubei Province, Wuhan city Wuchang District of Wuhan University Luojiashan

Patentee before: WUHAN University