WO2019104558A1 - Image processing method, photography equipment, unmanned aerial vehicle and ground end device - Google Patents

Abstract

Provided in an embodiment of the present invention are an image processing method, a photography equipment, an unmanned aerial vehicle (UAV), and a ground end device. The method comprises: a main processor acquires image data; the main processor encodes the image data to obtain image encoded data; and the main processor transmits the image encoded data to a slave processor to enable the slave processor to perform distortion calibration on the image data on the basis of the image encoded data. In the embodiment of the present invention, the image data is acquired by the main processor, and the image data is encoded to obtain the image encoded data, then the main processor transmits the image encoded data to a slave processor to enable the slave processor to perform distortion calibration on the image data on the basis of the image encoded data. The main processor does not need to perform distortion calibration on the image data immediately after the image data is acquired. Therefore, the performance overhead of the main processor is saved. When the performance of the main processor is insufficient, and when the code stream pixel of the image data is large, problems on the bandwidth and performance of the main processor can be effectively avoided.

Description

Image processing method, photographing equipment, drone and ground end equipment Technical field

The embodiments of the present invention relate to the field of drones, and in particular, to an image processing method, a photographing device, a drone, and a ground end device.

Background technique

In the prior art, the drone is equipped with a photographing device through the pan/tilt and controls the photographing device to perform aerial photography. Generally, the photographing device includes an image sensor, and the image sensor is used to sense image data, the processor or the photographing device of the drone. The processor needs to perform distortion calibration on the image data collected by the image sensor.

When the performance of the processor is insufficient, if the code stream of the image data collected by the image sensor is large, the bandwidth and performance of the processor may be problematic.

Summary of the invention

Embodiments of the present invention provide an image processing method, a photographing device, a drone, and a ground end device to solve the problem of insufficient performance of the processor, and the bandwidth and performance of the processor when the code stream of the image data collected by the image sensor is large. The problem.

A first aspect of the embodiments of the present invention provides an image processing method, including:

The main processor acquires image data;

The main processor encodes the image data to obtain image encoded data;

The main processor transmits the image encoded data to a slave processor to cause the slave processor to perform distortion calibration on the image data according to the image encoded data.

A second aspect of the embodiments of the present invention provides an image processing method, including:

Receiving, by the processor, image encoded data sent by the main processor, where the image encoded data is obtained by encoding, by the main processor, the image data acquired by the main processor;

The slave processor performs distortion calibration on the image data according to the image encoded data.

A third aspect of the embodiments of the present invention provides an image processing method, which is applied to a drone, and includes:

Obtain image data;

Encoding the image data to obtain image encoded data;

And transmitting the image encoded data to a ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data.

A fourth aspect of the present invention provides an image processing method, which is applied to a ground end device, and includes:

Receiving image encoded data sent by the drone, the image encoded data being obtained by encoding the image data acquired by the drone;

Distorting the image data according to the image encoded data.

A fifth aspect of the embodiments of the present invention provides an image processing apparatus, including: a main processor and a slave processor;

The main processor is used to:

Obtain image data;

Encoding the image data to obtain image encoded data;

Transmitting the image encoded data to a slave processor to cause the slave processor to perform distortion calibration on the image data according to the image encoded data.

A sixth aspect of the embodiments of the present invention provides an image processing apparatus, including: a main processor and a slave processor;

The slave processor is used to:

Receiving image encoded data sent by the main processor, where the image encoded data is obtained by encoding, by the main processor, the image data obtained by the main processor;

Distorting the image data according to the image encoded data.

A seventh aspect of the embodiments of the present invention provides a photographing apparatus, including:

An image sensor for acquiring image data;

And an image processing apparatus according to the fifth aspect or the sixth aspect.

An eighth aspect of the embodiments of the present invention provides a drone, including:

body;

a power system mounted to the fuselage for providing power;

a photographing device mounted on the body for collecting image data;

And an image processing apparatus according to the fifth aspect or the sixth aspect.

A ninth aspect of the embodiments of the present invention provides a drone, including:

body;

a power system mounted to the fuselage for providing power;

a photographing device mounted on the body for collecting image data;

One or more first processors, and a first communication interface;

The first processor is configured to:

Obtaining image data collected by the photographing device;

Encoding the image data to obtain image encoded data;

The first communication interface is used to:

And transmitting the image encoded data to a ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data.

A tenth aspect of the present invention provides a ground end device, including: a second communication interface and a second processor;

The second communication interface is configured to receive image encoded data sent by the drone, and the image encoded data is obtained by encoding, by the drone, image data obtained by the drone;

The second processor is configured to perform distortion calibration on the image data according to the image encoded data.

The image processing method, the photographing device, the drone and the ground end device provided by the embodiment obtain image data by the main processor, encode the image data, obtain image coded data, and send the image coded data to the slave processor. In order for the slave processor to perform distortion calibration on the image data according to the image encoded data, the main processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, thereby saving the performance overhead of the main processor and the performance of the main processor. When the number of code stream pixels of the image data is insufficient, the bandwidth and performance of the main processor can be effectively avoided.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in view of the drawings.

FIG. 1 is a flowchart of an image processing method according to an embodiment of the present invention;

2 is a schematic structural diagram of a photographing apparatus according to an embodiment of the present invention;

3 is a schematic structural diagram of a drone according to an embodiment of the present invention;

4 is a flowchart of an image processing method according to another embodiment of the present invention;

FIG. 5 is a flowchart of an image processing method according to another embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a photographing apparatus according to another embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a drone according to another embodiment of the present invention; FIG.

FIG. 8 is a flowchart of an image processing method according to another embodiment of the present invention;

FIG. 9 is a schematic diagram of a communication system according to another embodiment of the present invention; FIG.

FIG. 10 is a schematic diagram of a communication system according to another embodiment of the present invention; FIG.

FIG. 11 is a flowchart of an image processing method according to another embodiment of the present invention;

FIG. 12 is a flowchart of an image processing method according to another embodiment of the present invention;

FIG. 13 is a structural diagram of an image processing apparatus according to an embodiment of the present invention;

FIG. 14 is a structural diagram of an image processing apparatus according to another embodiment of the present invention;

15 is a structural diagram of a drone according to an embodiment of the present invention;

FIG. 16 is a structural diagram of a ground end device according to an embodiment of the present invention.

Reference mark:

20-Camera 21-Image Sensor 22-Main Processor

23- slave processor 30- drone 31-master processor

32-slave processor 33-memory 34-support device

35-Photographing Equipment 24-Memory 90-Unmanned Aerial Vehicle

91-ground terminal equipment 901-processor 902-support equipment

903-Photographing Equipment 904-Communication System 910-Antenna

911-processor 912-memory 130-image processing device

131-main processor 132-slave processor 140-image processing device

141-main processor 142-slave processor 150- drone

1501-Photographing device 1502-First processor 1503-First communication interface

1504-support equipment 1506-propeller 1507-motor

160-ground terminal equipment 161-second communication interface 162-second processor

1517-Electronic governor

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly described with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

It should be noted that when a component is referred to as being "fixed" to another component, it can be directly on the other component or the component can be present. When a component is considered to "connect" another component, it can be directly connected to another component or possibly a central component.

All technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. The terminology used in the description of the present invention is for the purpose of describing particular embodiments and is not intended to limit the invention. The term "and/or" used herein includes any and all combinations of one or more of the associated listed items.

Some embodiments of the present invention are described in detail below with reference to the accompanying drawings. The features of the embodiments and examples described below can be combined with each other without conflict.

In general, a photographing device such as a photographing device mounted on a handheld pan/tilt, a photographing device held by a user, a movable platform such as a photographing device mounted on the drone, and the like are provided with an image sensor that can capture image data in real time and The captured image data is sent to the processor, and the image data is distorted and calibrated by the processor to obtain the calibrated image data, and then the calibrated image data is encoded to obtain the encoded image data, and the encoded image data is obtained. Stored in memory. However, the role of the processor is not only to process the image data, but also to process other data. When the performance of the processor is insufficient, if the code stream of the image data is high, the processor will have bandwidth and performance problems. In order to solve the problem, the embodiment provides an image processing method, which is described below in conjunction with a specific embodiment.

Embodiments of the present invention provide an image processing method. FIG. 1 is a flowchart of an image processing method according to an embodiment of the present invention. As shown in FIG. 1, the method in this embodiment may include:

Step S101: The main processor acquires image data.

The image processing method described in this embodiment is applied to the photographing apparatus 20 shown in FIG. 2, and the photographing apparatus 20 includes an image sensor 21, a main processor 22, and a slave processor 23.

In other embodiments, the image processing method can also be applied to a mobile platform such as a drone equipped with a photographing device. In some embodiments, the method is also applicable to other forms of devices having image processing capabilities. As shown in FIG. 3, the drone 30 includes a main processor 31 and a slave processor 32, and the drone 30 is equipped with a photographing device 35 via a support device 34 such as a pan/tilt. The main processor 31 and the slave processor 32 are electrically connected, and the main processor 31 and the photographing device 35 are electrically connected.

Optionally, the main processor acquires image data, and includes the following feasible implementation manners:

A possible implementation manner is that the main processor acquires image data collected by an image sensor.

As shown in FIG. 2, the image sensor 21 can acquire image data in real time and transmit the image data to the main processor 22.

In other embodiments, as shown in FIG. 3, the photographing device 35 may be provided with an image sensor that collects image data in real time and transmits the image data to the main processor 31.

Another possible implementation manner is that the main processor acquires a captured image corresponding to the shooting control instruction.

As shown in FIG. 2, the photographing apparatus 20 may generate a photographing control command according to a photographing control operation of the user, and optionally, the photographing control command is generated by the main processor 22 according to a photographing control operation of the user. The photographing control operation may be an operation of a user's photographing button or button of the photographing device 20. In addition, in other embodiments, the photographing device 20 can also be in communication connection with a control device through which the user can control the photographing device 20, and the photographing control operation can also be an operation of a photographing button or button of the control device by the user. When the photographing device 20 generates a photographing control instruction according to the photographing control operation of the user, the main processor 22 can acquire the image data acquired by the image sensor 21 at the time of the photographing control command generation, and it can be understood that the image sensor 21 is acquired at the photographing control command generating timing. The image data is the captured image.

In other embodiments, as shown in FIG. 3, the main processor 31 can control the photographing device 35 to take image data according to the photographing control instruction, and further, the photographing device 35 can transmit the image data photographed by the photographing device 35 to the main processor 31. The shooting control command may be sent to the drone 30 by a control terminal of the drone 30, such as a remote controller.

Step S102, the main processor encodes the image data to obtain an image coding number. according to.

As shown in FIG. 2, after acquiring the image data collected by the image sensor 21, the main processor 22 encodes the image data collected by the image sensor 21 to obtain image encoded data.

In other embodiments, as shown in FIG. 3, the main processor 31 encodes image data captured by the photographing device 35 to obtain image encoded data.

Step S103: The main processor sends the image encoded data to a slave processor, so that the slave processor performs distortion calibration on the image data according to the image encoded data.

As shown in FIG. 2, the main processor 22 encodes the image data to obtain image encoded data, and then transmits the image encoded data to the slave processor 23, and the slave processor 23 can decode the image encoded data to obtain image data. Further, the image data obtained after decoding is subjected to distortion calibration. Optionally, the main processor 22 may send the image encoded data to the slave processor 23 in real time, and the slave processor 23 may decode the image encoded data in real time to obtain image data, and perform image data obtained after decoding. Distortion calibration.

In other embodiments, as shown in FIG. 3, the main processor 31 encodes the image data, and after obtaining the image encoded data, the image encoded data is sent to the slave processor 32, and the slave processor 32 can perform image encoded data. Decoding to obtain image data, and further performing distortion calibration on the decoded image data. Optionally, the main processor 31 may send the image encoded data to the slave processor 32 in real time, and the slave processor 32 may decode the image encoded data in real time to obtain image data, and perform image data obtained after decoding. Distortion calibration.

Optionally, the main processor sends the image encoded data to a slave processor, so that the slave processor performs distortion calibration on the image data according to the image encoded data, including: the main processor And transmitting the image encoded data to the slave processor, so that the slave processor stores the image encoded data, and performs distortion calibration on the image data according to the stored image encoded data.

As shown in FIG. 2, the main processor 22 encodes the image data, and after obtaining the image encoded data, transmits the image encoded data to the slave processor 23, and the slave processor 23 stores the image encoded data, for example, when When the processor 23 receives the image encoded data sent by the main processor 22, the image encoded data is first stored in the memory, and when the slave processor 23 is idle, the image encoded data is acquired from the memory, and the encoded data is decoded. To obtain image data, the image data obtained after decoding is further subjected to distortion calibration.

In other embodiments, as shown in FIG. 3, the main processor 31 encodes the image data, and after obtaining the image encoded data, transmits the image encoded data to the slave processor 32, and the slave processor 32 performs the encoded data on the image. For example, when the image encoded data transmitted by the main processor 31 is received from the processor 32, the image encoded data is first stored in the memory, and when the slave processor 32 is idle, the image encoded data is acquired from the memory, The image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration.

Optionally, the main processor sends the image encoded data to the slave processor, and the host processor sends the image encoded data to the slave processor through a data bus.

As shown in FIG. 2, the main processor 22 and the slave processor 23 are connected via a data bus, and the main processor 22 transmits the encoded image encoded data obtained by the main processor 22 to the slave processor 23 via the data bus.

In other embodiments, as shown in FIG. 3, the main processor 31 and the slave processor 32 are connected by a data bus, and the main processor 31 transmits the encoded image encoded data obtained by the main processor 31 to the slave processor 32 via the data bus.

In this embodiment, the image data is acquired by the main processor, the image data is encoded, the image encoded data is obtained, and the image encoded data is sent to the slave processor, so that the slave processor performs distortion calibration on the image data according to the image encoded data. The processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the main processor. When the performance of the main processor is insufficient, and the code stream of the image data is large, the main can be effectively avoided. There is a problem with the bandwidth and performance of the processor.

Embodiments of the present invention provide an image processing method. FIG. 4 is a flowchart of an image processing method according to another embodiment of the present invention. As shown in FIG. 4, the method in this embodiment may include:

Step S401: The slave processor receives image coded data sent by the main processor, where the image coded data is obtained by encoding, by the main processor, the image data acquired by the main processor.

Optionally, the slave processor receives the image encoded data sent by the main processor, and the slave processor receives the image encoded data sent by the main processor through the data bus.

As shown in FIG. 2, the image encoded data transmitted from the main processor 22 is received from the processor 23, and the main processor 22 and the slave processor 23 are connected via a data bus, and the slave processor 23 receives the image transmitted by the main processor 22 via the data bus. Encoded data.

In other embodiments, as shown in FIG. 3, the slave processor 32 receives the transmission from the main processor 31. The image encoded data, the main processor 31 and the slave processor 32 are connected by a data bus, and the slave processor 32 receives the image encoded data transmitted from the main processor 31 via the data bus.

Step S402: The slave processor performs distortion calibration on the image data according to the image encoded data.

As shown in FIG. 2, after receiving the image encoded data sent by the main processor 22 from the processor 23, the slave processor 23 can decode the image encoded data to obtain image data, and further perform distortion calibration on the decoded image data. .

In other embodiments, as shown in FIG. 3, after receiving the image encoded data sent by the main processor 31 from the processor 32, the slave processor 32 may decode the image encoded data to obtain image data, and further obtain the decoded data. The image data is subjected to distortion calibration.

In addition, in other embodiments, after the step S401 receives the image encoded data sent by the main processor from the processor, the method further includes: the slave processor storing the image encoded data into a memory. Correspondingly, the slave processor performs distortion calibration on the image data according to the image encoded data, and the method includes: the slave processor performing the image data according to the image encoded data stored in the memory. Distortion calibration.

As shown in FIG. 2, the main processor 22 encodes the image data, and after obtaining the image encoded data, transmits the image encoded data to the slave processor 23, and the slave processor 23 stores the image encoded data, for example, when When receiving the image encoded data sent by the main processor 22, the processor 23 first stores the image encoded data into the memory, and when the slave processor 23 is idle, acquires the image encoded data from the memory, and performs the encoded data on the image. Decoding to obtain image data, and further performing distortion calibration on the decoded image data.

In other embodiments, as shown in FIG. 3, the main processor 31 encodes the image data, and after obtaining the image encoded data, transmits the image encoded data to the slave processor 32, and the slave processor 32 performs the encoded data on the image. For example, when the image encoded data transmitted by the main processor 31 is received from the processor 32, the image encoded data is first stored in the memory, and when the slave processor 32 is idle, the image encoded data is acquired from the memory, The image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration.

In this embodiment, the image encoded data sent by the main processor is obtained by the processor, and the image encoded data is obtained by encoding, by the main processor, the image data obtained by the main processor, and the image is distorted by the processor according to the image encoded data. Calibration so that the main processor does not need to be acquired The image data is distorted immediately in the image data, which saves the performance overhead of the main processor. When the performance of the main processor is insufficient, the bandwidth and performance of the main processor can be effectively avoided.

Embodiments of the present invention provide an image processing method. FIG. 5 is a flowchart of an image processing method according to another embodiment of the present invention. As shown in FIG. 5, the method in this embodiment may include:

Step S501: Receive, by the processor, image encoded data sent by the main processor, where the image encoded data is obtained by encoding, by the main processor, the image data acquired by the main processor.

The specific principles and implementation manners of step S501 and step S401 are the same, and are not described here.

Step S502: The slave processor stores the image encoded data into a memory.

As shown in FIG. 6, the photographing apparatus 20 further includes a memory 24. The main processor 22 encodes the image data to obtain image encoded data, and then transmits the image encoded data to the slave processor 23, and the slave processor 23 stores the image encoded data in the memory 24. The memory 24 is specifically a Secure Digital Memory Card (SD card), which is only a schematic description, and does not limit the specific form of the memory 24. In other embodiments, the memory 24 may be other Form of memory.

In other embodiments, as shown in FIG. 7, the drone 30 further includes a memory 33. The main processor 31 encodes the image data to obtain image encoded data, and then transmits the image encoded data to the slave processor 32, and the slave processor 32 stores the image encoded data in the memory 33. The memory 33 may be a secure digital memory card (SD card), which is only a schematic description, and does not limit the specific form of the memory 24. In other embodiments, the memory 33 may be other. Form of memory.

Step S503: The slave processor acquires the image encoded data from a memory.

As shown in FIG. 6, when the slave processor 23 is relatively idle, the image encoded data is acquired from the memory 24.

In other embodiments, as shown in FIG. 7, image encoded data is retrieved from memory 33 when slave processor 32 is relatively idle.

Step S504: The slave processor decodes the image encoded data to obtain the image data.

As shown in FIG. 6, the image data is decoded from the processor 23 to obtain image data.

In other embodiments, as shown in FIG. 7, the image encoded data is decoded from the processor 32 to obtain image data.

Step S505: The slave processor performs distortion calibration on the image data to obtain calibrated image data.

As shown in FIG. 6, the processor 23 performs distortion calibration on the decoded image data to obtain calibrated image data.

In other embodiments, as shown in FIG. 7, the decoded image data is subjected to distortion calibration from the processor 32 to obtain calibrated image data.

Step S506: The slave processor encodes the calibrated image data to obtain calibrated image encoded data.

As shown in FIG. 6, the slave processor 23 further encodes the calibrated image data to obtain calibrated image encoded data.

In other embodiments, as shown in FIG. 7, the slave processor 32 further encodes the calibrated image data to obtain calibrated image encoded data.

Step S507: The slave processor stores the calibrated image encoded data into the memory.

Optionally, the slave processor stores the calibrated image encoded data into the memory, including: the slave processor storing the calibrated image encoded data into the memory, and The image encoded data stored in the memory is replaced with the calibrated image encoded data.

As shown in FIG. 6, the slave processor 23 further stores the calibrated image encoded data in the memory 24. Since the pre-calibrated image encoded data is stored in the memory 24, optionally, the slave processor 23 can replace the pre-calibrated image encoded data stored in the memory 24 with the calibrated image encoded data.

In other embodiments, as shown in FIG. 7, the slave processor 32 further stores the calibrated image encoded data to the memory 33. Since the pre-calibrated image encoded data is stored in the memory 33, alternatively, the slave processor 32 can replace the pre-calibrated image encoded data stored in the memory 33 with the calibrated image encoded data.

The embodiment performs an image on the image encoded data stored in the memory from the processor. The image encoded data stored in the memory may be obtained after the main processor encodes the captured image captured by the photographing device, so that the slave processor only needs to perform image calibration on the captured image captured by the photographing device, without The image data of each frame sensed by the image sensor is image-calibrated, which not only improves the calculation efficiency of the slave processor, but also the main processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, thereby saving the main The performance overhead of the processor can effectively avoid the problem of bandwidth and performance of the main processor when the performance of the main processor is insufficient and the code stream of the image data is large.

Embodiments of the present invention provide an image processing method. FIG. 8 is a flowchart of an image processing method according to another embodiment of the present invention. The image processing method provided by this embodiment is applied to a drone. As shown in FIG. 8, the method in this embodiment may include:

Step S801, acquiring image data.

The image processing method provided in this embodiment is applicable to the communication system shown in FIG. 9. As shown in FIG. 9, the communication system includes a drone 90 and a ground end device 91. The drone 90 includes a processor 901, a support device 902 such as a pan/tilt head, a photographing device 903, and a communication system 904, and the photographing device 903 is mounted on the drone 90 via the support device 902. Processor 901 may specifically be a dedicated or other general purpose processor. Communication system 904 can communicate wirelessly with ground end device 91. The ground end device 91 wirelessly communicates with the drone 90 via the antenna 910.

Optionally, the acquiring image data includes: acquiring image data collected by an image sensor of the photographing device mounted on the drone.

As shown in FIG. 9, the photographing apparatus 903 may be provided with an image sensor that collects image data in real time and transmits the image data to the processor 901.

Alternatively, the acquiring image data includes: acquiring image data captured by the photographing device mounted on the drone according to a shooting control instruction. For example, the communication system 904 can receive the photographing control command transmitted by the ground end device 91, the processor 901 controls the photographing device 903 to capture image data according to the photographing control command, and the photographing device 903 can transmit the image data captured by the photographing device 903 to the processor 901.

Step S802, encoding the image data to obtain image encoded data.

After the processor 901 acquires the image data, the image data is encoded to obtain image encoded data.

Step S803: Send the image encoded data to a ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data.

Further, the processor 901 transmits the image encoded data to the ground end device 91 corresponding to the drone 90 through the communication system 904, and the ground end device 91 performs distortion calibration on the image data according to the image encoded data. Specifically, the ground end device 91 pairs The image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration. Optionally, the processor 901 can send the image encoded data to the ground end device 91 in real time through the communication system 904, and the ground end device 91 can decode the image encoded data in real time to obtain image data, and obtain the decoded image. The data is calibrated for distortion.

Optionally, the image encoding data is sent to the ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data, including: Sending the image encoded data to the ground end device corresponding to the drone, so that the ground end device stores the image encoded data, and performs the image data according to the stored image encoded data. Distortion calibration.

As shown in FIG. 10, the ground terminal device 91 includes a processor 911 and a memory 912. The memory 912 may be a secure digital memory card (SD card). The specific form of the memory 912, in other embodiments, the memory 912 may also be other forms of memory. The processor 901 transmits the image encoded data to the ground end device 91 corresponding to the drone 90 via the communication system 904, and the processor 911 stores the image encoded data to the memory 912. When the processor 911 is relatively idle, the slave memory 912 The image encoded data is acquired, the image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration.

In this embodiment, the image data is acquired by the drone, the image data is encoded, the image encoded data is obtained, and the image encoded data is sent to the ground end device, so that the ground end device performs distortion calibration on the image data according to the image encoded data, The human-machine processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the processor of the drone, when the performance of the processor is insufficient, and the code stream of the image data is large. Can effectively avoid problems with the bandwidth and performance of the processor.

Embodiments of the present invention provide an image processing method. FIG. 11 is a flowchart of an image processing method according to another embodiment of the present invention. The image processing method provided in this embodiment is applied to a ground end device. As shown in FIG. 11, the method in this embodiment may include:

Step S1101: Receive image coded data sent by the drone, and the image coded data is obtained by encoding the image data acquired by the drone.

As shown in FIG. 10, after the processor 901 of the drone 90 acquires the image data captured by the photographing device 903, the image data is encoded to obtain image encoded data, and the image encoded data is transmitted to the ground end device 91, the ground end. The device 91 receives the image encoded data transmitted by the drone 90.

Step S1102: Perform distortion calibration on the image data according to the image encoded data.

After the ground end device 91 receives the image encoded data sent by the drone 90, the processor 911 of the ground end device 91 decodes the image encoded data to obtain image data, and further performs distortion calibration on the decoded image data.

In other embodiments, after receiving the image encoded data sent by the drone, step S1101 further includes: storing the image encoded data into a memory. Correspondingly, step S1102 performs distortion calibration on the image data according to the image encoded data, including: performing distortion calibration on the image data according to the image encoded data stored in the memory.

After the ground end device 91 receives the image encoded data sent by the drone 90, the processor 911 first stores the image encoded data into the memory, and when the processor 911 is idle, the image encoded data is acquired from the memory. The image encoded data is decoded to obtain image data, and the decoded image data is further subjected to distortion calibration.

In this embodiment, the image encoding data sent by the UAV is received by the ground end device, and the image encoded data is obtained by encoding the image data acquired by the UAV, and the ground end device distorts the image data according to the image encoded data. The calibration makes the UAV not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the UAV processor. When the performance of the UAV processor is insufficient, the UAV can effectively avoid the unmanned There is a problem with the bandwidth and performance of the machine.

Embodiments of the present invention provide an image processing method. FIG. 12 is a flowchart of an image processing method according to another embodiment of the present invention. As shown in FIG. 12, the method in this embodiment may include include:

Step S1201: Receive image encoded data sent by the drone, and the image encoded data is obtained by encoding, by the drone, image data acquired by the drone.

Step S1201 is consistent with the specific principles and implementation manners of step S1101, and details are not described herein again.

Step S1202: Store the image encoded data in a memory.

As shown in FIG. 10, after the ground end device 91 receives the image encoded data transmitted by the drone 90, the processor 911 first stores the image encoded data in the memory 912.

Step S1203: Acquiring the image encoded data from a memory.

Image encoded data is retrieved from memory 912 when processor 911 is relatively idle.

Step S1204: Decoding the image encoded data to obtain the image data.

The processor 911 decodes the image encoded data to obtain image data.

Step S1205: Perform distortion correction on the image data to obtain calibrated image data.

The processor 911 performs distortion calibration on the decoded image data to obtain calibrated image data.

Step S1206: Encoding the calibrated image data to obtain calibrated image encoded data.

The processor 911 further encodes the calibrated image data to obtain calibrated image encoded data.

Step S1207: Store the calibrated image encoded data into the memory.

Optionally, the storing the calibrated image encoded data to the memory comprises: storing the calibrated image encoded data into the memory, and encoding the image stored in the memory The data is replaced with the calibrated image encoded data.

Processor 911 further stores the calibrated image encoded data to memory 912. Since the pre-calibrated image encoded data is stored in the memory 912, optionally, the processor 911 may replace the pre-calibrated image encoded data stored in the memory 912 with the calibrated image encoded data.

In this embodiment, the image encoding data stored in the memory is image-calibrated by the ground end device, and the image encoded data stored in the memory may be obtained by the UAV encoding the captured image captured by the photographing device, so that the ground end is obtained. The device only needs to be photographed on the shooting device The image is captured for image calibration without image calibration for each frame of image data sensed by the image sensor, which not only improves the computational efficiency of the ground end device, but also does not require the drone to instantly acquire the image data. The image data is distorted and calibrated, which saves the performance overhead of the UAV processor. When the performance of the UAV processor is insufficient, and the code stream of the image data is large, the processor of the UAV can be effectively avoided. There is a problem with bandwidth and performance.

An embodiment of the present invention provides an image processing apparatus. FIG. 13 is a structural diagram of an image processing apparatus according to an embodiment of the present invention. As shown in FIG. 13, the image processing apparatus 130 includes a main processor 131 and a slave processor 132. The main processor 131 is configured to: acquire image data; encode the image data to obtain image encoded data; and send the image encoded data to a slave processor, so that the slave processor 132 encodes the data according to the image. The image data is subjected to distortion calibration.

Optionally, when acquiring the image data, the main processor 131 is specifically configured to: acquire image data collected by the image sensor.

Optionally, when acquiring the image data, the main processor 131 is specifically configured to: acquire a captured image corresponding to the shooting control instruction.

Optionally, the main processor 131 sends the image encoded data to the slave processor, so that when the slave processor 132 performs distortion calibration on the image data according to the image encoded data, specifically, the image is used to: The encoded data is transmitted to the slave processor to cause the slave processor 132 to store the image encoded data, and to perform distortion calibration on the image data based on the stored image encoded data.

Optionally, when the main processor 131 sends the image encoded data to the slave processor, the method is specifically configured to: send the image encoded data to the slave processor through a data bus.

The specific principles and implementations of the image processing device provided by the embodiment of the present invention are similar to the embodiment shown in FIG. 1 and are not described herein again.

In this embodiment, the image data is acquired by the main processor, the image data is encoded, the image encoded data is obtained, and the image encoded data is sent to the slave processor, so that the slave processor performs distortion calibration on the image data according to the image encoded data. The processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the main processor. When the performance of the main processor is insufficient, and the code stream of the image data is large, the main can be effectively avoided. There is a problem with the bandwidth and performance of the processor.

An embodiment of the present invention provides an image processing apparatus. FIG. 14 is a structural diagram of an image processing apparatus according to another embodiment of the present invention; as shown in FIG. 14, the image processing apparatus 140 includes a main processor 141 and a slave processor 142. The slave processor 142 is configured to: receive image coded data sent by the main processor, where the image coded data is obtained by the main processor 141 to encode the image data obtained by the main processor 141; and the image is encoded according to the image coded data. The data is calibrated for distortion.

Optionally, after receiving the image encoded data sent by the main processor, the processor 142 is further configured to: store the image encoded data into a memory; and correspondingly, the slave processor 142 pairs the image according to the image encoded data. When the data is subjected to distortion calibration, specifically, the image data is subjected to distortion calibration according to the image encoded data stored in the memory.

Optionally, when the processor 142 performs distortion calibration on the image data according to the image encoded data stored in the memory, the method is specifically configured to: acquire the image encoded data from a memory; and encode the data into the image. Decoding to obtain the image data; performing distortion correction on the image data to obtain calibrated image data; encoding the calibrated image data to obtain calibrated image encoded data; encoding the calibrated image Data is stored to the memory.

Optionally, when the calibrated image encoded data is stored by the processor 142 into the memory, the method is specifically configured to: store the calibrated image encoded data into the memory, and store the memory in the memory The image encoded data is replaced with the calibrated image encoded data.

Optionally, when receiving the image encoded data sent by the main processor from the processor 142, the method is specifically configured to: receive the image encoded data sent by the main processor by using a data bus.

The specific principles and implementations of the image processing device provided by the embodiment of the present invention are similar to the embodiment shown in FIG. 4 or FIG. 5, and details are not described herein again.

In this embodiment, by performing image calibration on the image encoded data stored in the memory from the processor, the image encoded data stored in the memory may be obtained by the main processor encoding the captured image captured by the photographing device, so that the processing is performed. It only needs to perform image calibration on the captured image captured by the shooting device without image calibration for each frame of image data sensed by the image sensor, which not only improves the calculation efficiency of the slave processor, but also does not support the main processor. It is necessary to perform distortion calibration on the image data immediately when the image data is acquired, thereby saving the performance of the main processor. Overhead, when the performance of the main processor is insufficient, and the code stream of the image data is large, the bandwidth and performance of the main processor can be effectively avoided.

An embodiment of the present invention provides a photographing apparatus including: an image sensor and the image processing apparatus 130 or the image processing apparatus 140 described in the above embodiments.

An embodiment of the present invention provides a drone that includes: a body, a power system, a photographing device, and an image processing device 130 or an image processing device 140 according to the above embodiments, wherein the power system is installed in the The body is for providing power; the photographing device is mounted on the body for collecting image data.

Embodiments of the present invention provide a drone. 15 is a structural diagram of a drone according to an embodiment of the present invention. As shown in FIG. 15, the drone 150 includes: a fuselage, a power system, a photographing device 1501, and one or more first processors 1502. a communication interface 1503, wherein a power system is mounted on the airframe for providing power, the power system comprising at least one of: a motor 1507, a propeller 1506, and an electronic governor 1517, the power system being mounted on the fuselage Used to provide flight power. The photographing device 1501 is mounted to the body through a support device 1504 such as a pan/tilt.

The first processor 1502 is configured to: acquire image data collected by the photographing device; and encode the image data to obtain image encoded data. The first communication interface 1503 is configured to: send the image encoded data to a ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data.

Optionally, when acquiring the image data collected by the photographing device, the first processor 1502 is specifically configured to: acquire image data collected by the image sensor of the photographing device 1501.

Optionally, when acquiring the image data collected by the photographing device, the first processor 1502 is specifically configured to: acquire image data captured by the photographing device 1501 according to the photographing control instruction.

Optionally, the first communication interface 1503 is configured to send the image encoded data to a ground end device corresponding to the drone, so that the ground end device distorts the image data according to the image encoded data. When calibrating, the method is specifically configured to: send the image coded data to a ground end device corresponding to the drone, so that the ground end device performs the image coded data And storing, and performing distortion calibration on the image data according to the stored image encoded data.

The specific principles and implementations of the UAV provided by the embodiments of the present invention are similar to those of the embodiment shown in FIG. 8, and are not described herein again.

In this embodiment, the image data is acquired by the drone, the image data is encoded, the image encoded data is obtained, and the image encoded data is sent to the ground end device, so that the ground end device performs distortion calibration on the image data according to the image encoded data, The human-machine processor does not need to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the processor of the drone, when the performance of the processor is insufficient, and the code stream of the image data is large. Can effectively avoid problems with the bandwidth and performance of the processor.

Embodiments of the present invention provide a ground end device. FIG. 16 is a structural diagram of a ground end device according to an embodiment of the present invention. As shown in FIG. 16, the ground end device 160 includes: a second communication interface 161 and a second processor 162. The second communication interface 161 is configured to receive an unmanned person. Image encoded data sent by the machine, the image encoded data is obtained by encoding the image data acquired by the drone; the second processor 162 is configured to perform the image data according to the image encoded data. Distortion calibration.

Optionally, after the second communication interface 161 receives the image coded data sent by the drone, the second processor 162 is further configured to: store the image coded data into the memory; correspondingly, the second processor 162 is configured according to the When the image encoded data is subjected to distortion calibration of the image data, specifically, the image data is subjected to distortion calibration according to the image encoded data stored in the memory.

Optionally, when the second processor 162 performs distortion calibration on the image data according to the image encoded data stored in the memory, specifically, the method is: acquiring the image encoded data from a memory; and encoding the image. Decoding the data to obtain the image data; performing distortion correction on the image data to obtain calibrated image data; encoding the calibrated image data to obtain calibrated image encoded data; and calibrating the image The encoded data is stored to the memory.

Optionally, when the second processor 162 stores the calibrated image encoded data in the memory, specifically, the method is: storing the calibrated image encoded data into the memory, and storing the The stored image encoded data is replaced with the calibrated image encoding data.

The specific principles and implementation manners of the ground end device provided by the embodiments of the present invention are the same as those in FIG. 11 or FIG. 12, and details are not described herein again.

In this embodiment, the image encoding data stored in the memory is image-calibrated by the ground end device, and the image encoded data stored in the memory may be obtained by the UAV encoding the captured image captured by the photographing device, so that the ground end is obtained. The device only needs to perform image calibration on the captured image captured by the shooting device without performing image calibration on each frame of image data sensed by the image sensor, which not only improves the calculation efficiency of the ground end device, but also does not It is necessary to perform distortion calibration on the image data immediately when the image data is acquired, which saves the performance overhead of the processor of the drone, and when the performance of the processor of the drone is insufficient, and the code stream of the image data is large, Effectively avoid problems with the bandwidth and performance of the drone's processor.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

The above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium. The above software functional unit is stored in a storage medium and includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or A network device or the like) or a processor performs part of the steps of the method described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

A person skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of each functional module described above is exemplified. In practical applications, the above function assignment can be completed by different functional modules as needed, that is, the device is installed. The internal structure is divided into different functional modules to perform all or part of the functions described above. For the specific working process of the device described above, refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present invention. range.

Claims (38)

An image processing method, comprising: The main processor acquires image data; The main processor encodes the image data to obtain image encoded data; The main processor transmits the image encoded data to a slave processor to cause the slave processor to perform distortion calibration on the image data according to the image encoded data. The method according to claim 1, wherein the main processor acquires image data, including: The main processor acquires image data collected by an image sensor. The method according to claim 1, wherein the main processor acquires image data, including: The main processor acquires a captured image corresponding to a shooting control command. The method according to any one of claims 1 to 3, wherein the main processor transmits the image encoded data to a slave processor, so that the slave processor encodes data according to the image. The image data is subjected to distortion calibration, including: The main processor sends the image encoded data to a slave processor, so that the slave processor stores the image encoded data, and the image data is distorted according to the stored image encoded data. calibration. The method according to claim 1, wherein the main processor sends the image encoded data to the slave processor, comprising: The main processor transmits the image encoded data to a slave processor via a data bus. An image processing method, comprising: Receiving, by the processor, image encoded data sent by the main processor, where the image encoded data is obtained by encoding, by the main processor, the image data acquired by the main processor; The slave processor performs distortion calibration on the image data according to the image encoded data. The method according to claim 6, wherein after the slave processor receives the image encoded data sent by the main processor, the method further includes: The slave processor stores the image encoded data to a memory; Correspondingly, the slave processor performs distortion calibration on the image data according to the image encoded data, including: The slave processor performs distortion calibration on the image data according to the image encoded data stored in the memory. The method according to claim 7, wherein the slave processor performs distortion calibration on the image data according to the image encoded data stored in the memory, including: The slave processor acquires the image encoded data from a memory; Decoding the image encoded data by the slave processor to obtain the image data; Performing distortion calibration on the image data by the slave processor to obtain calibrated image data; The slave processor encodes the calibrated image data to obtain calibrated image encoded data; The slave processor stores the calibrated image encoded data to the memory. The method of claim 8, wherein the slave processor stores the calibrated image encoded data to the memory, comprising: The slave processor stores the calibrated image encoded data to the memory and replaces the image encoded data stored in the memory with the calibrated image encoded data. The method according to claim 6, wherein the receiving, by the processor, the image encoded data sent by the main processor comprises: The slave processor receives image encoded data transmitted by the host processor through a data bus. An image processing method is applied to a drone, characterized in that it comprises: Obtain image data; Encoding the image data to obtain image encoded data; And transmitting the image encoded data to a ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data. The method according to claim 11, wherein the acquiring image data comprises: Acquiring image data collected by an image sensor of the photographing device mounted on the drone. The method according to claim 11, wherein the acquiring image data comprises: Acquiring image data captured by the photographing device mounted on the drone according to a shooting control command. The method according to any one of claims 11 to 13, wherein the image encoded data is sent to a ground end device corresponding to the drone, so that the ground end device according to the image The encoded data is subjected to distortion calibration of the image data, including: Transmitting the image encoded data to a ground end device corresponding to the drone, so that the ground end device stores the image encoded data, and the image data is stored according to the stored image encoded data. Perform distortion calibration. An image processing method is applied to a ground end device, and includes: Receiving image encoded data sent by the drone, the image encoded data being obtained by encoding the image data acquired by the drone; Distorting the image data according to the image encoded data. The method according to claim 15, wherein after the receiving the image encoded data sent by the drone, the method further comprises: Storing the image encoded data to a memory; Correspondingly, the performing distortion calibration on the image data according to the image encoded data comprises: Distorting the image data according to the image encoded data stored in the memory. The method according to claim 16, wherein the performing distortion calibration on the image data according to the image encoded data stored in the memory comprises: Acquiring the image encoded data from a memory; Decoding the image encoded data to obtain the image data; Performing distortion calibration on the image data to obtain calibrated image data; Encoding the calibrated image data to obtain calibrated image encoded data; The calibrated image encoded data is stored to the memory. The method of claim 17, wherein the storing the calibrated image encoded data to the memory comprises: The calibrated image encoded data is stored in the memory, and the image encoded data stored in the memory is replaced with the calibrated image encoded data. An image processing device, comprising: a main processor and a slave processor; The main processor is used to: Obtain image data; Encoding the image data to obtain image encoded data; Transmitting the image encoded data to a slave processor to cause the slave processor to perform distortion calibration on the image data according to the image encoded data. The image processing device according to claim 19, wherein when the main processor acquires image data, it is specifically used to: Obtain image data collected by the image sensor. The image processing device according to claim 19, wherein when the main processor acquires image data, it is specifically used to: Acquire a captured image corresponding to the shooting control command. The image processing device according to any one of claims 19 to 21, wherein said main processor transmits said image encoded data to a slave processor to cause said slave processor to encode data according to said image When the image data is subjected to distortion calibration, it is specifically used for: And transmitting the image encoded data to the slave processor, so that the slave processor stores the image encoded data, and performs distortion calibration on the image data according to the stored image encoded data. The image processing device according to claim 19, wherein when the main processor sends the image encoded data to the slave processor, specifically: The image encoded data is transmitted to the slave processor via a data bus. An image processing device, comprising: a main processor and a slave processor; The slave processor is used to: Receiving image encoded data sent by the main processor, where the image encoded data is obtained by encoding, by the main processor, the image data obtained by the main processor; Distorting the image data according to the image encoded data. The image processing device according to claim 24, wherein after receiving the image encoded data sent by the main processor, the slave processor is further configured to: Storing the image encoded data to a memory; Correspondingly, when the slave processor performs distortion calibration on the image data according to the image encoded data, specifically: Distorting the image data according to the image encoded data stored in the memory. The image processing device according to claim 25, wherein the slave processor performs distortion calibration on the image data according to the image encoded data stored in the memory, specifically for: Acquiring the image encoded data from a memory; Decoding the image encoded data to obtain the image data; Performing distortion calibration on the image data to obtain calibrated image data; Encoding the calibrated image data to obtain calibrated image encoded data; The calibrated image encoded data is stored to the memory. The image processing device according to claim 26, wherein when the slave processor stores the calibrated image encoded data to the memory, specifically: The calibrated image encoded data is stored in the memory, and the image encoded data stored in the memory is replaced with the calibrated image encoded data. The image processing device according to claim 24, wherein when the slave processor receives the image encoded data sent by the main processor, the method is specifically configured to: The image encoded data transmitted by the main processor is received through the data bus. A drone, characterized in that it comprises: body; a power system mounted to the fuselage for providing power; a photographing device mounted on the body for collecting image data; One or more first processors, and a first communication interface; The first processor is configured to: Obtaining image data collected by the photographing device; Encoding the image data to obtain image encoded data; The first communication interface is used to: And transmitting the image encoded data to a ground end device corresponding to the drone, so that the ground end device performs distortion calibration on the image data according to the image encoded data. The drone according to claim 29, wherein when the first processor acquires image data collected by the photographing device, it is specifically used to: Obtaining image data collected by an image sensor of the photographing device. The drone according to claim 29, wherein when the first processor acquires image data collected by the photographing device, it is specifically used to: Obtaining image data captured by the photographing apparatus according to a photographing control instruction. The drone according to any one of claims 29 to 31, wherein the first communication interface is configured to send the image encoded data to a ground end device corresponding to the drone to make When the ground end device performs distortion calibration on the image data according to the image encoded data, it is specifically used to: Transmitting the image encoded data to a ground end device corresponding to the drone, so that the ground end device stores the image encoded data, and the image data is stored according to the stored image encoded data. Perform distortion calibration. A ground end device, comprising: a second communication interface and a second processor; The second communication interface is configured to receive image encoded data sent by the drone, and the image encoded data is obtained by encoding, by the drone, image data obtained by the drone; The second processor is configured to perform distortion calibration on the image data according to the image encoded data. The ground end device according to claim 33, wherein after the second communication interface receives the image encoded data sent by the drone, the second processor is further configured to: Storing the image encoded data to a memory; Correspondingly, when the second processor performs distortion calibration on the image data according to the image encoded data, specifically, the method is: Distorting the image data according to the image encoded data stored in the memory. The ground end device according to claim 34, wherein when the second processor performs distortion calibration on the image data according to the image encoded data stored in the memory, it is specifically used to: Acquiring the image encoded data from a memory; Decoding the image encoded data to obtain the image data; Performing distortion calibration on the image data to obtain calibrated image data; Encoding the calibrated image data to obtain calibrated image encoded data; The calibrated image encoded data is stored to the memory. The ground end device according to claim 35, wherein when the second processor stores the calibrated image encoded data to the memory, specifically: The calibrated image encoded data is stored in the memory, and the image encoded data stored in the memory is replaced with the calibrated image encoded data. A photographing apparatus, comprising: An image sensor for acquiring image data; And an image processing apparatus according to any one of claims 19-28. A drone, characterized in that it comprises: body; a power system mounted to the fuselage for providing power; a photographing device mounted on the body for collecting image data; And an image processing apparatus according to any one of claims 19-28.

Images