Open AccessArticle

Joint Optimization of Relay Communication Rates in Clustered Drones under Interference Conditions

College of Electronic Information Engineering, Changchun University of Science and Technology, Changchun 130022, China

Author to whom correspondence should be addressed.

Drones 2024, 8(8), 381; https://doi.org/10.3390/drones8080381

Submission received: 9 July 2024 / Revised: 27 July 2024 / Accepted: 29 July 2024 / Published: 7 August 2024

Browse Figures

Figure 1
Schematic diagram of swarm drone relay communication under interference conditions. "> Figure 2
Schematic diagram of the dynamic interference process in the channel. "> Figure 3
Schematic diagram of the Discrete Soft Actor-Critic algorithm architecture. "> Figure 4
Comparison of resistances to Constant frequency interference effects. "> Figure 5
Comparison of resistance to sweep frequency interference effects. "> Figure 6
Comparison of resistance to hybrid frequency interference effects. "> Figure 7
Comparison of anti-interference effects with different hyperparameters. "> Figure 8
Action selection of the BO_DSAC algorithm under different interference modes. "> Figure 9
Action selection of the Conv_DQN algorithm under different interference modes. "> Figure 10
Action selection of the DQN algorithm under different interference modes. "> Figure 11
Different motion trajectories of the cluster drones. "> Figure 12
Joint optimization for different trajectories. "> Figure 13
Comparison of total communication rate values among different optimization schemes. "> Figure 14
Communication delay under different optimization schemes. "> Figure 15
Changes in the modulation order of the relay drone. "> Figure 16
Variation in the transmission power of the relay drone. "> Figure 17
Energy consumption variation in the relay drone during the mission period. "> Figure 18
Mission energy consumption value of the relay drone. "> Figure 19
Changes in the three-dimensional trajectory of the relay drone. "> Figure 20
Impact of variables across different dimensions on the optimization of total communication rate. ">

Versions Notes

Abstract

To address the issues of communication failure and inefficiency in clustered drone relay communication due to external malicious interference, this paper proposes a joint optimization method for relay communication rates under interference conditions for clustered drones. This method employs the following two-step processing framework: Firstly, the Discrete Soft Actor-Critic (DSAC) algorithm is used to train the relay drones for dynamic channel selection, effectively avoiding various types of interference. Simultaneously, the Bayesian optimization algorithm is applied to optimize the hyperparameters of the DSAC algorithm, further enhancing its performance. Subsequently, the modulation order, transmission power, trajectory of the relay drones, and power allocation factors of the clustered drones are jointly optimized. This complex problem is transformed into a convex subproblem for determining a solution, aiming to maximize the communication rate of the clustered drones. The simulation’s results demonstrate that the proposed algorithm exhibits excellent performances in terms of anti-interference capability, solution convergence, and stability. It effectively improves the mission efficiency of clustered drones under interference conditions and enhances their adaptability to dynamic environments.

Keywords:

interference conditions; clustered drone relay communication; Discrete Soft Actor-Critic algorithms; convex optimization

1. Introduction

In recent years, with the advancements in multi-agent technology, efficient computing technology, network convergence technology, and new communication technologies, various types of equipment have shed the “fight alone” approach. Instead, they operate in clusters, networking and collaborating to execute various tasks, thereby effectively enhancing the system’s ability to handle complexity, dynamics, and adverse conditions. However, in this process, effectively managing system resources, ensuring fair efficiency, scheduling tasks, and combating external interference have become new challenges in meeting application requirements. These challenges are currently hotspots in both academia and industry.

As an important application product of multi-agent systems, clustered drones have a broad application space [1]. They form an unmanned network system that is low cost with high dynamics and strong stability due to resource integration, self-organized collaboration, and intelligent scheduling. This provides strong support for the future construction of air, sky, earth, and sea globalized networks, the realization of communication–perception–computing integration, and the development and construction of unmanned platforms. Reference [2] introduces the use of multiple drones to form an integrated IoT between sky and Earth, redeploying multiple drones to a different location, changing the resource allocation, and offloading server computations by alternating heuristic greedy and successive convex approximation methods to minimize the total computational overhead of ground-based IoT devices. Reference [3] proposes a generalized intelligent collaborative task scheduling framework that switches between heuristic and deep reinforcement learning-based scheduling solutions to address the ever-changing task requests when dynamically served by multiple drones. Reference [4] applies multiple drones as airborne relays to serve clustered users on the ground and proposes a channel statistical model based on the probability of occurrence. It also establishes analytic expressions for the average rate and the average outage probability of this channel model, comparing its performance with those of the Rayleigh channel and Rice channel models. Reference [5] proposes a traffic load balancing scheme for a multi-drone-assisted Fog network to minimize wireless delays for IoT users. Reference [6] proposes a framework that utilizes a dynamic drone network for the heuristic optimization of coverage areas. This framework employs Bézier curves to plan the flight paths of drones, enabling intelligent adjustments to their positions and trajectories, which significantly enhances the network access quality for ground users. Reference [7] conducted an in-depth investigation into routing protocols based on clustered drone networks, comparing the characteristics and performances of various protocols to provide researchers with a basis for selecting appropriate ones.

Despite the promising results demonstrated by the solutions proposed in the studies referenced, there is a notable lack of consideration for real-world environments and complex scenarios, which introduces certain limitations. Specifically, in the presence of malicious dynamic interference and a complex electromagnetic environment, the communication processes of clustered drones are inevitably impacted. Under these conditions, the signal-to-noise ratios of the received signals are affected by more than just the drones’ trajectories and transmission powers. Therefore, it is essential to conduct a thorough analysis of the physical impact of the interference on the entire communication process and to develop tailored algorithms designed to mitigate such external interference. Reference [8] proposes a convergent rate-partitioning-based interference management resource allocation and clustering algorithm for iteratively optimizing drone transmission power, drone pairing, wireless resource scheduling, and wireless resource pricing to achieve interference management in cellular drone networks. Reference [9] proposes a drone-path-tracking scheme that uses a radial basis neural network to approximate the adaptive approximation law of the gyroscopic effect function to balance the effects of system uncertainties and nonlinearities, improving the convergence speed and error accuracy of the controller. Reference [10] designed a joint optimization model that provides optimal truck and drone routing policies to address mission abortion when truck and clustered drone routes are subjected to random attacks and to minimize the total cost. Reference [11] designed a dynamic-based communication antijamming decision-making method to solve the problem of intelligent antijamming decision making for battlefield communication, enabling the intelligent system to effectively avoid jamming while ensuring uninterrupted communication as much as possible.

Current research on antijamming primarily concentrates on methods to eliminate and mitigate the impacts of interference on communication processes. However, this focus often overlooks the crucial aspect of enhancing the system performance of clustered drones post-elimination of interference, which is essential for bolstering their mission execution capabilities. Consequently, the joint optimization of the “residual performance” of clustered drones following antijamming efforts holds significant research value, promising substantial improvements in their operational effectiveness. Reference [12] jointly optimizes the trajectory and scheduling plan of each drone, combining convex optimization and ant colony optimization algorithms to obtain the proposed optimal solution, maximizing the total amount of relay data. Reference [13] proposes a drone relay vehicle networking architecture based on relay protocols with nonorthogonal multiple access and maximum ratio combination techniques, as well as improved particle swarm optimization algorithms to achieve increased data rates for cell-edge vehicles in rural road scenarios. Reference [14] investigates the problem of clustered drone deployment for better coverage and proposes a genetic algorithm that encodes the solution of the problem as a chromosome and simulates the process of biological evolution to find favorable solutions, balancing between the energy consumption of all drones and the maximization of the lifetime of the full coverage network. Reference [15] proposes a 3D layout analysis framework for coordinating a fleet of drone relays and uses it to construct a complex mixed-integer nonconvex planning problem for network performance optimization in terms of user throughput fairness through a parallel alpha-fair drone deployment method. Reference [16] considers a scenario in which flying drones provide wireless services to multiple ground nodes simultaneously and proposes an effective joint transmission power and trajectory optimization algorithm that can maximize the minimum average throughput for a given length of time.

Based on the considerations outlined above, this paper delves deeper into the subject, and the contributions are as follows:

-: Aiming to maximize the communication rate among clustered drones, this study constructs a multivariable coupled 0–1 mixed nonlinear integer programming model. This model incorporates various constraints, including the physical limitations of drones, channel selection, and communication performance, among others;
-: The Discrete Soft Actor-Critic (DSAC) algorithm is deployed on relay drones, training them for channel selection to adeptly navigate and mitigate dynamic interference;
-: The Bayesian optimization algorithm is utilized to refine the hyperparameters of the DSAC algorithm, such as the learning rate, discount factor, and target entropy, enhancing the algorithm’s capability to resist interference;
-: The joint optimization problem is systematically decomposed and transformed into manageable convex subproblems, including the modulation order of relay drones, their transmission power, trajectories, and the power allocation factors for clustered drones, all aimed at iterative solutions to maximize the total communication rate. The experimental results demonstrate that the algorithm proposed in this paper effectively addresses the efficient communication challenges faced by clustered drones in environments with interference. Furthermore, the algorithm exhibits robust stability, strong convergence, and impressive task performance.

The structure of this article is as follows: Section 1 introduces related work and provides an overview of the current research status. Section 2 describes the system model, detailing the tasks, channels, and interference. Section 3 presents the problem description, integrating the model to express specific research questions. Section 4 proposes a DSAC algorithm using Bayesian optimization. Section 5 presents a joint optimization algorithm for communication rates. Section 6 covers experimental simulations. Section 7 concludes the paper. Table 1 presents a description of the key parameters used throughout the paper.

2. System Model

2.1. Task Model

When performing tasks, there is generally a considerable spatial distance between the swarm of drones and the ground station, requiring the involvement of relay drones for real-time information transmission. Suppose that relay drones assisting in communications suffer from malicious interference by an enemy, causing communication disruptions. Relay drones need to sense enemy interference, intelligently select channels, and optimize their transmission power, trajectory, and modulation methods to maximize communication rates, thereby forming real-time, efficient, and stable relay communications. Figure 1 shows a schematic diagram of relay communication by a swarm of drones under interference conditions.

Assuming a relay drone assists in communication for a duration of T, it is divided into

K

time intervals,

δ_{T}

and

M

, where

Ω = {1, 2, ..., M}

drones form a cluster, with the coordinates

F_{i} = (x_{i} (k), y_{i} (k), z_{i} (k)) i \in Ω, k \in K

. The coordinates of the ground station are

(0, 0, 0)

, the jammer’s coordinates are

(x^{J}, y^{J}, 0)

, and the relay drone’s coordinates are

Q = (x_{r}^{u a v} (k), y_{r}^{u a v} (k), z_{r}^{u a v} (k)) k \in K

. Changes in positions during each time slot

k

are negligible. Additionally, the relay drone updates the position coordinates of the cluster drones every

δ

time interval, during which changes in the cluster drones’ coordinates can also be neglected. The relay drone uses NOMA multiplexing to communicate, enhancing the real-time transmission efficiency of the cluster drones, and employs amplify-and-forward in a half-duplex communication mode for data transmission.

2.2. Channel Model

According to Reference [17], the operational trajectory of the relay drone in the air exhibits strong Line-of-Sight (LOS) characteristics with both the cluster of drones and the ground station. Therefore, the channel model between the relay drone and both the cluster of drones and the ground station can be approximated by the free-space propagation model of electromagnetic waves, which is described as follows:

H^{w} (k) = G_{t} G_{r} {(λ^{G})}^{2} / {(4 π D (k))}^{2} \forall k

(1)

In this model,

G_{t} G_{r}

represents the product of the antenna gains of the transmitter and receiver,

λ^{G}

denotes the wavelength of the electromagnetic waves at different frequencies, and

D (k)

signifies the distance between the relay drone and either the cluster of drones or the ground station. For example, the expression for calculating the free-space path loss between the relay drone and the ground station node can be illustrated as follows:

D {(k)}_{R}^{S} = {‖ Q (k) ‖}_{2} = \sqrt{{(x_{r}^{u a v} (k))}^{2} + {(y_{r}^{u a v} (k))}^{2} + {(z_{r}^{u a v} (k))}^{2}} \forall k

(2)

Jamming devices are typically hidden in relatively concealed areas to prevent detection and subsequent damage or destruction. Therefore, according to the 3GPP [18], the channel model between the jammer and the relay drone is usually a probabilistic loss model that mixes LOS and Non-Line-of-Sight (NLOS) components. The calculation expression for this model is as follows:

Λ = P^{L} (L^{D} + ϑ) + P^{N L} (L^{D} + δ + ϑ)

(3)

In this model,

L^{D}

represents the spatial propagation loss of the electromagnetic waves. The calculation formula for this loss is as follows:

L^{D} = 32.45 + 20 \lg (D {(k)}_{J}^{R}) + 20 \lg (f_{c}^{u a v})

(4)

In this model,

D {(k)}_{J}^{R} = \sqrt{{(x_{r}^{u a v} (k) - x^{J} (k))}^{2} + {(y_{r}^{u a v} (k) - y^{J} (k))}^{2} + {(z_{r}^{u a v} (k))}^{2}}

represents the distance between the relay drone and the jammer,

f_{c}^{u a v}

denotes the communication frequency of the relay drone,

δ

refers to shadow fading, which is typically modeled as a log-normal distribution, and

ϑ

indicates the Rician distribution. The Rician distribution is used to model small-scale fading, as jammers typically have high transmission power, and there is always a line-of-sight component present in the signal, making this distribution more suitable for real-world conditions.

2.3. Interference Model

During the transmission of information by the relay drone, the jammer exhibits energy detection and power suppression capabilities, thereby inducing communication interference in the relay drone. The interference detection model based on energy is formulated as follows:

{\begin{cases} C_{1}^{u a v} : r s (k) = 0 \\ C_{2}^{u a v} : r s (k) = t s (k) \\ C_{3}^{u a v} : r s (k) = t s (k) + j (k) \end{cases}

(5)

In this context,

r s (k)

denotes the currently captured signal,

t s (k)

represents the communication signal, and

j (k)

signifies the interference signal. After the subband signals pass through the band-pass filter, their signal power,

ρ^{u} (k)

, is computed. Subsequently, the energy levels for each frequency band are calculated. The corresponding calculation expression is given as follows:

E^{t} = \sum_{i = 1}^{v} {| ρ^{u} (k) |}^{2}

(6)

In this context,

v

denotes the number of subbands. By comparing

E^{t}

with the respective thresholds,

λ_{C_{1}}, λ_{C 2}

, within the model, the channel state is determined. The relay drone also exhibits spectrum-sensing capabilities, enabling it to intelligently select the operating frequency,

f_{n}^{R u a v} (1 \leq n \leq F)

, from the available frequency set

F_{f}^{R u a v} = [f_{l}^{R u a v}, f_{h}^{R u a v}]

through frequency hopping, thereby avoiding the impact of interference on relay transmission. The subfrequency ranges within each frequency point do not overlap, and the available frequency set contains

F

frequency points, with subchannel bandwidths of

B_{n}^{R u a v}

. The limits for the working frequency range of the relay drone, shared with the jammers, are defined by

f_{l}^{R u a v}

and

f_{h}^{R u a v}

. Assuming the maximum transmission power of cluster drone

i

P_{i}^{F} (k), k \in K

, the transmission power of the relay drone is

P^{R} (k)

, and the maximum transmission power of the jammer is

P^{J}

, with the product of the antenna gains between them being

H_{1}^{*}

. In a relay scenario in which the cluster drones need to transmit detection information back to the ground station, the Signal-to-Interference-plus-Noise Ratio (SINR) expression for a relay drone working at frequency

f_{n}^{R u a v}

receiving the signal transmitted by cluster drone

i

is as follows:

S I N R_{i, R} = \frac{P_{i}^{F} (k) H_{i}^{w} (k)}{P^{J} H_{1}^{*} Λ + n_{0} B_{n}^{R u a v}} i \in M, \forall k

(7)

In this model,

n_{0}

represents the noise power spectral density,

H_{i}^{w} (k)

denotes the channel gain between the cluster drone,

i

, and the relay drone. The magnitude of the

S I N R_{i, R}

directly reflects the interference condition of the relay drone; the smaller its value, the greater the degree of interference, and conversely, the larger its value, the lesser the degree of interference. Assuming the normal communication threshold is

D_{n}

S I N R_{i, R} \geq D_{n}

indicates the normal transmission of relay information, while

S I N R_{i, R} < D_{n}

signifies a failure in the information relay.

The interference model proposed in this paper encompasses constant frequency interference, sweep frequency interference, and hybrid frequency interference. The corresponding expressions are as follows:

\begin{matrix} I_{f i x e d}^{i n t e r f} (k, f) = A_{f i x e d} δ (f - f_{f i x e d}) \\ I_{s w e e p}^{i n t e r f} (k, f) = A_{s w e e p} S a (\frac{f - f_{s w e e p} (k)}{B_{n}^{R u a v}}) \\ I_{m i x e d}^{i n t e r f} (k, f) = I_{f i x e d}^{i n t e r f} (k, f) + I_{s w e e p}^{i n t e r f} (k, f) \end{matrix}

(8)

In this context,

I_{f i x e d}^{i n t e r f} (k, f), I_{s w e e p}^{i n t e r f} (k, f), I_{m i x e d}^{i n t e r f} (k, f)

represent the interference intensity of constant frequency interference, sweep frequency interference, and hybrid frequency interference, respectively, which are functions of time and frequency.

A_{f i x e d}, A_{s w e e p}

denote the amplitude of constant frequency interference and sweep frequency interference signals,

f_{f i x e d}

represents the set of constant frequencies,

f_{s w e e p} (k)

signifies the set of frequencies at time

k

for sweep frequency interference, and

S a (\cdot)

denotes the sampling function. The constant frequency interference mode is characterized as a bandwidth-limited random process with a central frequency of

f_{f i x e d}

, while the sweep frequency interference mode is a random process with frequencies varying over time. The hybrid frequency interference mode combines both constant frequency interference and sweep frequency interference, making it a more complex random process. As a result, external malicious interference exhibits high randomness and instability. Combined with the extremely limited prior knowledge of the interference signals, achieving efficient and accurate prediction and estimation is challenging. This renders the electromagnetic environment faced by the relay drone exceptionally complex and severe, posing significant challenges to communication performance. The dynamic interference process is illustrated in Figure 2.

Under the context of the above interference modes, the expression for the SINR of the relay drone working on the

f_{n}^{R u a v}

channel, receiving signals from cluster drone

i

, can be reformulated as follows:

\begin{matrix} S I N R_{i, R} = \frac{P^{F} (k) H_{i}^{w} (k)}{P_{T}} = \frac{P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(4 π D {(k)}_{F i}^{R})}^{2} P_{T}} i \in M, \forall k \\ P_{T} = Υ (f, f_{x}) (P^{J} H_{1}^{*} (P^{L} (L^{D} + ϑ) + P^{N L} (L^{D} + δ + ϑ)) / S) + n_{0} B_{n}^{R u a v} \\ L^{D} = 32.45 + 20 \lg (D {(k)}_{J}^{R}) + 20 \lg (f_{n}^{R u a v}) D {(k)}_{F i}^{R} = {‖ Q (k) - F_{i} (k) ‖}_{2} \end{matrix}

(9)

In this context,

S = {0, 1, ..., F^{J}}

represents the number of channels affected by interference, and

Υ (f, f_{n}^{R u a v})

denotes the channel interference indicator function, which takes a value of 0 or 1. When

f = f_{n}^{R u a v}

, this signifies that the

f_{n}^{R u a v}

channel is experiencing interference,

Υ (f, f_{n}^{R u a v}) = 1

. Conversely, when

f \neq f_{n}^{R u a v}

, this indicates that the

f_{n}^{R u a v}

frequency channel is not subject to interference,

Υ (f, f_{n}^{R u a v}) = 0

2.4. Power Transmission Model and Energy Model

Transmission power is closely related to the communication rate, ensuring reliable data transmission across varying distances and environments, while also being significant for optimizing energy consumption. Hence, a power transmission model is introduced to achieve precise measurement. The corresponding calculation expression is given by the following:

P_{T X}^{u a v} = P_{R X}^{u a v} - L^{u a v} + G_{r}^{u a v} + G_{t}^{u a v}

(10)

In this context,

P_{R X}^{u a v}

denotes the received signal power,

P_{T X}^{u a v}

represents the transmitted signal power,

L^{u a v}

indicates the path loss,

G_{r}^{u a v}

signifies the receiver antenna gain, and

G_{t}^{u a v}

signifies the transmitter antenna gain. In a drone cluster relay communication system, the system energy consumption primarily comprises the flight energy consumption and communication energy consumption of the drones. The calculation expression for flight energy consumption is given by the following:

E_{f l y}^{u a v} = \sum_{k = 1}^{K} \frac{m g}{2} {(v (k))}^{2} δ_{T}

(11)

In this context,

P^{F / R} (k)

represents the transmission power of the drone, while the calculation expression for communication energy consumption is given by the following:

E_{c}^{u a v} = \sum_{k = 1}^{K} P^{F / R} (k) δ_{T}

(12)

In this context,

v (k)

represents the drone’s flight speed at time k. In summary, the total system energy consumption,

E^{s}

, can be expressed as follows:

E^{s} = E_{f l y}^{u a v} + E_{c}^{u a v}

(13)

3. Problem Description

This section focuses on a scenario in which swarm drones transmit collected information back to the ground station in real time, exploring the modeling issues of real-time communication between the swarm drones and the ground station with the assistance of a relay drone. Assume that the working frequency of the relay drone is

f_{n}^{R u a v}

, and this channel comprises

M / 2

subcarriers for the paired transmission of information from the swarm drones. During this process, the relay drone often faces a certain degree of malicious interference. Considering the high-speed movement of drones and the complex electromagnetic environment, their channel state information (CSI) may become imperfect or outdated. In this context, the traditional assumption of perfect CSI is unrealistic. By referring to current mainstream methods for CSI estimation and prediction, such as deep learning algorithms [19], adaptive algorithms [20], and coding techniques [21], it is possible to dynamically predict and compensate for the quality of the CSI, thereby addressing the issues related to outdated and imperfect CSI. According to the principles of Non-Orthogonal Multiple Access (NOMA), the sender transmits data by allocating different levels of transmission power to different users, and the receiver decodes the signals sequentially by detecting the gain differences of the received signals, thus canceling the links with smaller channel gains [22]. In the designated task scenario, the communication link gains from swarm drone

i

and swarm drone

j

to the relay drone are

H_{i}^{w} (k)

and

H_{j}^{w} (k)

, respectively. When

H_{i}^{w} (k) < H_{j}^{w} (k)

, the relay drone decodes the link from swarm drone

j

first; otherwise, it decodes the link from swarm drone

i

first. The power allocation factor set for the swarm drones is

λ = {λ_{1}, λ_{2}, ..., λ_{M}}

, and since the swarm drones select only one channel for data transmission in different time slots, in time slot

k

, the SINR for the relay drone working on the frequency

f_{n}^{R u a v}

channel and receiving the transmission signal from swarm drone

i

is as follows:

\begin{matrix} S I N R_{i, R}^{R u a v} = \frac{λ_{i} (k) P^{F} (k) H_{i}^{w} (k)}{Υ (f, f_{x}) (P^{J} H_{1}^{*} Λ / S) + \sum_{j \in Γ} λ_{j} (k) P^{F} (k) H_{j}^{w} (k) + n_{0} B_{n}^{R u a v}} i \in M, \forall k \\ = \frac{λ_{i} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(4 π {‖ Q (k) - F_{i} (k) ‖}_{2})}^{2} (Υ (f, f_{x}) (P^{J} H_{1}^{*} Λ / S) + \sum_{j \in Γ} λ_{j} (k) P^{F} (k) H_{j}^{w} (k) + n_{0} B_{n}^{R u a v}} \\ Γ = {j | H_{j}^{w} (k) < H_{i}^{w} (k), 1 < j \leq M} \end{matrix}

(14)

Similarly, in time slot

k

, the Signal-to-Noise Ratio (SNR) for the ground station receiving the transmission signal from the relay drone is as follows:

S N R_{R, S}^{R u a v} = \frac{P^{R} (k) H^{w} (k)}{n_{0} B_{n}^{R u a v}} = \frac{P^{R} (k) G_{t} G_{r} {(λ^{G})}^{2}}{n_{0} B_{n}^{R u a v} {(4 π {‖ Q (k) ‖}_{2})}^{2}} \forall k

(15)

The relay drone employs an amplify-and-forward method to transmit information. As stated in reference [23], in time slot k, the SNR for the ground station receiving the two-hop link from cluster drone

i

is as follows:

\begin{matrix} S N R_{i, S} = \frac{S I N R_{i, R}^{C u a v} S N R_{R, S}^{R u a v}}{S I N R_{i, R}^{C u a v} + S N R_{R, S}^{R u a v} + 1} i \in M, \forall k \\ = \frac{λ_{i} (k) P^{F} (k) P^{R} (k) H_{i}^{w} (k) H^{w} (k)}{λ_{i} (k) P^{F} (k) n_{0} B_{n}^{R u a v} H_{i}^{w} (k) + I_{3}^{t} (P^{R} (k) H^{w} (k) + n_{0} B_{n}^{R u a v})} \\ = \frac{λ_{i} (k) P^{F} (k) P^{R} (k) {(G_{t} G_{r} {(λ^{G})}^{2})}^{2}}{I_{1}^{t} + I_{2}^{t} I_{3}^{t}} \\ I_{1}^{t} = λ_{i} (k) P^{F} (k) n_{0} B_{n}^{R u a v} G_{t} G_{r} {(λ^{G})}^{2} {(4 π {‖ Q (k) ‖}_{2})}^{2} \\ I_{2}^{t} = P^{R} (k) G_{t} G_{r} {(λ^{G})}^{2} {(4 π {‖ Q (k) - F_{i} (k) ‖}_{2})}^{2} + n_{0} B_{n}^{R u a v} {(4 π {‖ Q (k) ‖}_{2})}^{2} {(4 π {‖ Q (k) - F_{i} (k) ‖}_{2})}^{2} \\ I_{3}^{t} = Υ (f, f_{x}) (P^{J} H_{1}^{*} Λ / S) + \sum_{j \in Γ} λ_{j} (k) P^{F} (k) H_{j}^{w} (k) + n_{0} B_{n}^{R u a v} \end{matrix}

(16)

From this, it is evident that in time slot

k

, the communication rate between the cluster drone

i

and the ground station is as follows:

\begin{array}{l} R_{i} (k) & = B_{n}^{R u a v} l o g_{2} (1 + S N R_{i, S}) \forall k \\ = B_{n}^{R u a v} l o g_{2} (1 + \frac{λ_{i} (k) P^{F} (k) P^{R} (k) {(G_{t} G_{r} {(λ^{G})}^{2})}^{2}}{I_{1}^{t} + I_{2}^{t} I_{3}^{t}}) \end{array}

(17)

Because of the dynamic characteristics of the relay drone, the channel conditions for its assisted communication change in real time. Therefore, dynamically adjusting the channel modulation method is an important strategy to flexibly respond to real-time channel variations. Hence, this paper introduces Quadrature Amplitude Modulation (QAM) technology into relay drone-assisted communication to maximize high-performance auxiliary communication. Consequently, in time slot

k

, the communication rate between cluster drone

i

and the ground station becomes the following:

\begin{array}{l} R_{i} (k) = η B_{n}^{R u a v} l o g_{2} (1 + S N R_{i}) \forall k \\ = η B_{n}^{R u a v} l o g_{2} (1 + \frac{λ_{i} (k) P^{F} (k) P^{R} (k) {(G_{t} G_{r} {(λ^{G})}^{2})}^{2}}{I_{1}^{t} + I_{2}^{t} I_{3}^{t}}) \\ η = l o g_{2} (U (k)) \end{array}

(18)

In this case,

U (k)

represents the modulation order at time

k

, and the set of selectable modulation orders is given by

{2, 4, 16, 64}

. Therefore, the expression for the total communication rate between the cluster drone and the ground station over the entire mission period is as follows:

\begin{array}{l} R_{S}^{C} & = \sum_{k = 1}^{K} \sum_{i = 1}^{M} R_{i} (k) \forall k \\ = \sum_{k = 1}^{K} \sum_{i = 1}^{M} l o g_{2} (U (k)) B_{n}^{R u a v} l o g_{2} (1 + \frac{λ_{i} (k) P^{F} (k) P^{R} (k) {(G_{t} G_{r} {(λ^{G})}^{2})}^{2}}{I_{1}^{t} + I_{2}^{t} I_{3}^{t}}) \end{array}

(19)

From the above formula, it is evident that optimizing the total communication rate achievable by the ground station is dependent on the power allocation factors of the cluster drones, the modulation method, transmission power, and three-dimensional trajectory of the relay drone. However, it is also necessary to consider the physical constraints of the communication equipment and the performance constraints of the communication process.

The transmission power of the relay drone must not exceed its maximum transmission power,

P_{m a x}^{R}

, and the energy consumption for auxiliary communication must not exceed the maximum energy storage capacity of the relay drone,

E_{m a x}^{R u a v}

, expressed as follows:

0 \leq P^{R} (k) \leq P_{m a x}^{R}

(20)

0 \leq E_{R u a v}^{s} \leq E_{m a x}^{R u a v}

(21)

Similarly, similar constraints exist for all drones in the cluster, namely, the following:

0 \leq λ_{i} (k) P^{F} (k) \leq P_{m a x}^{F}

(22)

0 \leq E_{F u a v}^{s} \leq E_{m a x}^{F u a v}

(23)

Additionally, in each time slot, the horizontal movement trajectory of the relay drone cannot exceed the trajectory length that would be covered at its maximum horizontal speed,

V_{m a x}^{h o r}

, expressed as follows:

[{(x_{r}^{u a v} (k + 1) - x_{r}^{u a v} (k))}^{2} + {(y_{r}^{u a v} (k + 1) - y_{r}^{u a v} (k))}^{2}] \leq {(V_{m a x}^{h o r} δ_{T})}^{2} \forall k

(24)

Similarly, the vertical movement trajectory of the relay drone cannot exceed the trajectory length that would be covered at its maximum vertical speed,

V_{m a x}^{v e r}

, and the altitude of the relay drone must be within an appropriate range, expressed as follows:

(z_{r}^{u a v} (k + 1) - z_{r}^{u a v} (k))^{2} \leq {(V_{m a x}^{v e r} δ_{T})}^{2} \forall k

(25)

H_{l} \leq z_{r}^{u a v} (k) \leq H_{h} \forall k

(26)

In this case,

H_{l}, H_{h}

represent the flight altitude limit for the relay drone, with the starting point coordinates given by

U A V_{i n i t}^{R} = (x^{i n i t}, y^{i n i t}, z^{i n i t})

and the ending point coordinates by

U A V_{t e r m}^{R} = (x^{t e r m}, y^{t e r m}, z^{t e r m})

. The movement constraints also need to satisfy the following:

[{(x_{r}^{u a v} (1) - x^{i n i t})}^{2} + {(y_{r}^{u a v} (1) - y^{i n i t})}^{2}] \leq {(V_{m a x}^{h o r} δ_{T})}^{2} \forall k

(27)

[{(x^{t e r m} - x_{r}^{u a v} (K))}^{2} + {(y^{t e r m} - y_{r}^{u a v} (K))}^{2}] \leq {(V_{m a x}^{h o r} δ_{T})}^{2} \forall k

(28)

(z_{r}^{u a v} (1) - z^{i n i t})^{2} \leq {(V_{m a x}^{v e r} δ_{T})}^{2} \forall k

(29)

(z^{t e r m} - z_{r}^{u a v} (K))^{2} \leq {(V_{m a x}^{v e r} δ_{T})}^{2} \forall k

(30)

Since the auxiliary communication operates on a two-hop model, there is a single time slot delay in relay transmission [24]. Therefore, the transmission power of the cluster drones at the final time slot and the relay drone at the initial time slot is set to zero, expressed as follows:

λ_{i} (K) = 0

(31)

P^{R} (1) = 0

(32)

Here,

K

represents the last task slot. To ensure that the data volume returned by the cluster drones is sufficient for analysis and decision making at the ground station, it is required that the total communication rate of the cluster drones exceed a specified threshold

R_{l}

, expressed as follows:

R_{S}^{C} \geq R_{l}

(33)

Because of the relay drone’s ability to switch between different modulation modes based on varying channel conditions and according to different modulation orders, the formula for calculating the bit error rate (BER) also varies [25], expressed as follows:

\begin{matrix} B E R_{{2}} ≐ e r f c (\sqrt{2 S N R_{R, S}}) B E R_{{4}} ≐ 4 e r f c (\sqrt{S N R_{R, S}} \\ B E R_{{16}} ≐ 4 e r f c (\sqrt{3 S N R_{R, S} / 15} B E R_{{64}} ≐ 4 e r f c (\sqrt{3 S N R_{R, S} / 63} \\ e r f c (x) = \frac{2}{\sqrt{π}} \int_{x}^{\infty} e^{- t^{2}} d t \end{matrix}

(34)

Therefore, for each communication link formed between the cluster drones and the ground station, the BER must meet the threshold requirement

B E R_{q}

, as follows:

B E R_{{2, 4, 16, 64}} \leq B E R_{q}

(35)

Thus, the optimization model for the communication rate of cluster drones under interference conditions is constructed as follows:

C_{1} : \max_{{P^{R}, λ, U, Q}} : R_{S}^{C} \forall k

(36)

s . t . \frac{P^{F} β^{o}}{D^{2} (k) P_{T}} \geq D_{n}

(37)

Υ = {0, 1}

(38)

0 \leq λ_{i} (k) \leq 1

(39)

U = {2, 4, 16, 64}

(40)

(13) - (26), (28)

The optimization model for the communication rate of cluster drones under interference conditions integrates several constraints, as follows: channel selection constraints (37) and (38), physical constraints of relay-assisted communication (20)–(30), and communication service performance constraints (31)–(33), (35), (39), and (40). The problem is formulated as a highly coupled nonlinear mixed-integer optimization model involving both discrete and continuous variables, which presents significant computational challenges due to its complexity. To effectively tackle this, the approach is divided into the following two strategic steps: firstly, employing an intelligent channel selection scheme to choose interference-free channels, and secondly, conducting the joint optimization of the relay’s modulation method, transmission power, three-dimensional trajectory, and the cluster drones’ power allocation factors. This method aims to maximize the data transmission rate of the relay drone under interference conditions while simplifying the optimization process and enhancing solution efficiency.

4. Intelligent Channel Selection Scheme

Because of the dynamic nature of jammer interference patterns, conventional anti-interference algorithms often struggle to cope effectively. Deep reinforcement learning, as a potent decision-making tool within artificial intelligence, is capable of adapting to dynamic environments and achieving optimal outcomes. However, deep reinforcement learning algorithms strongly depend on hyperparameters, which can be a limiting factor in their application. Therefore, this section proposes an intelligent channel selection scheme based on Bayesian optimization within a Discrete Soft Actor-Critic (BO_DSAC) framework. This approach is designed to facilitate channel selection for relay drones, thereby mitigating the impact of dynamic interference on the communication of cluster drones.

4.1. Discrete Soft Actor-Critic Algorithm

The DSAC algorithm requires discussion primarily around the agent and the interfering environment. The agent mainly consists of the following three components: an action selection network, a value evaluation network, and an entropy structure. The Soft Actor-Critic algorithm adds an entropy component to the previous actor-critic network architecture to enhance the algorithm’s ability to explore global solutions and prevent convergence to local optima. Entropy is a measure of uncertainty, with higher randomness corresponding to greater entropy values. In the DSAC algorithm, the presence of entropy shifts the focus of the evaluation network from determining the cumulative rewards of actions for each state to exploring actions that could yield the maximum cumulative rewards in a given state. This adjustment addresses the shortcomings of traditional AC architectures in deep reinforcement learning. The working framework of this architecture is illustrated in Figure 3. For ease of expression, this paper denotes the soft predictive value network with

Q^{s o f t}

Q

From the diagram, it can be observed that the Discrete Soft Actor-Critic algorithm architecture consists of one policy network and four evaluation networks

Q (\cdot)

. The evaluation networks are divided into two sets, as follows: predictive value networks and their corresponding target value networks. The configuration within each set aims to prevent overestimation and ensure the stability of the learning process, while the arrangement among the sets is designed to enhance learning efficiency and further improve stability. In terms of the evaluation network, unlike algorithms for continuous action spaces, in discrete action spaces, the state is solely input into the evaluation network, which then outputs values for all possible actions corresponding to that state, denoted as

S \to ℝ^{| A |}

. The expression for the loss function is

L_{Q} (ω^{u})

L_{Q} (ω^{u}) = \underset{(s, a, r, s^{'}) ~ D_{J}}{E} [{(Q (s_{t}, a_{t}; ω^{u}) - (\begin{matrix} r_{t} (s_{t}, a_{t}) + π {(s^{'}; θ^{u})}^{T} \\ [\underset{ω_{i}^{u}}{m i n} {Q (s^{'}; ω_{i}^{u})} - α \ln (π (s^{'}; θ^{u}))] \end{matrix}))}^{2}]

(41)

In this setup,

r_{t} (s_{t}, a_{t})

represents the expected reward value when action

a_{t}

is executed in state

s_{t}

α

denotes the entropy coefficient, which signifies the weight of the entropy,

H (π (s^{'}; θ^{u})) = - α π {(s^{'}; θ^{u})}^{T} \ln (π (s^{'}; θ^{u}))

represents the entropy itself, and

D_{J}

indicates the experience replay buffer used to store interaction data between the agent and the environment. Regarding the policy network, its input remains the state, but its output is the probability distribution of discrete actions, unlike the output of mean and variance in continuous action spaces, denoted as

S \to {[0, 1]}^{| A |}

. The expression for the loss function of the policy network is

L_{π} (θ^{u})

L_{π} (θ^{u}) = \underset{s ~ D_{J}}{E} [π {(s; θ^{u})}^{T} [α \ln π (s; θ^{u}) - \underset{ω_{i}^{u}}{m i n} Q (s; ω_{i}^{u})]]

(42)

Regarding entropy, the entropy coefficient, α, determines the importance of the entropy term. To enable the adaptive variation in the entropy coefficient at different stages of learning, its loss function is designed as

L (α)

, as follows:

L (α) = α \underset{s ~ D_{J}}{E} [π {(s; θ^{u})}^{T} \ln π (s; θ^{u}) - \tilde{H}]

(43)

In this context,

\tilde{H}

represents the target entropy. The application of gradient descent to these loss functions facilitates the updating of network parameters. Under the framework of the Discrete Soft Actor-Critic algorithm, and in consideration of the actual conditions of the interference environment, the required basic elements are defined as follows:

State Space: The state space

S^{i}

should include the physical quantities that the relay drone needs to compute, primarily characterized by the energy state of the channel. Its definition is as follows:

S^{i} = {S_{m i x}^{1}, ..., S_{m i x}^{F}}

(44)

In this context,

S_{m i x}^{i}

represents the energy state of channel

i

. When the channel is idle, only environmental noise

n_{0} B_{n}^{R u a v}

is present. During normal communication, the energy is characterized by

P_{n c}

. Based on prior information, when the cluster drones communicate at full power, the maximum reception power of the relay drone is

P_{r \max} = λ_{i} P^{F} H_{i}^{w}

. When there is interference, the channel energy is represented by

P_{i f}

. The current energy state of the channel can be used to determine the communication status of the channel.

Action Space: The action space

A^{i}

represents the possible choices the relay drone can make regarding channel selection during time slot

k

. Therefore, the action space is designed as follows:

A^{i} = {a | a \in {0, 1}, 1 \leq a \leq F}

(45)

Therefore, the size of the action space is

2^{F}

, where

a = 0

indicates that the channel is not selected, and

a = 1

indicates that the channel is selected. During each interaction, the agent selects only one channel for communication at a time.

Reward Function: The real-time reward function

R^{i}

measures the approval of action

a

and is designed with the following expression:

R^{i} = {\begin{cases} 1 & (S I N R_{i, R} \geq D_{n}) \cap (f_{i} = f_{n}^{R u a v}) \\ 1 - Δ r & (S I N R_{i, R} \geq D_{n}) \cap (f_{i} \neq f_{n}^{R u a v}) \\ 0 & (S I N R_{i, R} < D_{n}) \cap (f_{i} = f_{n}^{R u a v}) \\ - Δ r & (S I N R_{i, R} < D_{n}) \cap (f_{i} \neq f_{n}^{R u a v}) \end{cases}

(46)

The relay drone receives the maximum reward if it successfully avoids interference while maintaining the same working channel. A slightly lower reward is given if the drone successfully avoids interference by switching channels. If the drone fails to avoid interference and does not change channels, it receives zero reward. However, if it changes channels and still fails to avoid interference, a penalty is applied. In this setup,

Δ r

is set to 0.4 to measure the reward for channel switching decisions.

4.2. Bayesian Optimization Algorithm

Bayesian Optimization is an efficient global optimization algorithm that can optimally extract hyperparameters with minimal evaluations using a statistical approach. The fundamental principle is based on Bayes’ theorem [26], and its expression is as follows:

p (X | Θ) = \frac{p (Θ | X) p (X)}{p (Θ)}

(47)

In this context,

X

represents the distribution model to be specified,

Θ = {(x_{i}, y_{i}) | y_{i} = X (x_{i})}

is the set of observed data,

p (Θ | X)

denotes the likelihood distribution, and

p (X)

is the prior probability. In this paper, the decision variables

{x_{i}}

include the discount factor

γ^{u}

, learning rate

l r^{u}

, and target entropy

\tilde{H}

. The cumulative reward function is the target value,

y_{i}

. The prior and posterior models use Gaussian distributions for simulation. The acquisition function, based on Expected Improvement function, is used to select the next evaluation point,

x_{i + 1}

, from the posterior distribution, and its expression is as follows:

\begin{matrix} x_{i + 1} = a r g m a x E I (x_{i}, Θ) \\ E I (x_{i}) = E [(m a x (X (x_{i}) - X^{+}), 0) | x_{i}] \end{matrix}

(48)

In this context,

X^{+}

represents the current known maximum value simulated by the Gaussian process. This establishes a probabilistic optimization model concerning the discount factor, learning rate, and target entropy. By continuously selecting the next point through the acquisition function, the process iterates until the optimal solution is found.

Summarizing, the implementation process of the BO_DSAC intelligent channel selection scheme is detailed as follows in Algorithm 1.

Algorithm 1 BO_DSAC intelligent channel selection scheme

Inputs: discount factor

γ^{u}

, learning rate

l r^{u}

, and target entropy

\tilde{H}

, each with their respective optimization ranges. Additionally, inputs include the entropy coefficient

α

, maximum number of training iterations num_episodes, Markov chain length

M D P L

, soft update parameter

τ^{l s}

, capacity of the replay buffer p_size, batch size for data processing

b a t c h_s i z e

, and the number of iterations for Bayesian Optimization

B S N

. These parameters collectively set the framework for optimizing the intelligent channel selection strategy.

Output: policy function

π^{B} (\cdot)

and value function

Q^{B} (\cdot)

1: Inputs discount factor

γ^{u}

, learning rate

l r^{u}

, and target entropy

\tilde{H}

, each with their respective optimization ranges. Additionally, inputs include the entropy coefficient

α

, maximum number of training iterations num_episodes, Markov chain length

M D P L

, soft update parameter

τ^{l s}

, capacity of the replay buffer

p_{s i z e}

, batch size for data processing

b a t c h_s i z e

, and the number of iterations for Bayesian optimization,

B S N

. These parameters collectively set the framework for optimizing the intelligent channel selection strategy.

2: Initialize the replay buffer

D_{J} \neq \emptyset

and the network parameters of the policy and evaluation networks

{θ^{u}, ω_{i}^{u}, i = 1, 2, 3, 4}

f o r i = 1 : n u m_e p i s o d e s

4: Initial state:

s = s^{0}

f o r j = 1 : M D P L

6: Choose action:

a \sim π (\cdot)

7: Perform the action, receive the reward, and form the data tuple:

< s, a, r, s^{'} >

8: Update State:

s = s^{'}

9: Store the data tuple in the replay buffer:

D = D \cup < s, a, r, s^{'} >

10:

i f | D_{J} | > p_{s i z e}

11: Delete the earliest data tuple from the replay buffer.

12:

e n d i f

13:

i f | D_{J} | \geq b a t c h_s i z e

14: Randomly sample

b a t c h_s i z e

data tuples

{< s, a, r, s^{'} >}

15: Calculate the loss function of the soft predictive value network

L_{Q} (ω^{u})

16: Calculate the loss function of the soft policy network

L_{π} (θ^{u})

17: Calculate the entropy loss function

L (α)

18: Minimize the

L_{Q} (ω^{u})

loss using the data tuples to train the network parameters.

19: Maximize the

L_{π} (θ^{u})

using the data tuples to train the network parameters.

20: Minimize the

L (α)

loss using the data tuples to adaptively adjust the entropy coefficient.

21: Soft update:

{ω^{'}}_{i}^{u} = τ^{l s} ω_{i}^{u} + (1 - τ^{l s}) ω_{i}^{u} i = 1, 2

22:

e n d i f

23:

e n d f o r

24:

e n d f o r

25: Output the final policy function

π^{*} (\cdot)

and value function

Q^{*} (\cdot)

26: Obtain the initial observation set

Θ

based on prior probabilities, and construct the prior Gaussian distribution and the posterior Gaussian distribution.

27:

f o r n = 1 : BSN

28: Determine

x_{i + 1}

using the

E I

function.

29: Calculate the target function value

y_{i + 1}

corresponding to

x_{i + 1}

30: Update the Gaussian model.

31:

e n d f o r

32: Output the optimal discount factor

γ^{u *}

, learning rate

l r^{u *}

, and target entropy

{\tilde{H}}^{*}

33: Incorporate the optimal hyperparameters

γ^{u *}, l r^{u *}, {\tilde{H}}^{*}

, and re-execute Steps 2–25.

The computational overhead of the Bayesian optimized Discrete Flexible Actor-Critic algorithm comprises the following two parts: the Discrete Flexible Actor-Critic algorithm and the Bayesian optimization process. The Discrete Flexible Actor-Critic algorithm includes the complexity of network forward and backward propagation

O (N^{n})

, where

N^{n}

denotes the number of network layers; the complexity of experience replay

O (b a t c h_s i z e)

; and the complexity of policy and value updates

O (k N^{n})

, with

k

being the number of network updates. Hence, its computational overhead is

O (b a t c h_s i z e \cdot N^{n})

. The second part involves the Bayesian optimization process, which includes the complexity of Gaussian process training

O (u^{3})

, prediction complexity

O (u)

, and acquisition function optimization complexity

O (u^{r})

, with

u^{r}

representing the number of evaluated candidate points. Therefore, the overall computational complexity of the proposed algorithm in this section is

O (u^{3} + u \cdot u^{r} + u^{r} + b a t c h_s i z e \cdot N^{n})

, primarily dominated by

O (u^{3})

5. Joint Optimization Scheme for Communication Rate

After intelligent channel selection, the relay drone can avoid the impact of jammers and achieve information transmission on interference-free channels. To further efficiently decouple the multivariable mixed-integer nonlinear optimization problem, this section divides the joint optimization problem of communication rate into the following four subproblems: relay drone modulation method optimization, relay drone transmission power optimization, three-dimensional trajectory optimization, and cluster drone power allocation factor optimization. These subproblems are iteratively solved alternately until convergence, thereby maximizing the communication rate of the cluster drones.

5.1. Relay Drone Modulation Method Optimization

To study the optimization problem of the relay drone modulation method

U (k)

, it is necessary to fix variables such as the power allocation factor,

λ^{(c - 1)} (k)

, of the cluster drones, the transmission power,

P^{R, (c - 1)} (k)

, of the relay drone, and trajectory,

Q^{(c - 1)} (k)

, during the

(c - 1)

th iteration in time slot

k

. Thus, the original optimization problem

C_{1}

can be simplified as follows:

\begin{matrix} C_{2} : \max_{{U}} : \sum_{k = 1}^{K} \sum_{i = 1}^{M} c_{1}^{U} l o g_{2} (U^{(c)} (k)) \\ (33), (35), (40) \end{matrix}

(49)

Wherein

c_{1}^{U} = B_{n}^{R u a v} l o g_{2} (1 + λ_{i}^{(c - 1)} (k) P^{F} (k) P^{R, (c - 1)} (k) {(G_{t} G_{r} {(λ^{G})}^{2})}^{2} / {(I_{1}^{t} + I_{2}^{t} I_{3}^{t^{'}})}^{(c - 1)})

, and

I_{3}^{t^{'}}

is given by

\sum_{j \in Γ} λ_{j}^{(c - 1)} (k) P^{F} (k) H_{j}^{w, (c - 1)} (k) + n_{0} B_{n}^{R u a v}

. Because of the inconvenient computation of discrete variable values involved in the above optimization problem, the auxiliary variables

b_{1}, b_{2}, b_{3}, b_{4}

are introduced, thus transforming the optimization problem

C_{2}

into the following:

\begin{matrix} C_{3} : \max_{{U, b_{1}, b_{2}, b_{3}, b_{4}}} : \sum_{k = 1}^{K} \sum_{i = 1}^{M} c_{1}^{U} l o g_{2} (U^{(c)} (k)) \\ U^{(c)} (k) = 2 b_{1} + 4 b_{2} + 16 b_{3} + 64 b_{4} \\ \frac{P^{R, (c - 1)} (k) H_{i}^{w, (c - 1)} (k)}{n_{0} B_{n}^{R u a v}} \geq b_{1} f_{1} + 3 b_{2} f_{2} + 15 b_{3} f_{2} + 63 b_{4} f_{2} \\ f_{1} = {(e r f c^{- 1} (B E R_{q}))}^{2} / 2 f_{2} = {(e r f c^{- 1} (B E R_{q} / 4))}^{2} / 3 \\ b_{1} + b_{2} + b_{3} + b_{4} = 1 b_{1}, b_{2}, b_{3}, b_{4} \in {0, 1} \\ (33) \end{matrix}

(50)

The above issue is a 0–1 integer programming problem, which can be directly solved using the Mosek optimization toolkit to obtain the

U^{(c)} (k)

value.

5.2. Optimization of Relay Drone Transmission Power

To study the optimization problem of relay drone transmission power

P^{R} (k)

, it is necessary to fix the power allocation factor

λ^{(c - 1)} (k)

of the cluster drones in the

(c - 1)

th round of iterations within time slot

k

, the 3D trajectory

Q^{(c - 1)} (k)

, and the modulation method

U^{(c)} (k)

of the relay drones that was just updated in the

(c)

th round. Consequently, the original optimization problem

C_{1}

can be simplified as follows:

\begin{matrix} C_{4} : \max_{{P^{R}}} : R_{S}^{C, P^{R}} \\ (21), (22), (32), (33), (35) \end{matrix}

(51)

From Equation (34), the computational relationship between

B E R_{{2, 4, 16, 64}}

and the transmission power

P

is as follows:

P = {\begin{cases} {(e r f c^{- 1} (B E R))}^{2} n_{0} B_{n}^{R u a v} / 2 H_{i}^{w} U = {2} \\ {(e r f c^{- 1} (B E R / 4))}^{2} (U - 1) n_{0} B_{n}^{R u a v} / 3 H_{i}^{w} U = {4, 16, 64} \end{cases}

(52)

Then, constraint (35) can be transformed into:

P^{R, (c)} (k) \geq \frac{n_{0} B_{n}^{R u a v} {(b_{1} f_{1} + 3 b_{2} f_{2} + 15 b_{3} f_{2} + 63 b_{4} f_{2})}^{(c - 1)} {(4 π {‖ Q (k) ‖}_{2}^{(c - 1)})}^{2}}{G_{t} G_{r} {(λ^{G})}^{2}}

(53)

Then the above optimization problem

C_{4}

becomes the following:

\begin{matrix} C_{5} : \max_{{P^{R}}} : R_{S}^{C, P^{R}} \\ (21), (22), (32), (33), (53) \end{matrix}

(54)

The above optimization problem

C_{5}

is convex and can be solved by directly applying the CVX toolkit to obtain a

P^{R, (c)} (k)

global suboptimal solution.

Proof: Let the expression of the objective function,

c_{1}^{p r} = λ_{i}^{(c - 1)} (k) P^{F} (k) H_{i}^{w, (c - 1)} (k)

H^{w, (c - 1)} (k)

c_{2}^{p r} = I_{3}^{t} H^{w, (c - 1)} (k)

c_{3}^{p r} = λ_{i}^{(c - 1)} (k) P^{F} (k) n_{0} B_{n}^{R u a v} H_{i}^{w, (c - 1)} (k) + {\tilde{I}}_{3}^{t} n_{0} B_{n}^{R u a v}

R_{S}^{C, P^{R}}

be:

f (P^{R, (c)} (k)) = \frac{c_{1}^{p r} P^{R, (c)} (k)}{c_{2}^{p r} P^{R, (c)} (k) + c_{3}^{p r}}

(55)

Taking the second order derivative of

f (P^{R, (c)} (k))

results in the following:

f^{″} (P^{R, (c)} (k)) = \frac{- 2 c_{1}^{p r} c_{2}^{p r} c_{3}^{p r}}{{(c_{2}^{p r} P^{R, (c)} (k) + c_{3}^{p r})}^{3}}

(56)

From the constraint restrictions and physical significance of each variable,

c_{1}^{p r} \geq 0, c_{2}^{p r} \geq 0, c_{3}^{p r} \geq 0, P^{R, (c)} (k) \geq 0

, the second order derivative

f^{″} (P^{R, (c)} (k)) \leq 0

, and the objective function is concave. Also, the restriction constraints satisfy the convex set property and hence the optimization problem

C_{5}

is convex, as evidenced.

5.3. Optimization of Three-Dimensional Trajectory for Relay Drones

To study the optimization problem of the three-dimensional trajectory,

Q (k)

, of the relay drones, it is necessary to fix the power allocation factor,

λ^{(c - 1)} (k)

, of the cluster drones during the

(c - 1)

th round of iterations within time slot

k

and the modulation method

U^{(c)} (k)

of the relay drones that was just updated in the

(c)

th round, as well as the transmission power

P^{R, (c)} (k)

. Consequently, the original optimization problem

C_{1}

can be simplified as follows:

\begin{matrix} C_{6} : \max_{{Q}} : R_{S}^{C, Q} \\ (24) - (30), (33) \end{matrix}

(57)

In optimization problem

C_{6}

, the objective function expression is nonconvex, has a nonlinear relationship with

Q (k)

, and its components are interdependent. Additionally, the limiting constraints do not meet the requirements for a convex problem, thus making problem

C_{6}

a nonlinear nonconvex optimization problem. Because

\partial S N R_{i, S} / \partial S I N R_{i, R}^{R u a v}

\geq 0

and

\partial S N R_{i, S} / S N R_{R, S}^{R u a v} \geq 0

, the objective function can be transformed as follows:

\max_{{Q}} : \min {\sum_{k = 1}^{K} \sum_{i = 1}^{M} \frac{λ_{i} (k) P^{F} (k) H_{i}^{w} (k)}{\sum_{j \in Γ} λ_{j} (k) P^{F} (k) H_{j}^{w} (k) + n_{0} B_{n}^{R u a v}}, \sum_{k = 1}^{K} \frac{P^{R} (k) H^{w} (k)}{n_{0} B_{n}^{R u a v}}}

(58)

The changed objective function is still difficult to solve, in order to effectively solve this problem, this paper adopts the quadratic transformation to deal with the above objective function expression, so that

ϕ_{1}, ϕ_{2}

is the new objective function value, then the objective function becomes the following:

\begin{matrix} \max_{{Q}} : \min {ϕ_{1}^{(c)}, ϕ_{2}^{(c)}} \\ ϕ_{1}^{(c)} = \sum_{k = 1}^{K} \sum_{i = 1}^{M / 2} c_{1}^{Q} l o g_{2} (a_{i} \sqrt{\begin{array}{l} \frac{λ_{i}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{(c)} {(k)}_{F i}^{R})}^{2}} + \\ \frac{λ_{j}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{(c)} {(k)}_{F j}^{R})}^{2}} + n_{0} B_{n}^{R u a v} \end{array}} - a_{i}^{2} n_{0} B_{n}^{R u a v}) \\ ϕ_{1}^{(c)} = \sum_{k = 1}^{K} c_{1}^{Q} l o g_{2} (1 + b_{i} \sqrt{P^{R, (c)} (k) G_{t} G_{r} {(λ^{G} / 4 π)}^{2} / n_{0} B_{n}^{R u a v}} - b_{i}^{2} {(D^{(c)} {(k)}_{R}^{S})}^{2}) \\ c_{1}^{Q} = l o g_{2} (U^{(c)} (k)) B_{n}^{R u a v} \end{matrix}

(59)

where the

a_{i}, b_{i}

expressions are, respectively, as follows:

a_{i} = \frac{\sqrt{\frac{λ_{i}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{*} {(k)}_{F i}^{R})}^{2}} + \frac{λ_{j}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{*} {(k)}_{F j}^{R})}^{2}} + n_{0} B_{n}^{R u a v}}}{n_{0} B_{n}^{R u a v}} k \in K, i \in M

(60)

b_{i} = \frac{\sqrt{P^{R, (c)} (k) G_{t} G_{r} {(λ^{G} / 4 π)}^{2} / n_{0} B_{n}^{R u a v}}}{{(D^{*} {(k)}_{R}^{S})}^{2}} k \in K, i \in M

(61)

where

D^{*} {(k)}_{F i}^{R}, D^{*} {(k)}_{F j}^{R}, D^{*} {(k)}_{R}^{S}

are the distances between the two hops obtained in the previous iteration, respectively, and the optimization problem becomes the following:

\begin{matrix} C_{7} : \max_{{Q, H_{i}^{w}, H^{w}}} : (59) \\ (24) - (30), (33) \end{matrix}

(62)

Unfortunately, because

1 / D^{*} {(k)}_{F i}^{R}, 1 / D^{*} {(k)}_{F j}^{R}

exist, this leads to the problem that

C_{7}

is still nonconvex, and the solution to this problem is still difficult. So this paper applied continuous convex approximation to solve the problem [27]. First-order Taylor expansion was carried out for

(\frac{λ_{i}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{(c)} {(k)}_{F i}^{R})}^{2}} + \frac{λ_{j}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{(c)} {(k)}_{F j}^{R})}^{2}}

+ n_{0} B_{n}^{R u a v})^{- 0.5}

. Since

Q (k) = (x_{r}^{u a v} (k), y_{r}^{u a v} (k), z_{r}^{u a v} (k))

, the first-order derivative value of the above expression for

Q (k)

should be a trajectory along three dimensions (X,Y,Z). The first-order partial derivative values of

O_{X}^{c}, O_{Y}^{c}, O_{Z}^{c}

for the Taylor expansion expression of

Q (k)

are as follows:

\begin{array}{l} O_{X}^{c} = - \frac{\begin{array}{l} λ_{i}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2} (x_{r}^{u a v} {(k)}^{(c - 1)} - x_{i} (k)) / {(D^{(c - 1)} {(k)}_{F i}^{R})}^{4} + \\ λ_{j}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2} (x_{r}^{u a v} {(k)}^{(c - 1)} - x_{j} (k)) / {(D^{(c - 1)} {(k)}_{F j}^{R})}^{4} \end{array}}{X} \\ O_{Y}^{c} = - \frac{\begin{array}{l} λ_{i}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2} (y_{r}^{u a v} {(k)}^{(c - 1)} - y_{i} (k)) / {(D^{(c - 1)} {(k)}_{F i}^{R})}^{4} + \\ λ_{j}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2} (y_{r}^{u a v} {(k)}^{(c - 1)} - y_{j} (k)) / {(D^{(c - 1)} {(k)}_{F j}^{R})}^{4} \end{array}}{X} \\ O_{Z}^{c} = - \frac{\begin{array}{l} λ_{i}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2} (z_{r}^{u a v} {(k)}^{(c - 1)} - z_{i} (k)) / {(D^{(c - 1)} {(k)}_{F i}^{R})}^{4} + \\ λ_{j}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2} (z_{r}^{u a v} {(k)}^{(c - 1)} - z_{j} (k)) / {(D^{(c - 1)} {(k)}_{F j}^{R})}^{4} \end{array}}{X} \\ X = \sqrt{\frac{λ_{i}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{(c - 1)} {(k)}_{F i}^{R})}^{2}} + \frac{λ_{j}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{(c - 1)} {(k)}_{F j}^{R})}^{2}} + n_{0} B_{n}^{R u a v}} \end{array}

(63)

The expanded objective function expression becomes the following:

\begin{matrix} \sqrt{\frac{λ_{i}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{(c)} {(k)}_{F i}^{R})}^{2}} + \frac{λ_{j}^{(c - 1)} (k) P^{F} (k) G_{t} G_{r} {(λ^{G})}^{2}}{{(D^{(c)} {(k)}_{F j}^{R})}^{2}} + n_{0} B_{n}^{R u a v}} ≐ X \\ + O_{X}^{c} (k) (x_{r}^{u a v} {(k)}^{(c)} - x_{r}^{u a v} {(k)}^{(c - 1)}) + O_{Y}^{c} (k) (y_{r}^{u a v} {(k)}^{(c)} - y_{r}^{u a v} {(k)}^{(c - 1)}) \\ + O_{Z}^{c} (k) (z_{r}^{u a v} {(k)}^{(c)} - z_{r}^{u a v} {(k)}^{(c - 1)})] \end{matrix}

(64)

The optimization problem

C_{8}

is obtained by bringing Equations (60), (61), and (64) into problem

C_{7}

for collation, as follows:

\begin{matrix} C_{8} : \max_{{Q, x_{r}^{u a v}, y_{r}^{u a v}, z_{r}^{u a v}}} : {\tilde{φ}}_{q}^{(c)} \\ {\tilde{φ}}_{q}^{(c)} \geq R_{l} \\ (24) - (30) \end{matrix}

(65)

The

l o g_{2} (\cdot)

is a monotonically increasing concave function, the quadratic transformation of the partition in problem

C_{6}

must be a concave function, and in Reference [28], the optimization problem

C_{8}

after the quadratic transformation of problem

C_{6}

is a convex problem, which can be solved by applying the CVX toolkit in order to obtain the global suboptimal solution.

5.4. Optimization of Power Allocation Factor for Cluster Drones

To study the optimization problem of the power allocation factor

λ

for cluster drones, it is necessary to fix variables such as the recently updated modulation method

U^{(c)} (k)

of the relay drone, the transmission power

P^{R, (c)} (k)

, and the three-dimensional trajectory

Q^{(c)} (k)

during the

c

th round within time slot

k

. Given

\partial S N R_{i, S} / \partial S I N R_{i, R}^{R u a v}

\geq 0

, and with the other variables held constant, the optimization of the transmission power for cluster drones across different time slots is independent [29]. Let

φ

be the new objective function value, with its expression as follows:

φ^{(c)} = l o g_{2} (U^{(c)} (k)) B_{n}^{R u a v} \sum_{i = 1}^{M} l o g_{2} (1 + \frac{λ_{i}^{(c)} (k) P^{F} (k) H_{i}^{w, (c)} (k)}{\sum_{j \in Γ} λ_{j}^{(c)} (k) P^{F} (k) H_{j}^{w, (c)} (k) + n_{0} B_{n}^{R u a v}})

(66)

However, the objective function

φ^{(c)}

is still not a convex function and the problem is still difficult to solve; fortunately, Equation (66) is satisfied, as follows:

\begin{matrix} φ^{(c)} = l o g_{2} (U^{(c)} (k)) B_{n}^{R u a v} \sum_{i = 1}^{M} l o g_{2} (1 + \frac{λ_{i}^{(c)} (k) P^{F} (k) H_{i}^{w, (c)} (k)}{\sum_{j \in Γ} λ_{j}^{(c)} (k) P^{F} (k) H_{j}^{w, (c)} (k) + n_{0} B_{n}^{R u a v}}) \\ = l o g_{2} (U^{(c)} (k)) B_{n}^{R u a v} l o g_{2} (\frac{λ_{i}^{(c)} (k) P^{F} (k) H_{i}^{w, (c)} (k) + \sum_{j \in Γ} \begin{array}{l} λ_{j}^{(c)} (k) P^{F} (k) H_{j}^{w, (c)} (k) \\ + n_{0} B_{n}^{R u a v} \end{array}}{\sum_{j \in Γ} λ_{j}^{(c)} (k) P^{F} (k) H_{j}^{w, (c)} (k) + n_{0} B_{n}^{R u a v}} \\ \times \frac{\sum_{j \in Γ} λ_{j}^{(c)} (k) P^{F} (k) H_{j}^{w, (c)} (k) + n_{0} B_{n}^{R u a v}}{\sum_{l \in Γ / {j}} λ_{l}^{(c)} (k) P^{F} (k) H_{l}^{w, (c)} (k) + n_{0} B_{n}^{R u a v}} \times ... \times \frac{λ_{s}^{(c)} (k) P^{F} (k) H_{s}^{w, (c)} (k) + n_{0} B_{n}^{R u a v}}{n_{0} B_{n}^{R u a v}}) \\ = l o g_{2} (U^{(c)} (k)) B_{n}^{R u a v} l o g_{2} (\frac{\sum_{j \in Γ \cup {i}} λ_{i}^{(c)} (k) P^{F} (k) H_{i}^{w, (c)} (k) + n_{0} B_{n}^{R u a v}}{n_{0} B_{n}^{R u a v}}) = {\tilde{φ}}^{(c)} \end{matrix}

(67)

Thus, the original optimization problem

C_{1}

can be varied as follows:

\begin{matrix} C_{9} : \max_{{λ}} : {\tilde{φ}}^{(c)} \\ (22), (23), (31), (33), (39) \end{matrix}

(68)

In the above optimization problem

C_{9}

, the objective function

{\tilde{φ}}^{(c)}

is a concave function of the optimization variables

λ^{(c)} (k)

Proof: Take the second order derivative of

{\tilde{φ}}^{(c)}

, as follows:

\frac{d^{″} {\tilde{φ}}^{(c)}}{d^{″} λ_{i}^{(c)} (k)} = - \frac{l o g_{2} (U^{(c)} (k)) B_{n}^{R u a v} n_{0} B_{n}^{R u a v} {(P^{F} (k) H_{i}^{w, (c)} (k))}^{2}}{\ln 2 {(\sum_{j \in Γ \cup {i}} λ_{i}^{(c)} (k) P^{F} (k) H_{i}^{w, (c)} (k) + n_{0} B_{n}^{R u a v})}^{2}}

(69)

From the constraints on the variables and the physical significance of the constraints, it can be seen for the second-order derivative that

d^{″} {\tilde{φ}}^{(c)} / d^{″} λ_{i}^{(c)} (k) \leq 0

; then, the objective function is a concave function. Also, the restriction constraints satisfy the convex set property, so the optimization problem is convex, as proven. Therefore problem

C_{9}

can be solved with CVX toolkit to obtain a global suboptimal solution for

λ^{(c)} (k)

In summary, because of the solver employs mixed-integer computation and the interior-point method to solve each subproblem, the computational complexity of the joint optimization of the relay drone communication rate is

O ({(W I V)}^{3})

, where

W

is the total number of iterations of the algorithm,

I

is the maximum number of iterations for the subproblems, and

V

is the maximum dimension of the decision variables. The algorithm execution process is shown in Algorithm 2.

Algorithm 2 Flow of joint communication rate optimization algorithm

Input: Use the coordinates of the cluster drones, the relay drone, the jammer, the ground station, and constraints such as the BER threshold to calculate the initial feasible solutions

P^{R, (0)}

Q^{(0)}

λ^{(0)}

, and

U^{(0)}

. Initialize the optimal value storage space

ℵ

, the current iteration count c = 1, the maximum number of iterations

c_{\max}

, and the precision

μ

Output:

U^{*}, {(P^{R})}^{*}, Q^{*}, λ^{*}

1: Input the coordinates of the cluster drones, the relay drone, the jammer, the ground station, and constraints such as the BER threshold to calculate the initial feasible solution

P^{R, (0)}

Q^{(0)}

λ^{(0)}

, and

U^{(0)}

. Initialize the optimal value storage space

ℵ

, the current iteration count

c = 1

, the maximum number of iterations

c_{\max}

, and the precision

μ

W h i l e ((c \leq c_{\max}) & & (τ \leq μ))

3: Optimize the modulation method of the relay drone:

4: Initial value of input

{(P^{R})}^{(c - 1) *}

Q^{(c - 1) *}

λ^{(c - 1) *}

5: Apply the Mosek toolkit to compute problem

C_{3}

and obtain a Suboptimal solution for the relay drone modulation method

U^{(c) *}

6: Updating the suboptimal solution for the relay drone modulation method obtained in the

(c - 1)

th round of iterations

U^{(c) *}

7: Optimize the transmission power of the relay drone:

8: Initial value of input

U^{(c) *}

Q^{(c - 1) *}

λ^{(c - 1) *}

9: From Equation (52), the transmission power constraint corresponding to the BER threshold is calculated.

10: Use the CVX toolkit to solve problem

C_{5}

and obtain the suboptimal solution

P^{R, (c) *}

for the transmission power of the relay drone.

11: Update the suboptimal solution

P^{R, (c) *}

for the relay drone’s transmission power obtained in the

(c)

th round of iteration.

12: Optimize the three-dimensional trajectory of the relay drone’s transmission:

13: Initial value of input

U^{(c) *}

P^{R, (c) *} (k)

, and

λ^{(c - 1) *}

14: Input the two-hop channel gain value obtained by solving in the previous iteration and calculate the value of expressions

a_{i}

and

b_{i}

according to Equation (60) and Equation (61).

15: Rewriting the objective function from Equation (59) yields

φ_{q}^{(c)}

16: Calculate the values of the derivatives of

\sqrt{H_{i}^{w, (c)} (k) H^{w, (c)} (k)}

with respect to

Q (k)

in three different dimensions to obtain

O_{X}^{c}, O_{Y}^{c}, O_{Z}^{c}

, and calculate the first-order Taylor expansion expression to obtain

\sqrt{H_{i}^{w, (c)} (k) H^{w, (c)} (k)}

17: Substituting the first-order Taylor expansion expression into problem

C_{7}

collates problem

C_{8}

18: Use the CVX toolbox to solve problem

C_{8}

and obtain the suboptimal solution

Q^{(c) *}

for the three-dimensional trajectory of the relay drone.

19: Update the suboptimal solution

Q^{(c) *}

for the three-dimensional trajectory of the relay drone obtained in the

(c)

th round of iteration.

20: Optimize the power allocation factor for cluster drones:

21: Initial value of input

U^{(c) *}

P^{R, (c) *} (k)

, and

Q^{(c) *}

22: Group the cluster drones to obtain user sets for different subcarriers

U^{u}

23: Calculate and rank the signal-to-noise ratios of the cluster drones within each group reaching the relay drone to determine the order of channel gains

{A_{i, R}^{R u a v} > ... > A_{j, R}^{R u a v}}

24: Referring to the channel gain order, rewriting the objective function according to Equation (67) yields

{\tilde{φ}}^{(c)}

25: Use the CVX toolbox to solve problem

C_{9}

and obtain the suboptimal solution

λ^{(c) *}

for the power allocation factor of cluster drones.

26: Update the suboptimal solution

λ^{(c) *}

for the power allocation factor of cluster drones obtained in the

(c)

th round of iteration.

27:

ℵ^{(c)} = [U^{(c) *}, {(P^{R})}^{(c) *}, Q^{(c) *}, λ^{(c) *}]

28: Calculate

τ^{(c)} = | R_{S}^{C, (c)} - R_{S}^{C, (c - 1)} |

29:

c = c + 1, τ = τ^{(c)}

30:

E n d W h i l e

6. Experimental Simulation

To verify the joint optimization effect of the communication rate for cluster drones under interference conditions as proposed in this paper, two comparative experiments were set up. These experiments allowed for observing the channel selection performance of the relay drone and the optimization of the total communication rate. It is assumed that the formation spacing of cluster drones is 200m, and their information backhaul tasks are performed within a three-dimensional space of 3500 m × 3500 m × 100 m. Table 2 lists the experimental parameters set for this study.

6.1. Verification of the BO_DSAC Algorithm’s Effectiveness

The purpose of the first set of experiments was to verify whether the relay drone’s channel selection using the BO_DSAC algorithm can effectively avoid external malicious interference. The types of interference set in this study included constant frequency interference, sweep frequency interference, and hybrid frequency interference. The number of channels affected by the constant frequency interference was

F / 2

, the number of sweep frequency interference channels was

F / 2

, and each cycle moved down six channels in the decreasing frequency direction. The hybrid frequency interference consisted of constant frequency interference on three channels combined with sweep frequency interference that moved down three channels each cycle in the decreasing frequency direction. To verify the effectiveness and superiority of the algorithm presented in this paper, it is compared with the Convolutional Neural Network-Enhanced DQN (Conv_DQN) algorithm, the standard DQN algorithm, and random selection. To ensure fairness, the three algorithms were set with the same number of training sessions

(n u m_e p i s o d e s = 1000)

, number of interactions

(M D P L = 200)

, network layers

(N^{n} = 256)

, learning rate

(l r^{u} = 5 \times 10^{- 4})

, discount factor

(γ^{u} = 0.97)

, and training batch size

(b a t c h_s i z e = 48)

. Figure 4, Figure 5 and Figure 6 respectively show the anti-interference effects of the three types of interference.

Figure 4 illustrates the anti-interference performance of the BO_DSAC algorithm under a constant frequency interference scenario. From the graph, it is evident that the BO_DSAC begins to converge gradually between 70 and 100 training rounds, achieving maximum reward values upon convergence. Although the constant frequency interference model is relatively simple, the algorithm demonstrates optimal performance, profoundly reflecting the superiority of the approach proposed in this paper. The introduction of entropy in the BO_DSAC algorithm leads to occasional fluctuations in reward values across some rounds; however, this also confirms that the BO_DSAC maintains a robust search for solutions throughout the training process, preventing it from settling into local optima. The graph also shows that compared to the Conv_DQN and DQN algorithms, the proposed algorithm converges faster and exhibits more pronounced resistance to interference.

Figure 5 shows the comparison of resistance to sweep frequency interference effects. From the graph, it is clear that due to the high dynamism of sweep frequency interference, the resistance effect is not as pronounced as that against constant frequency interference. However, the BO_DSAC algorithm begins to converge after about 30 rounds, demonstrating better performance in terms of timeliness and accuracy compared to the Conv_DQN and DQN algorithms. It adapts more quickly and flexibly to the dynamic environment’s impact on the information backhaul of cluster drones.

Figure 6 presents a comparison of resistance to hybrid frequency interference effects. From the chart, it is evident that the BO_DSAC algorithm begins to gradually converge around 100 rounds and maintains a relatively stable resistance to interference after convergence, showing a more significant performance compared to other algorithms. Even though hybrid frequency interference is a challenging type of interference in real-world scenarios, and the high dynamism of the interference caused by Δr settings leads to an increased frequency of channel selection, which might reduce the reward results, the algorithm proposed in this article still offers a robust anti-interference effect.

Figure 7 illustrates the impact of different hyperparameter sets on the BO_DSAC algorithm, conducted against a backdrop of hybrid frequency interference using Bayesian optimization. The optimized hyperparameters include the discount factor

γ^{u *}

, learning rate

l r^{u *}

, and target entropy

{\tilde{H}}^{*}

. The discount factor is used to determine the current value of future rewards. A higher discount factor makes the algorithm place more emphasis on future rewards, allowing the channel selection to achieve the best results over the entire task period. Conversely, a low discount factor will lead to “short-sightedness”, ignoring beneficial long-term behaviors and increasing interference avoidance errors. The learning rate determines the speed of policy updates. An excessively high learning rate may cause instability during training, resulting in significant fluctuations in anti-interference performance. On the other hand, a low learning rate will slow down the learning process, making the anti-interference effects less noticeable and convergence slower. The target entropy is used to control the randomness of the policy. A high entropy value encourages the policy to explore more of the state space, maintaining channel stability while finding channels that successfully avoid interference. Conversely, a low entropy value makes the policy more deterministic, which can lead to local optima and unsatisfactory interference avoidance results. Therefore, appropriate hyperparameters can enhance the algorithm’s ability to handle interference and improve the efficiency of relay communication in the drone cluster.

To enhance the anti-interference performance throughout the entire task period and achieve efficient policy exploration, this paper implements dynamic control of the entropy coefficient, maintaining the entropy coefficient without exponential changes with training iterations. The initial set of hyperparameters is denoted as (0.9,1.0 × 10⁻⁴,10). According to [30], mainstream hyperparameter sets that have shown better performance are used for comparison experiments, denoted as

(0.99, 5 \times 10^{- 4}, 0 . 98)

, corresponding to the BO_DSAC_III and BO_DSAC_II curves in the figure. BO_DSAC_I represents the parameter set optimized by Bayesian optimization, denoted as

(0.97, 5 \times 10^{- 4}, 0 . 6)

. As shown in the figure, the parameter set optimized by Bayesian optimization exhibits a certain degree of improvement in anti-interference performance. Compared to the parameters during optimization and the nonoptimized parameter set, the total reward values increased by 7.1% and 3.3947 times, respectively. This demonstrates that the proposed algorithm can significantly enhance the problem-solving capability of the algorithm.

Figure 8, Figure 9 and Figure 10 illustrate the action selection of three anti-interference algorithms under various interference modes. As seen from the figures, the proposed algorithm exhibits the lowest action selection uniformity under the constant frequency interference mode. This is closely related to the constant frequency interference mode, as the algorithm, after convergence, consistently selects the optimal action, which corresponds to its reward performance. In contrast, under the hybrid frequency interference mode, the action selection uniformity is the highest due to the presence of entropy, which prompts the relay drone to continuously explore new solutions to maximize entropy.

With the Conv_DQN and DQN algorithms, the action selection was relatively uniform across the three interference modes, with each action in the action space having an occurrence frequency of approximately 10%. Additionally, the Conv_DQN algorithm demonstrated better action uniformity compared to the DQN algorithm.

6.2. Verification of the Joint Optimization Algorithm Effectiveness for Cluster Drone Communication Rates

The second set of experiments primarily verified the joint optimization results for the communication rates of cluster drones. To maintain generality and verify the adaptability of the algorithm proposed in this paper to different task trajectories of cluster drones, three distinct types of trajectories are set up as follows: circular, oscillatory, and complex trajectories. Their specific performances are depicted in Figure 11.

Figure 12 shows the optimization results for the total communication rate of cluster drones corresponding to the above trajectories. From the graph, it is evident that the circular trajectory has the highest total communication rate, while the oscillatory trajectory has the lowest. The total communication rate for the circular trajectory is approximately

3.3 \times 10^{7} bps

higher than that for the complex trajectory, and the complex trajectory’s rate is about

1 \times 10^{9} bps

higher than that for the oscillatory trajectory. The total communication rates for all trajectories have been effectively jointly optimized and, ultimately, maintained stability.

To verify the optimization effectiveness of the joint optimization proposal presented in this paper, a circular trajectory is used for the motion of the cluster drones, and it is compared with three other optimization schemes. The first scheme optimizes the relay drone’s trajectory, transmission power, and power allocation of cluster drones. The second scheme focuses on optimizing the relay drone’s trajectory, modulation, and power allocation for cluster drones, while the third optimizes the relay drone’s modulation, transmission power, and power allocation for cluster drones. According to the results depicted in Figure 13, the joint optimization scheme outperformed the others, improving the communication rate by

1.2932 \times 10^{11} bps

, which represents 47.68% of the total optimized communication rate. Moreover, the system achieves stability after the fourth round of optimization iterations, showing good convergence.

Communication delay is primarily composed of propagation delay and transmission delay. Propagation delay is calculated on the basis of the transmission distance and propagation speed, whereas the transmission delay is related to the amount of data to be transmitted and the communication rate. In this study, since all optimization schemes are conducted under the same task context, the propagation delay is identical for different schemes. Therefore, this paper mainly considers the significant impact of transmission delay on the total communication delay. Assuming the total amount of data to be transmitted is 50 Gbit, Figure 14 compares the communication delays of the different schemes. As shown in the figure, the proposed scheme exhibited excellent delay performance during both the initial optimization phase and the stable optimization phase, reducing communication delay by 47.68%, 49.04%, and 49.78% compared to other schemes.

Figure 15 illustrates the changes in the modulation order of the relay drone during the iterative process of joint optimization. As seen in the graph, with the progression of the task cycle and while meeting the bit error rate requirements, the choice of relay drone modulation order shifts from 16 to 4, then gradually changes from 64 to 16, and, finally, stabilizes with 64 occurring 88% of the time and 16 occurring 12% of the time.

Figure 16 shows the changes in the transmission power of the relay drone over iterations. Since the trajectory of the cluster drones varies during the task cycle, the transmission power of the relay drone needs to compensate for the signal-to-noise ratio differences caused by the spatial displacement between the two. As the trajectory of the cluster drones follows a circular pattern, the transmission power level exhibits an oscillatory behavior, aligning with the changes in the scenario.

Figure 17 illustrates the energy consumption variations in the relay drone during the mission execution period. As shown in the figure, with the continuous advancement of the mission, the drone needs to adjust physical variables such as trajectory and transmission power to adapt to the highest signal-to-noise ratio for information transmission. Consequently, throughout the mission period, there is a linear increase in flight energy consumption, total energy consumption per iteration, and total energy consumption when the relay drone operates at full power.

Figure 18 shows the energy consumption of the relay drone during the mission. As seen in the figure, flight energy consumption constitutes the major portion of the total energy consumption. After optimization, the total energy consumption of the relay drone at a stable transmission power is 0.61% lower compared to operating at full power. Additionally, the total energy consumption required to complete the mission accounts for 70.89% of the system’s stored energy.

Figure 19 shows the changes in the three-dimensional trajectory of the relay drone. Initially, the relay drone’s trajectory is a straight line. As the number of iterations increases, the relay drone adjusts its path to balance the maximization of the signal-to-noise ratio across two-hop links, curving from the starting point to the termination point. The trajectory is smooth, without any interruptions, facilitating a stable communication link between the two hops.

For the power allocation issue of cluster drones, given the significant spacing between the drone formations, substantial channel variations can occur among individual drones. Additionally, each subcarrier transmits data from two different drones. The relay drone can effectively distinguish the power levels of different cluster drones during serial interference cancellation. Therefore, throughout the entire mission cycle, the cluster drones can operate at full power for transmitting information.

Figure 20 illustrates the impact of variables across different dimensions on the total communication rate. The graph shows that individually optimizing each variable can effectively enhance the objective of increasing the total communication rate. Compared to the unoptimized initial values, the degrees of improvement are substantial, with the respective increases reaching, 8.0654 × 10¹⁰ bps,

3 . 7722 \times 10^{10} bps

3 . 2535 \times 10^{10} bps

, and

1 . 1919 \times 10^{11} bps

Based on the results of the two sets of simulation experiments, the joint optimization method for cluster drone relay communication rates under interference conditions proposed in this paper demonstrates good performance. It effectively avoids external interference while significantly enhancing the total communication rate of the cluster drones.

7. Conclusions

To address the optimization problem of cluster drone relay communication rates under interference conditions, this paper introduced a dual-step algorithmic architecture. By integrating a Bayesian optimization-based Discrete Soft Actor-Critic (BO_DSAC) algorithm, the relay drones were trained for efficient and precise channel selection, effectively mitigating the impact of malicious interference on the communication link. The performance of the algorithm was further enhanced through the optimization of the hyperparameters. Subsequently, joint optimization of the relay drone’s modulation order, transmission power, trajectory, and the cluster drone’s power allocation factor was performed. Complex problems were decomposed into subproblems and transformed into convex problems for solving, thus reducing the complexity of the algorithm and effectively increasing the overall communication rate of the cluster drones. The experimental results demonstrate that the proposed algorithm excels in terms of convergence speed, task effectiveness, and solution stability. Compared to other solutions, the proposed algorithm improved the total reward value by 3.3947 times in anti-interference and increased the total communication rate by

1.2932 \times 10^{11} bps

in the joint optimization scheme. Therefore, the proposed algorithm can flexibly handle constant frequency, sweep frequency, and hybrid frequency interference while effectively enhancing the information backhaul efficiency of cluster drones, increasing task throughput, reducing information backhaul latency, and improving task performance. Future research will explore the impact of highly dynamic channel state information on channel selection algorithms to enhance the stability and efficiency of algorithms in broader applications. Considering the potential impact of efficient energy use for drones, further studies will also focus on the organic integration of wireless energy harvesting technologies with drone communication to address the energy limitations of drones. Additionally, this study primarily concentrated on optimization strategies for relay communication in drone clusters under interference. Future research can be extended to investigate information backhaul methods for ultra-large-scale drone clusters in complex electromagnetic environments. This will reveal more details about drone cluster communication and yield more valuable conclusions.

Author Contributions

Conceptualization, G.C.; Methodology, X.G.; Validation, X.G.; Formal analysis, G.W.; Investigation, G.W.; Data curation, X.G.; Writing—original draft, X.G.; Visualization, C.W.; Project administration, G.C.; Funding acquisition, G.C. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by the National Natural Science Foundation of China (61540022).

Data Availability Statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Conflicts of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Esubonteng, P.K.; Rojas-Cessa, R. RESTORE: Low-Energy Drone-Assisted NLoS-FSO Emergency Communications. IEEE Access 2022, 10, 115282–115294. [Google Scholar] [CrossRef]
Shi, Y.; Zhang, J.; Gao, Y.; Xia, Y. Inter-Server Computation Offloading and Resource Allocation in Multi-Drone Aided Space-Air-Ground Integrated IoT Networks. J. Commun. Netw. 2022, 24, 324–335. [Google Scholar] [CrossRef]
Zhang, Z.; Wu, D.; Zhang, F.; Wang, R. DECCo-A Dynamic Task Scheduling Framework for Heterogeneous Drone Edge Cluster. Drones 2023, 7, 513. [Google Scholar] [CrossRef]
Goel, N.; Gupta, V. Drone-Assisted-Cooperation for Multi-Cluster Disaster Scenario in Next Generation Wireless Communication. Wirel. Pers. Commun. 2024, 135, 1679–1699. [Google Scholar] [CrossRef]
Fan, Q.; Ansari, N. Towards Traffic Load Balancing in Drone-Assisted Communications for IoT. IEEE Internet Things J. 2019, 6, 3633–3640. [Google Scholar] [CrossRef]
Arribas, E.; Mancuso, V.; Cholvi, V. Coverage Optimization with a Dynamic Network of Drone Relays. IEEE Trans. Mob. Comput. 2020, 19, 2278–2298. [Google Scholar] [CrossRef]
Arafat, M.Y.; Moh, S. A Survey on Cluster-Based Routing Protocols for Unmanned Aerial Vehicle Networks. IEEE Access 2019, 7, 498–516. [Google Scholar] [CrossRef]
Hassan, M.Z.; Kaddoum, G.; Akhrif, O. Interference Management in Cellular-Connected Internet of Drones Networks With Drone-Pairing and Uplink Rate-Splitting Multiple Access. IEEE Internet Things J. 2022, 9, 16060–16079. [Google Scholar] [CrossRef]
Li, D.; Zhou, J.; Huang, J.; Zhang, D.; Li, P.; Law, R.; Wu, E.Q. State Prediction and Anti-Interference-Based Flight Path-Following for UAVs. IEEE Trans. Intell. Transport. Syst. 2023, 24, 15236–15247. [Google Scholar] [CrossRef]
Yan, R.; Zhu, X.; Zhu, X.; Peng, R. Joint Optimisation of Task Abortions and Routes of Truck-and-Drone Systems under Random Attacks. Reliab. Eng. Syst. Saf. 2023, 235, 109249. [Google Scholar] [CrossRef]
Song, B.; Xu, H.; Jiang, L.; Rao, N. A Deep Reinforcement Learning-based Intelligent Decision-Making Approach for Communication Anti-Jamming. J. Northwest. Polytech. Univ. 2021, 39, 641–649. [Google Scholar] [CrossRef]
Iacovelli, G.; Grieco, L. Drone Swarm as Mobile Relaying System: A Hybrid Optimization Approach. IEEE Trans. Veh. Technol. 2021, 70, 12272–12277. [Google Scholar] [CrossRef]
He, Y.; Huang, F.; Wang, D.; Zhang, R.; Gu, X.; Pan, J. NOMA- and MRC-Enabled Framework in Drone-Relayed Vehicular Networks: Height/Trajectory Optimization and Performance Analysis. IEEE Internet Things J. 2023, 10, 22305–22319. [Google Scholar] [CrossRef]
Zhang, X.; Xiang, X.; Lu, S.; Zhou, Y.; Sun, S. Evolutionary Optimization of Drone-Swarm Deployment for Wireless Coverage. Drones 2022, 7, 8. [Google Scholar] [CrossRef]
Arribas, E.; Mancuso, V.; Cholvi, V. Optimizing Fairness in Cellular Networks with Mobile Drone Relays. Comput. Netw. 2023, 224, 109623. [Google Scholar] [CrossRef]
Wang, H.; Ren, G.; Chen, J.; Ding, G.; Yang, Y. Unmanned Aerial Vehicle-Aided Communications: Joint Transmit Power and Trajectory Optimization. IEEE Wirel. Commun. Lett. 2018, 7, 522–525. [Google Scholar] [CrossRef]
Zeng, C.; Wang, J.-B.; Ding, C.; Zhang, H.; Lin, M.; Cheng, J. Joint Optimization of Trajectory and Communication Resource Allocation for Unmanned Surface Vehicle Enabled Maritime Wireless Networks. IEEE Trans. Commun. 2021, 69, 8100–8115. [Google Scholar] [CrossRef]
Dong, Z.; Liu, C.; Huang, L. A dynamic multi-UAV communication resource allocation method for area coverage. J. Beijing Univ. Aeronaut. Astronaut. 2022, 1–17. [Google Scholar] [CrossRef]
Ge, L.; Guo, Y.; Zhang, Y.; Chen, G.; Wang, J.; Dai, B.; Li, M.; Jiang, T. Deep Neural Network Based Channel Estimation for Massive MIMO-OFDM Systems with Imperfect Channel State Information. IEEE Syst. J. 2022, 16, 4675–4685. [Google Scholar] [CrossRef]
Yu, X.; Yang, C.; Leung, S.; Rui, Y.; Dang, X.; Guizani, M.; Bai, J. Discrete-Rate Adaptive Modulation Schemes for Uplink Massive MIMO Systems with Imperfect CSI. IEEE Trans. Veh. Technol. 2024, 73, 5426–5442. [Google Scholar] [CrossRef]
González-Coma, J.P.; López-Martínez, F.J.; Castedo, L. Joint User Scheduling and Precoding for XL-MIMO Systems with Imperfect CSI. IEEE Wirel. Commun. Lett. 2023, 12, 1657–1661. [Google Scholar] [CrossRef]
Ju, J.; Sun, Q. Performance Analysis of Partial NOMA-Based Layered D2D Communications. J Wirel. Commun. Netw. 2024, 2024, 45. [Google Scholar] [CrossRef]
Liu, C.; Zhang, Y.; Niu, G.; Jia, L.; Xiao, L.; Luan, J. Towards Reinforcement Learning in UAV Relay for Anti-Jamming Maritime Communications. Digit. Commun. Netw. 2023, 9, 1477–1485. [Google Scholar] [CrossRef]
Zeng, Y.; Zhang, R.; Lim, T.J. Throughput Maximization for Mobile Relaying Systems 2016. In Proceedings of the 2016 IEEE Globecom Workshops, Washington, DC, USA, 4–8 December 2016. [Google Scholar]
Wong, C.Y.; Cheng, R.S.; Lataief, K.B.; Murch, R.D. Multiuser OFDM with Adaptive Subcarrier, Bit, and Power Allocation. IEEE J. Select. Areas Commun. 1999, 17, 1747–1758. [Google Scholar] [CrossRef]
Christodoulou, P. Soft Actor-Critic for Discrete Action Settings. arXiv 2019, arXiv:1910.07207. [Google Scholar]
Hebbal, A.; Balesdent, M.; Brevault, L.; Melab, N.; Talbi, E.-G. Deep Gaussian Process for Multi-Objective Bayesian Optimization. Optim. Eng. 2023, 24, 1809–1848. [Google Scholar] [CrossRef]
Na, Z.; Ji, C.; Lin, B.; Zhang, N. Joint Optimization of Trajectory and Resource Allocation in Secure UAV Relaying Communications for Internet of Things. IEEE Internet Things J. 2022, 9, 16284–16296. [Google Scholar] [CrossRef]
Shen, K.; Yu, W. Fractional Programming for Communication Systems—Part I: Power Control and Beamforming. IEEE Trans. Signal Process. 2018, 66, 2616–2630. [Google Scholar] [CrossRef]
Ji, W.; Yang, X.; Li, F.; Li, D.; Liang, Y.; Song, Y. Energy consumption optimization based on optimal SIC order in UAV-assisted MEC systems. J. Commun. 2024, 45, 18–30. [Google Scholar]

Figure 1. Schematic diagram of swarm drone relay communication under interference conditions.

Figure 2. Schematic diagram of the dynamic interference process in the channel.

Figure 3. Schematic diagram of the Discrete Soft Actor-Critic algorithm architecture.

Figure 4. Comparison of resistances to Constant frequency interference effects.

Figure 5. Comparison of resistance to sweep frequency interference effects.

Figure 6. Comparison of resistance to hybrid frequency interference effects.

Figure 7. Comparison of anti-interference effects with different hyperparameters.

Figure 8. Action selection of the BO_DSAC algorithm under different interference modes.

Figure 9. Action selection of the Conv_DQN algorithm under different interference modes.

Figure 10. Action selection of the DQN algorithm under different interference modes.

Figure 11. Different motion trajectories of the cluster drones.

Figure 12. Joint optimization for different trajectories.

Figure 13. Comparison of total communication rate values among different optimization schemes.

Figure 14. Communication delay under different optimization schemes.

Figure 15. Changes in the modulation order of the relay drone.

Figure 16. Variation in the transmission power of the relay drone.

Figure 17. Energy consumption variation in the relay drone during the mission period.

Figure 18. Mission energy consumption value of the relay drone.

Figure 19. Changes in the three-dimensional trajectory of the relay drone.

Figure 20. Impact of variables across different dimensions on the optimization of total communication rate.

Table 1. Description of key parameters.

Key Parameter	Description
$F (k)$	Trajectory Coordinate Set of the Drone Cluster
$H_{i}^{w} (k)$	Channel Gain between Drone $i$ in the Cluster and the Relay Drone
$Λ$	Channel Attenuation between the Jammer and the Relay Drone
$D_{n}$	Normal Communication Threshold of the Relay Drone
$E^{s}$	Total Energy Consumption of the System
$F_{f}^{R u a v}$	Operating Frequency Set of the Relay Drone
$B_{n}^{R u a v}$	Subchannel Bandwidth
$P^{F}$	Maximum Transmission Power of the Drone Cluster
$P^{J}$	Maximum Transmission Power of the Jammer
$Υ (\cdot)$	Channel Interference Indicator Function
$S I N R_{i, R}$	SINR of the Signal Received by the Relay Drone from Drone $i$ in the Cluster
$S N R_{R, S}^{R u a v}$	SINR of the Signal Received by the Ground Station from the Relay Drone
$S N R_{i, S}$	SNR of the Two-Hop Link from Drone $i$ in the Cluster Received by the Ground Station
$R_{i}$	Communication Rate between Drone $i$ in the Cluster and the Ground Station
$R_{s}^{c}$	Total Communication Rate between the Drone Cluster and the Ground Station
$V_{m a x}^{h o r}$	Maximum Horizontal Speed
$V_{m a x}^{v e r}$	Maximum Vertical Speed
$R_{l}$	Threshold for the Total Communication Rate
$B E R_{{i}}$	Bit Error Rate Corresponding to the i-th Order Modulation Scheme between the Drone Cluster and the Ground Station
$Q^{s o f t}$	Soft Predictive Value Network
$L_{Q} (ω^{u})$	Loss Function of the Value Network
$L_{π} (θ^{u})$	Loss Function of the Policy Network
$L (α)$	Loss Function of the Entropy Coefficient
$π (\cdot)$	Policy Network
$Q (\cdot)$	Value Network
$S^{i}$	State Space
$A^{i}$	Action Space
$α$	Entropy Coefficient
$R^{i}$	Real-Time Reward Function
$Δ r$	Reward Metric for Channel Switching
$γ^{u}$	Discount Factor
$l r^{u}$	Learning Rate
$N^{n}$	Number of Network Layers
$b a t c h_s i z e$	Batch Size of Data Sampled from the Experience Pool
$τ^{l s}$	Soft Update Parameter
$M D P L$	Markov Chain Length
$n u m_e p i s o d e s$	Maximum Number of Training Iterations
$λ^{(c - 1)} (k)$	Power Allocation Factor of the Drone Cluster in the (c-1)-th Round
$P^{R, (c - 1)} (k)$	Transmission Power of the Relay Drone in the (c-1)-th Round
$Q^{(c - 1)} (k)$	Trajectory Coordinates of the Relay Drone in the (c-1)-th Round
$U^{(c)} (k)$	Modulation Order Set of the Relay Drone in the c-th Round
$O_{{X, Y, Z}}^{c}$	First-Order Partial Derivative Value with Respect to $Q (k)$
$ℵ^{(c)}$	Solution Set in the c-th Round

Table 2. Experimental parameters.

Parameter	Value	Parameter	Value
$T$	$100$	$M$	$16$
$(x^{J}, y^{J}, 0)$	$(1500, 1500, 0)$	$G_{t} G_{r}$	$20$
$F_{f}^{R u a v}$	$[1.95 GHz, 2.05 GHz]$	$B_{n}^{R u a v}$	$10 Mbps$
$P^{F}$	$30 dBm$	$H_{1}^{*}$	$13 dB$
$H_{l}$	$30 m$	$H_{h}$	$100 m$
$P_{m a x}^{R}$	$40 dBm$	$E_{m a x}^{R u a v} = E_{m a x}^{F u a v}$	$5 \times 10^{4} J$
$V_{m a x}^{v e r}$	$1 m / s$	$R_{l}$	$40 Mbps$
$n_{0}$	$- 150 dBm / Hz$	$P^{J}$	$8000 W$
$P^{L} = P^{L}$	$0.5$	$V_{m a x}^{h o r}$	$12 m / s$
$F$	$10$	$B E R_{q}$	$10^{- 6}$
$D_{n}$	$- 90 dBm$	$m g$	$19.6 N$

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Gu, X.; Chen, G.; Wu, G.; Wen, C. Joint Optimization of Relay Communication Rates in Clustered Drones under Interference Conditions. Drones 2024, 8, 381. https://doi.org/10.3390/drones8080381

AMA Style

Gu X, Chen G, Wu G, Wen C. Joint Optimization of Relay Communication Rates in Clustered Drones under Interference Conditions. Drones. 2024; 8(8):381. https://doi.org/10.3390/drones8080381

Chicago/Turabian Style

Gu, Xinglong, Guifen Chen, Guowei Wu, and Chenghua Wen. 2024. "Joint Optimization of Relay Communication Rates in Clustered Drones under Interference Conditions" Drones 8, no. 8: 381. https://doi.org/10.3390/drones8080381

Article Menu

Joint Optimization of Relay Communication Rates in Clustered Drones under Interference Conditions

Abstract

1. Introduction

2. System Model

2.1. Task Model

2.2. Channel Model

2.3. Interference Model

2.4. Power Transmission Model and Energy Model

3. Problem Description

4. Intelligent Channel Selection Scheme

4.1. Discrete Soft Actor-Critic Algorithm

4.2. Bayesian Optimization Algorithm

5. Joint Optimization Scheme for Communication Rate

5.1. Relay Drone Modulation Method Optimization

5.2. Optimization of Relay Drone Transmission Power

5.3. Optimization of Three-Dimensional Trajectory for Relay Drones

5.4. Optimization of Power Allocation Factor for Cluster Drones

6. Experimental Simulation

6.1. Verification of the BO_DSAC Algorithm’s Effectiveness

6.2. Verification of the Joint Optimization Algorithm Effectiveness for Cluster Drone Communication Rates

7. Conclusions

Author Contributions

Funding

Data Availability Statement

Conflicts of Interest

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI