CN116017717A - Hybrid intelligent spectrum management method in unmanned system cluster network - Google Patents

Abstract

The invention discloses a hybrid intelligent spectrum management method in an unmanned system cluster network, which constructs a weighted interference graph model, and can automatically optimize and adjust a working channel of the network by adopting a reinforcement learning method based on measurement information and communication performance feedback of nodes, thereby reducing adverse effects of frequency conflict in the system and interference outside the system. The method takes the communication performance actually perceived by the communication node as feedback to guide the spectrum decision, can adapt to the complex dynamic spectrum environment and the wireless channel environment, and can support the large-scale unmanned system networking communication spectrum management. The hybrid optimization structure combining the centralized optimization (business data network channel selection) and the distributed optimization (wireless access network channel selection based on reinforcement learning) is provided, and has learning capability and lower algorithm complexity.

Description

A hybrid intelligent spectrum management method in unmanned system cluster network

Technical Field

The present invention relates to the technical field of unmanned system cluster communication, and more specifically, to a hybrid intelligent spectrum management method in an unmanned system cluster network.

Background Art

A large number of unmanned systems will perform various tasks by forming clusters, which will become an important working mode in certain specific fields in the future. In order to successfully complete communication tasks in certain specific fields, unmanned systems need to make reasonable use of wireless spectrum. Their use of spectrum mainly includes three types of frequency-using equipment: First, communication equipment, which supports the transmission of control commands from the control station to the unmanned system, the real-time return of high-speed data reconnaissance information of the unmanned system, and the real-time command and control of the unmanned system. It has the largest spectrum demand and is the key to successfully completing the task. Second, sensor equipment, which provides communication and data transmission support for navigation positioning, target indication, and friend-or-foe identification, mainly including satellite navigation, synthetic aperture radar, mobile target indicator, friend-or-foe identification, and remote sensing detectors. Third, vehicle/aircraft/shipborne electronic countermeasure equipment, which includes jamming equipment and passive jamming equipment to suppress and interfere with enemy electronic systems or conduct electronic deception. On the other hand, unmanned system clusters generally work in harsh environments, with complex terrain and wireless channel characteristics and the threat of enemy interference signals, which makes the use of frequency by unmanned system clusters face many challenges.

Coordinating frequency conflicts within unmanned system clusters and adapting to complex external spectrum environments requires efficient and reliable spectrum management. However, existing spectrum management methods for unmanned systems are mainly based on simple fixed allocation, which is difficult to adapt to dynamic and complex environments and large-scale cluster communication networks. Some literature has proposed a spectrum allocation method for unmanned aerial vehicles with joint task allocation, which regards unmanned aerial vehicles that complete the same task as the same alliance. UAVs within the alliance share the same channel, while different channels are used between task alliances. In order to improve the utilization of spectrum, some literature has also shared some overlapping channels in cluster-based unmanned aerial vehicle networks. In order to overcome the influence of uncertain state information caused by channels and unmanned aerial vehicle movement, a distributed channel distribution algorithm is designed using fuzzy decision-making and game learning algorithms. Related literature proposes to use millimeter wave frequency bands in unmanned aerial vehicle-based cellular networks. In order to alleviate the interference between base stations and between base stations and wireless backhaul links, a 3D interference graph model is constructed. The spectrum allocation is modeled as a continuous time period optimization problem combined with constraints such as unmanned aerial vehicle mobility, energy consumption and interference. The above-mentioned methods provide references for spectrum management of unmanned system clusters, but they are all based on theoretical analysis, lack actual protocols and network verification, and are difficult to adapt to the application scenarios of large-scale cluster networks.

Summary of the invention

In response to at least one defect or improvement need in the prior art mentioned in the background technology section, the present invention provides a hybrid intelligent spectrum management method in an unmanned system cluster network, so as to overcome the technical defects of the prior art that lacks actual protocols and network verification and is difficult to adapt to application scenarios of large-scale cluster communication networks.

To achieve the above object, the present invention provides a hybrid intelligent spectrum management method in an unmanned system cluster network, comprising:

S1. Constructing an external weighted interference relationship matrix to measure the number of non-system WiFi networks on each channel; the external weighted interference relationship matrix includes a matrix for a service data network and a matrix for a wireless access network;

S2, selecting a channel that minimizes the actual weighted interference received by the service data network on each channel as the channel of the service data network;

S3, initializing channel selection of the wireless access network;

S4, constructing a weighted interference relationship matrix within the system to describe the potential interference relationship between wireless switching nodes within the system;

S5. In the current time slot, the average link quality is updated according to the received WiFi signals of each channel detected by the terminal;

S6. Update the interference distribution vector and use it to obtain the actual weighted interference of the wireless access network, and update the average link quality estimation value of the wireless access network based on the actual weighted interference of the wireless access network and the average link quality;

S7. At the beginning of the next time slot, updating the channel selection of the wireless access network according to the probability rule;

S8. If the channel of the service data network needs to be readjusted or the network status changes, jump to step S1; otherwise, jump to step S4.

Further, the formula for selecting the channel that minimizes the actual weighted interference received by the service data network on each channel as the channel of the service data network includes:

Wherein, I ₀ (c) represents the actual weighted interference received by the service data network on each channel, a ₀ represents the channel of the service data network, un _,c represents the number of WiFi networks outside the system on channel c detected by the wireless switching node n itself, δ(a ₁ ,a ₂ ) is a channel interference function, n represents the number of the wireless switching node, and c represents the number of the wireless communication channel available in the system.

Further, the initializing the channel selection of the wireless access network specifically includes:

All wireless switching nodes randomly select channels from channels outside the protection channel set of the service data network.

Furthermore, the specific definition of the protection channel set of the service data network is:

表示业务数据网络使用的信道。Wherein, C ₀ represents the protection channel set of the service data network, and the wireless communication channel set available to the system is

Indicates the channel used by the business data network.

Further, the average link quality is obtained by calculating by defining the link signal quality;

For each single-hop wireless communication link, the link signal quality is defined as quality*(A+signal), where quality and signal represent the current communication link parameters detected by WiFi respectively, quality is a score not exceeding 1, and the larger the score, the better the signal; signal represents the signal strength in dBm, which is usually negative, and the preset positive number A is added to ensure that (A+signal) is a positive value; each wireless switching node calculates the average link quality of the link with the connected terminal, that is, the average link quality of the wireless access network link.

Furthermore, the formula for updating the interference distribution vector and obtaining the actual weighted interference of the wireless access network specifically includes:

Wherein, _Ln = { _ln (c)} is the interference distribution vector of wireless access network n, _In is the actual weighted interference of the wireless access network, _an is the channel used by the wireless access network of wireless switching node n, and δ( _a1 , _a2 ) is the channel interference function.

Furthermore, the expression of the channel interference function δ(a ₁ , a ₂ ) is:

Further, the formula for updating the average link quality estimation value of the wireless access network based on the actual weighted interference and the average link quality of the wireless access network specifically includes:

表示所述平均链路质量，e_n(a_n)表示所述无线接入网络的平均链路质量估计值，T(a_n)表示所选信道变量a_n被无线接入网络n选中的总次数。in,

represents the average link quality, _en (a _n ) represents the average link quality estimation value of the wireless access network, and T (a _n ) represents the total number of times the selected channel variable a _n is selected by the wireless access network n.

Furthermore, the formula of the probability rule specifically includes:

a _n is

Wherein, _En ={ _en (c)} is the average link quality estimation value when the wireless switching node n uses the channel c, and ε is a preset probability value that decreases as the number of time slots increases.

Furthermore, the formula for the preset probability value specifically includes:

Among them, B is (0,1), and t is the current number of time slots.

In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

The present invention provides a hybrid intelligent spectrum management method in an unmanned system cluster network, which uses the communication performance actually perceived by the communication node as feedback to guide spectrum decision-making, can adapt to complex and dynamic spectrum environments and wireless channel environments, and can support large-scale unmanned system networking communication spectrum management. It proposes a hybrid optimization structure that combines centralized optimization (service data network channel selection) with distributed optimization (radio access network channel selection based on reinforcement learning), has learning capabilities and low algorithm complexity.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a flow chart of a hybrid intelligent spectrum management method in an unmanned system cluster network provided by an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

The terms "including" or "having" and any variations thereof in the specification, claims or drawings of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

Aiming at the future large-scale unmanned system cluster application scenario, the present invention proposes a hybrid spectrum management solution based on the layered wireless mesh network architecture. It constructs a weighted interference graph model, based on the measurement information and communication performance feedback of the nodes, and adopts a reinforcement learning method, which can autonomously optimize and adjust the working channel of the network, reduce the frequency conflicts within the system and the adverse effects of interference outside the system. Through the hybrid intelligent spectrum management method of the present invention, the unmanned system cluster can autonomously optimize the working channel and enhance the network's ability to adapt to complex confrontation environments.

Some prerequisites for the method of the present invention to work:

(1) Hierarchical wireless mesh network structure. To support the communication needs of large-scale unmanned system clusters, a two-layer network is used to organize various communications of the unmanned system cluster. Some unmanned system nodes are selected as wireless mesh nodes (called wireless switching nodes). The wireless switching nodes work in the same frequency band and can build a wireless backbone network through multi-hop. This layer of network is called the business data network. The wireless switching nodes each provide a wireless local area network. Other unmanned systems in the cluster (called wireless terminal nodes) select the wireless local area network with the strongest signal to access the network and realize wireless networking communication. This layer of network is called the wireless access network.

(2) Random access mode: Both the service data network and the wireless access network adopt the IEEE802.11 wireless LAN mode based on random multiple access.

(3) Wireless control link: A physically independent control layer network is built between wireless switching nodes, which has low-speed reliable transmission capabilities, collects and processes wireless communication status information, and sends centralized spectrum management results.

(4) Controller: The entity that performs centralized spectrum management can be a wireless switching node or a ground control station, which manages the unmanned system cluster communication spectrum through a wireless control link.

Definitions of some terms and variables:

N个无线交换节点集合

系统可用的无线通信信道集合

(1) Basic variables. The set of M terminal nodes is

N wireless switching nodes

The set of wireless communication channels available to the system

表示业务数据网络使用的信道，定义业务数据网络的保护信道集为

无线交换节点n的无线接入网络使用的信道记为a_n，为减小无线接入网络和业务数据网络间的干扰，无线交换节点的信道选择在业务数据网络信道确定之后且有保护，即只能从保护信道集中随机选择信道

也就是保证足够的信道间隔。记无线交换节点所选信道集合为向量a＝(a₁，...，a_N)。(2) Channel selection variables.

Indicates the channels used by the business data network, and defines the protection channel set of the business data network as

The channel used by the wireless access network of wireless switching node n is denoted as a _n . To reduce the interference between the wireless access network and the service data network, the channel selection of the wireless switching node is after the service data network channel is determined and protected, that is, the channel can only be randomly selected from the protection channel set.

That is to ensure sufficient channel spacing. Let the channel set selected by the wireless switching node be vector a=(a ₁ , ..., a _N ).

定义并初始化规模为

的全零向量E_n＝{e_n(c)}为无线交换节点n使用信道c时的平均链路质量估计值，注意，由于这里不选择业务数据网络的保护信道集，所以需要在C个信道中减去

其中每个元素e_n，c表示无线交换节点n采用信道c时的平均链路质量的估计值，它主要用于指导业务数据网络信道更新。两者的关系见下面的式(5)。(3) Estimated variables related to link quality. For each single-hop wireless communication link, the link signal quality is defined as quality*(A+signal). When A is 200, it is quality*(200+signal), where quality and signal represent the current communication link parameters detected by WiFi. Quality is a score not exceeding 1, and the larger the score, the better the signal. Signal represents the signal strength in dBm, which is usually negative. The 200 added here ensures that (200+signal) is a positive value. Each wireless switching node calculates the average link quality of its link with the connected terminal, that is, the average link quality of the wireless access network link.

Define and initialize the scale as

The all-zero vector _En = { _en (c)} is the average link quality estimation value when the wireless switching node n uses channel c. Note that since the protection channel set of the service data network is not selected here, it is necessary to subtract

Each element _en,c represents the estimated value of the average link quality when the wireless switching node n adopts the channel c, which is mainly used to guide the update of the service data network channel. The relationship between the two is shown in the following formula (5).

的全零向量L_n＝{l_n(c)}为无线接入网络n的干扰分布向量，用于表示终端检测到的各信道上存在的系统内以及系统外的WiFi网络数量。定义并初始化规模为

的全零向量I_n为加权干扰，I_n与L_n的关系见下面的式(3)。(4) Interference distribution vector. Define and initialize the scale as

The all-zero vector L _n ={l _n (c)} is the interference distribution vector of the wireless access network n, which is used to represent the number of WiFi networks inside and outside the system on each channel detected by the terminal. Define and initialize the scale as

The all-zero vector _In is weighted interference, and the relationship between _In and _Ln is shown in the following formula (3).

的全零向量T_n＝(0，...，0)，记录截至当前时隙，该节点无线接入网络选择各信道的累计总次数。(5) Channel selection history variables. Each wireless switching node defines and initializes the scale as

The all-zero vector T _n =(0, ..., 0) records the total number of times the node selects each channel in the wireless access network up to the current time slot.

The present invention takes time T as a cycle, and all wireless terminal nodes update local status information. At the same time, the controller updates all kinds of information collected in the entire network, including topology information, node information, wireless link information and service information, and maintains the information storage of the latest K cycles. Based on the detected wireless link information, a reinforcement learning mechanism is used to periodically adjust the working channels of the service data network and the wireless access network. The specific wireless link information includes: the ID, channel, power, quality, signal strength, bandwidth and delay of the wireless access network currently accessed by the terminal; the access network of other switching node networks detected by the terminal and other identifiable network-related information in the surrounding environment, such as ID, channel, power, quality and signal strength.

As shown in FIG1 , in one embodiment, a hybrid intelligent spectrum management method in an unmanned system cluster network is provided, and the method mainly includes the following steps:

Step 1: Construct an out-of-system weighted interference relationship matrix to measure the number of non-system WiFi networks on each channel. There are two out-of-system weighted interference matrices: one is U = {u _{n, c} } for the service data network, with a dimension of N*C, and its element u _{n, c} represents the number of out-of-system (non-wireless access network) WiFi networks on channel c detected by the wireless switching node n itself; the other is V = {v _{n, c} } for the wireless access network, with a dimension of N*C, and its element v _{n, c} represents the number of out-of-system WiFi networks (non-wireless access networks and service data networks) on each channel detected by the terminal to which the wireless switching node belongs.

定义为Step 2: Update the channel selection of the service data network. The controller updates the actual weighted interference received by the service data network on each channel.

Defined as

The controller selects the channel that minimizes I ₀ (c) as the channel of the service data network, that is, the channel with the least number of non-system WiFi networks detected, that is,

Step 3: Initialize the channel selection of the wireless access network. All wireless switching nodes randomly select channels from channels outside the protection channel set of the service data network.

其元素w_n1→n2表示接入无线交换节点n₂的终端中能够接收到无线交换节点n₁的信号的终端的数量，也即当n₂与n₁同信道工作时，n₂中受到n₁信号干扰的终端的数量。注意，由于不同节点的功率可能不同，该矩阵不一定是对称矩阵。Step 4: Construct a potential intra-system weighted interference relationship matrix to describe the potential interference relationship between wireless switching nodes in the system. Each terminal node can detect the access network signals of other wireless switching nodes working on each channel. These access network signals can be received and processed. If they use the same channel, they will compete with each other, which is a measure of intra-system network interference. The spectrum management controller constructs a system weighted interference relationship matrix with a dimension of N*N based on the collected status information.

Its element w _n1→n2 represents the number of terminals that can receive the signal of wireless switching node n ₁ among the terminals accessing wireless switching node n ₂ , that is, the number of terminals in n ₂ that are interfered by the signal of n ₁ when n ₂ and n ₁ work on the same channel. Note that since the power of different nodes may be different, this matrix is not necessarily a symmetric matrix.

Step 5: Wireless link quality measurement. In the current time slot, the wireless switching node updates the average link quality based on the WiFi signals of each channel detected by the terminal.

Step 6: Update the interference state variable. The wireless switching node updates the interference distribution vector L _n and uses it to calculate the actual weighted interference I _n .

Where δ(a ₁ ,a ₂ ) is the channel interference function, defined as

Step 7: Update the channel selection state variable. Update the total number of times the selected channel variable a _n is selected by the wireless access network n T _n (a _n ) = T _n (a _n ) + 1; Update the average link quality estimation value of the wireless access network.

Step 8: Update the wireless access network channel selection. At the beginning of the next time slot, each wireless switching node adjusts the access network channel a _n according to the following rules:

t为当前的时隙数量。Among them, ε can be initially set to a small number in the range of (0,1), such as 0.3, and decreases as the number of time slots increases. For example, it is inversely proportional to the number of time slots.

t is the current time slot number.

Step 9: If the controller needs to readjust the channel of the service data network or the network status changes, return to step 1; otherwise, return to step 4.

The hybrid intelligent spectrum management method of the present invention is mainly used in future unmanned confrontation scenarios, and can provide an effective intelligent spectrum management method for large-scale unmanned system cluster networking communications. According to the tasks and communication capabilities undertaken, drones can be divided into wireless switching nodes, spectrum controllers and terminal nodes according to the above requirements. The wireless switching nodes and spectrum controllers work in coordination according to the method scheme of the present invention to achieve autonomous spectrum allocation and adjustment of the network. The present invention is also applicable to spectrum management in large-scale hierarchical wireless mesh networks.

The present invention discloses a hybrid intelligent spectrum management method in an unmanned system cluster network. A weighted interference graph model is constructed. Based on the measurement information and communication performance feedback of the nodes, a reinforcement learning method is adopted to autonomously optimize and adjust the working channels of the network, reduce the frequency conflicts within the system and the adverse effects of interference outside the system. The method uses the communication performance actually perceived by the communication nodes as feedback to guide spectrum decisions, can adapt to complex and dynamic spectrum environments and wireless channel environments, and can support large-scale unmanned system networking communication spectrum management. It proposes a hybrid optimization structure that combines centralized optimization (service data network channel selection) with distributed optimization (radio access network channel selection based on reinforcement learning), has learning capabilities, and has low algorithm complexity.

The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of other embodiments of the present disclosure. The present invention is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A hybrid intelligent spectrum management method in an unmanned system cluster network, characterized by comprising: S1. Constructing an external weighted interference relationship matrix to measure the number of non-system WiFi networks on each channel; the external weighted interference relationship matrix includes a matrix for a service data network and a matrix for a wireless access network; S2, selecting a channel that minimizes the actual weighted interference received by the service data network on each channel as the channel of the service data network; S3, initializing channel selection of the wireless access network; S4, constructing a weighted interference relationship matrix within the system to describe the potential interference relationship between wireless switching nodes within the system; S5. In the current time slot, the average link quality is updated according to the received WiFi signals of each channel detected by the terminal; S6. Update the interference distribution vector and use it to obtain the actual weighted interference of the wireless access network, and update the average link quality estimation value of the wireless access network based on the actual weighted interference of the wireless access network and the average link quality; S7. At the beginning of the next time slot, updating the channel selection of the wireless access network according to the probability rule; S8. If the channel of the service data network needs to be readjusted or the network status changes, jump to step S1; otherwise, jump to step S4. 2. The hybrid intelligent spectrum management method according to claim 1, wherein the formula for selecting the channel that minimizes the actual weighted interference received by the service data network on each channel as the channel of the service data network comprises:

Wherein, I ₀ (c) represents the actual weighted interference received by the service data network on each channel, a ₀ represents the channel of the service data network, un _,c represents the number of WiFi networks outside the system on channel c detected by the wireless switching node n itself, δ(a ₁ ,a ₂ ) is a channel interference function, n represents the number of the wireless switching node, and c represents the number of the wireless communication channel available in the system. 3. The hybrid intelligent spectrum management method according to claim 1, wherein the initializing the channel selection of the wireless access network specifically comprises: All wireless switching nodes randomly select channels from channels outside the protection channel set of the service data network. 4. The hybrid intelligent spectrum management method according to claim 3, characterized in that the specific definition of the protection channel set of the service data network is:

Wherein, C ₀ represents the protection channel set of the service data network, and the wireless communication channel set available to the system is

Indicates the channel used by the business data network. 5. The hybrid intelligent spectrum management method according to claim 1, wherein the average link quality is obtained by calculating by defining the link signal quality; For each single-hop wireless communication link, the link signal quality is defined as quality*(A+signal), where quality and signal represent the current communication link parameters detected by WiFi respectively, quality is a score not exceeding 1, and the larger the score, the better the signal; signal represents the signal strength in dBm, which is a negative value, and the preset positive number A is added to ensure that (A+signal) is a positive value; each wireless switching node calculates the average link quality of the link with the connected terminal, that is, the average link quality of the wireless access network link. 6. The hybrid intelligent spectrum management method according to claim 1, wherein the formula for updating the interference distribution vector and obtaining the actual weighted interference of the wireless access network specifically comprises:

Wherein, _Ln = { _ln (c)} is the interference distribution vector of wireless access network n, _In is the actual weighted interference of the wireless access network, _an is the channel used by the wireless access network of wireless switching node n, and δ( _a1 , _a2 ) is the channel interference function. 7. The hybrid intelligent spectrum management method according to claim 2 or 6, characterized in that the expression of the channel interference function δ(a ₁ , a ₂ ) is:

8. The hybrid intelligent spectrum management method according to claim 6, wherein the formula for updating the average link quality estimate of the wireless access network based on the actual weighted interference and the average link quality of the wireless access network specifically comprises:

in,

represents the average link quality, _en (a _n ) represents the average link quality estimation value of the wireless access network, and T (a _n ) represents the total number of times the selected channel variable a _n is selected by the wireless access network n. 9. The hybrid intelligent spectrum management method according to claim 1, wherein the formula of the probability rule specifically includes:

Wherein, _En ={ _en (c)} is the average link quality estimation value when the wireless switching node n uses the channel c, and ε is a preset probability value that decreases as the number of time slots increases. 10. The hybrid intelligent spectrum management method according to claim 9, wherein the formula for the preset probability value specifically comprises:

Among them, B is (0,1), and t is the current number of time slots.

Images