CN108307510A - A kind of power distribution method in isomery subzone network - Google Patents

Abstract

The invention provides a power distribution method in a heterogeneous cell network, which can maximize the energy efficiency and throughput of the cell. The method includes: acquiring a delay parameter, and determining the energy efficiency of the small base station according to the acquired delay parameter; describing the power allocation problem as maximizing the energy efficiency of the small base station; determining the transmit power when the power allocation problem reaches Nash equilibrium; Based on the determined transmit power, the energy efficiency in power allocation is regarded as the reward value function in Q-learning, and the optimal power allocation scheme is solved by a guess-based multi-user Q-learning algorithm. The invention is applicable to power distribution operations.

Description

A Power Allocation Method in Heterogeneous Cell Network

technical field

The invention relates to the field of mobile communication, in particular to a power allocation method in a heterogeneous cell network.

Background technique

With the rapid development of science and technology, the advent of the 5G era is not far away. As more and more smart mobile devices are connected to wireless networks, frequency resources are becoming less and less, which has caused a heavy burden on macro cells. The deployment of small base stations can alleviate the overload traffic from macro cells and increase system capacity. However, the same-layer and cross-layer interferences are becoming more and more serious, resulting in the reduction of system capacity and quality of service (QoS). In order to better handle this situation, a more reasonable and efficient power allocation method needs to be proposed. In the OFDMA spectrum sharing network, there have been a lot of researches to mitigate interference and improve energy efficiency. However, in the prior art, there is no research on using delay constraints to improve the quality of service (QoS).

Contents of the invention

The technical problem to be solved by the present invention is to provide a power allocation method in a heterogeneous cell network to solve the problem in the prior art that the delay constraint has not been used to improve the service quality.

In order to solve the above technical problems, an embodiment of the present invention provides a power allocation method in a heterogeneous cell network, including:

Obtaining a time delay parameter, and determining the energy efficiency of the small base station according to the obtained time delay parameter;

Formulate the power allocation problem as maximizing the energy efficiency of small base stations;

Determine the transmit power at which the power allocation problem reaches Nash equilibrium;

Based on the determined transmit power, the energy efficiency in power allocation is regarded as the reward value function in Q-learning, and the optimal power allocation scheme is solved by a guess-based multi-user Q-learning algorithm.

Further, the determining the energy efficiency of the small base station according to the acquired delay parameter includes:

Determine the received SINR according to the acquired time delay parameter;

Determine the total received data rate according to the determined received SINR;

Determine the effective capacity based on the determined total received data rate;

Based on the determined effective capacity, the energy efficiency of the small base station is determined.

Further, the received SINR is expressed as:

in, Indicates the received signal-to-interference-noise ratio of user f on sub-channel n of small base station k; Indicates the channel power gain from small base station k to user f on subchannel n; Indicates the transmit power of small base station k on subchannel n; Indicates the transmit power of other small base stations except small base station k; is the delay parameter, which represents the interference of other small base stations and macro cells except small base station k on subchannel n to user f under the kth small base station; δ ² represents Gaussian white noise.

Further, the total received data rate is expressed as:

in, Indicates the total received data rate of user f on subchannel n, K indicates the number of small base stations, Yes the shorthand form of Indicates the receiving data rate of user f on sub-channel n of small base station k, T _f represents the transmission time, B represents the bandwidth, Yes abbreviated form of .

Further, the effective capacity is expressed as:

in, is the effective capacity, which means that the small base station k can support the maximum data rate of user f; θ _kf represents the service quality index of user f under small base station k; E( ) represents the error function; for abbreviated form of .

Further, the energy efficiency of the small base station is expressed as:

in, Represents the energy efficiency of user f under small base station k, T _f represents the transmission time, P _k represents the total downlink transmission power, and P _c represents the circuit power.

Further, the power allocation problem is formulated as maximizing the energy efficiency of small base stations:

in, Yes the shorthand form of Indicates the preset signal-to-interference and noise ratio threshold, p _mask indicates the preset hidden constraint power, K ^' indicates the set of small base stations, ^F'indicates the set of users contained in the small base station, ^N'indicates the set of subchannels, denote the maximum and minimum transmit power of the small base station k, respectively, defined as

Further, the update rule of Q value in Q learning algorithm for:

in, s' _k and s _k represent the state of small base station k at time slot t and t+1 respectively, α _t represents the learning rate, Represents the reward value function, a _k represents the action set of small base station k, a _-k represents the action set of other small base stations except small base station k, and A _-k represents the actions of all small base stations except small base station k. The action set of the base station, β represents the cost factor in Q learning, β _k represents the action in the action set A _k of the small base station k, Denotes the policy set of small base station j, Indicates the transmit power of small base station k on sub-channel n under action α _k , and I _k represents the state of received SINR of user f in small base station k.

Further, the update rule for the Q value in the Q learning algorithm To update, the update rule of the guess-based Q value is:

in, Represents a guess-based linear function.

Further, the guess-based linear function is expressed as:

in, is a constant parameter, a positive scalar constant value; and Denote conjecture, strategy set and strategy set respectively, and τ represents the heat value.

The beneficial effects of above-mentioned technical scheme of the present invention are as follows:

In the above scheme, the delay parameter is obtained, and the energy efficiency of the small base station is determined according to the obtained delay parameter; the power allocation problem is described as maximizing the energy efficiency of the small base station; the transmission power when the power allocation problem reaches Nash equilibrium is determined; based on With a certain transmit power, the energy efficiency in power allocation is regarded as the reward value function in Q-learning, and the optimal power allocation scheme is solved by a guess-based multi-user Q-learning algorithm. In this way, the quality of service of the cell is guaranteed by introducing a delay parameter (also called: delay constraint), and the power allocation problem is modeled as a non-cooperative supermodel game, and the transmit power when the power allocation problem reaches Nash equilibrium is determined. Based on the determined transmit power, the optimal power allocation scheme is solved by the guess-based multi-user Q learning algorithm. This method can maximize the energy efficiency and throughput of the cell under the premise of ensuring the quality of service and considering energy efficiency. Meet the rapidly increasing access demand.

Description of drawings

FIG. 1 is a schematic flowchart of a power allocation method in a heterogeneous cell network according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a system model provided by an embodiment of the present invention.

Detailed ways

In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will describe in detail with reference to the drawings and specific embodiments.

Aiming at the existing problem that the delay constraint has not been used to improve the service quality, the present invention provides a power allocation method in a heterogeneous cell network.

As shown in Figure 1, the power allocation method in the heterogeneous cell network provided by the embodiment of the present invention

S101. Obtain a time delay parameter, and determine energy efficiency of the small base station according to the obtained time delay parameter;

S102, describing the power allocation problem as maximizing the energy efficiency of the small base station;

S103. Determine the transmit power when the power allocation problem reaches Nash equilibrium;

S104, based on the determined transmit power, regard energy efficiency in power allocation as a reward value function in Q learning, and solve an optimal power allocation scheme through a guess-based multi-user Q learning algorithm.

The power allocation method in the heterogeneous cell network described in the embodiment of the present invention acquires a delay parameter, and determines the energy efficiency of the small base station according to the acquired delay parameter; the power allocation problem is described as maximizing the energy efficiency of the small base station; Determine the transmit power when the power allocation problem reaches Nash equilibrium; based on the determined transmit power, the energy efficiency in power allocation is regarded as the reward value function in Q-learning, and the optimal power allocation is solved by a guess-based multi-user Q-learning algorithm Program. In this way, the quality of service of the cell is guaranteed by introducing a delay parameter (also called: delay constraint), and the power allocation problem is modeled as a non-cooperative supermodel game, and the transmit power when the power allocation problem reaches Nash equilibrium is determined. Based on the determined transmit power, the optimal power allocation scheme is solved by the guess-based multi-user Q learning algorithm. This method can maximize the energy efficiency and throughput of the cell under the premise of ensuring the quality of service and considering the energy efficiency. Meet the rapidly increasing access demand.

In order to better understand the power allocation method in the heterogeneous cell network described in the embodiment of the present invention, it will be described in detail, and the method may include:

Step 1, set up the system model of Orthogonal Frequency Division Multiple Access (OFDMA) cell downlink data link, for example, as shown in Figure 2, adopt a radius to be the macrocell of 500 meters, small base station (Small base station) cell) are freely distributed in the macro cell, the minimum distance between the small base station and the macro cell is 40 meters, the minimum distance between the small base stations is 300 meters, the carrier frequency is 2GHz, the bandwidth B is 1MHz, and the number of subchannels N is 30, Among them, N ₀ is the power spectral density of the added Gaussian white noise, the value of N ₀ can be -174dBm/Hz, and σ ² represents the added Gaussian white noise power. Initialize relevant parameters in the system model, for example, the transmit power of small base station k on subchannel n, the link between small base station k and user f, and the channel power of small base station k to user f on subchannel n gain and the channel power gain from small base station j to user f on subchannel n.

It should be noted:

1) Since the system model of the downlink data link of the OFDMA cell is established, only one user is allowed to be accessed in the same time slot on the sub-channel of the cell.

2) A small base station is a low-power wireless access node with a small transmission power between 100mW and 5W and a light weight between 2 and 10kg, and is used for coverage in hotspot areas.

Step 2, calculate the received SINR and received data rate

1) Introduce delay parameters According to the introduced delay parameter Calculate the received SINR:

in, Indicates the received signal-to-interference-noise ratio of user f on sub-channel n of small base station k; Indicates the channel power gain from small base station k to user f on subchannel n; Indicates the transmit power of small base station k on subchannel n; Indicates the transmit power of other small base stations except small base station k; is the delay parameter, which represents the interference of other small base stations and macro cells except small base station k on subchannel n to user f under the kth small base station; δ ² represents Gaussian white noise.

2) Calculate the received data rate within the transmission time _Tf according to the received SINR:

in, Indicates the receiving data rate of user f on sub-channel n of small base station k, T _f represents the transmission time, B represents the bandwidth, Yes abbreviated form of .

3) Calculate the total received data rate:

in, Indicates the total received data rate of user f on subchannel n, K indicates the number of small base stations, Yes abbreviated form of .

Step 3, according to the calculated total received data rate Calculate Effective Capacity

in, is the effective capacity, which means that the small base station k can support the maximum data rate of user f; θ _kf represents the service quality index of user f under small base station k; E( ) represents the error function; for the shorthand form of for abbreviated form of .

In this embodiment, the energy efficiency is defined as the ratio of the effective capacity to the total energy consumption of the cell. In the case of known circuit power and transmission power, the energy efficiency of the user f under the small base station k is calculated according to the obtained effective capacity

in, Yes The abbreviated form of , represents the energy efficiency of user f under small base station k, T _f represents the transmission time, P _k represents the total downlink transmission power, and P _c represents the circuit power.

In this example,

Step 4, in this embodiment, the overall goal is to maximize the energy efficiency of the small base station Therefore, power allocation can be formulated as the problem:

In this embodiment, the receiving signal-to-interference-noise ratio γ _kf of user f under small base station k must also satisfy the preset SINR threshold, and in order to ensure the quality of service of macrocell users, the transmit power It must be less than or equal to the hidden constraint power p _mask introduced, namely: Therefore, the power allocation problem can be described as the following convex optimization problem:

in, Represents the preset SINR threshold, p _mask represents the preset hidden constraint power, K' represents the set of small base stations, F' represents the set of users contained in the small base station, N' represents the set of subchannels, denote the maximum and minimum transmit power of the small base station k, respectively, defined as

Step 5, regard the power allocation problem as a non-cooperative game, choose an appropriate strategy for each small base station, when the energy efficiency Reaching the optimal value or close to the optimal value is said to reach the Nash equilibrium.

Step 6, through verification, the power allocation scheme in step 5 meets the conditions of the supermodel game and can be rewritten expression, and at least reach a set of Nash equilibria.

In this embodiment, after satisfying the supermodular game, the interference and noise from other cells are much smaller than the received signal power, so the energy efficiency of the system can be updated to:

Step 7, under the supermodular game condition, when the small base station network reaches the maximum benefit and the best response of each user is single-valued, then the user updates the transmission power scheme from the maximum value (or minimum value) of its strategy space, will converge to the Nash equilibrium, and after reaching the Nash equilibrium, the updated transmit power is expressed as

in, Represents the condition of reaching Nash equilibrium, the transmit power of small base station k on subchannel n; Indicates the conditions for reaching Nash equilibrium, the transmit power of other small base stations except k.

Step 8, according to the determined transmit power Perform power allocation, introduce a multi-user Q learning algorithm in power allocation, each user is independent of each other and does not exchange any information, define small base stations as agents, power allocation as actions, and define action sets to update the energy at this time efficiency specific:

1) In the process of solving the convex optimization problem by the multi-user Q-learning algorithm, define the state

In this example, in, Indicates the state of small base station k at time slot t, Indicates the transmit power of small base station k on sub-channel n under action α _k , and the value of I _k is {0,1}, indicating the state of receiving SINR of user f under small base station k, defined as

Among them, γ _kf represents the received signal-to-interference-noise ratio of user f under small base station k, Indicates the preset SINR threshold, Indicates the received signal-to-interference-noise ratio of user f on sub-channel n of small base station k, Denotes the actions in the action set of small base station k on subchannel n, Indicates the action set of other small base stations except small base station k on subchannel n.

2) The energy efficiency in power distribution Think of it as the reward-value function in Q-learning as follows:

Among them, s _k represents the state discrete set of small base station k.

3) Power selection of small base station k in state s _k to ensure QoS and maximize energy efficiency for the target, namely: Among them, π _k ( ) represents the policy set of small base station k; π _-k ( ) represents the policy set of other small base stations except small base station k, β represents the cost factor in Q learning, and the value of β belongs to 0 to 1 .

Step 9, initialize the current state Then select any action a _k from the action set to iterate and reach the next state The Q value update rule is When the return value reaches the maximum value, the optimal solution is obtained

In this embodiment, the Q value update rule is:

in, s' _k and s _k represent the state of small base station k at time slot t and t+1, respectively, α _t represents the learning rate, a _k represents the action concentration of small base station k, and a _-k represents the actions except for small base station k A _-k is the action set of all small base stations except small base station k, and β _k is the action in the action set A _k of small base station k.

When the return value reaches the maximum value, the optimal solution is obtained According to the optimal solution obtained Determine the power allocation plan.

Step 10, since it is difficult to obtain all the information of all small base stations, a multi-user-based Q-learning algorithm conjecture is proposed, and a linear function is given expression, and rewrite the Q function based on the conjecture to get a linear function based on the conjecture

In this example, the obtained linear function based on conjecture It can be expressed as:

in, is a positive scalar constant value, and denote the conjecture and policy set in the last slot, respectively.

In this embodiment, after the guess-based strategy is completed, the Q value update rule in step 9 becomes:

Step 11, it is beneficial to strengthen the selection of the power strategy during the learning process. Greedy selection is an effective way to balance exploration and utilization, but the selection of equal status will cause disadvantages. In this embodiment, the Boltzmann distribution is introduced. Select the next action α _k to iterate until the optimal power allocation scheme is obtained.

In this example, the strategy set based on Boltzmann distribution It can be expressed as:

Among them, τ is a positive parameter, representing the heat value.

In this embodiment, the heat value τ is analyzed, and the larger the τ, the smaller the difference between the corresponding probability values. This shows that the power allocation method in the heterogeneous cell network described in this embodiment can maximize the energy efficiency and cell throughput.

In summary, the power allocation method in the heterogeneous cell network described in this application guarantees the quality of service by introducing delay constraints and effective capacity in the cell, models the power allocation problem as a non-cooperative supermodel game, and proves its It converges to Nash equilibrium, and then transforms into a convex optimization problem, which is solved by a guess-based multi-user Q learning algorithm to maximize energy efficiency and cell throughput.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them.

The above description is a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, these improvements and modifications It should also be regarded as the protection scope of the present invention.

Claims (10)

1. A power allocation method in a heterogeneous cell network, characterized in that, comprising: Obtaining a time delay parameter, and determining the energy efficiency of the small base station according to the obtained time delay parameter; Formulate the power allocation problem as maximizing the energy efficiency of small base stations; Determine the transmit power at which the power allocation problem reaches Nash equilibrium; Based on the determined transmit power, the energy efficiency in power allocation is regarded as the reward value function in Q-learning, and the optimal power allocation scheme is solved by a guess-based multi-user Q-learning algorithm. 2. The method for allocating power in a heterogeneous cell network according to claim 1, wherein said determining the energy efficiency of the small base station according to the acquired time delay parameter comprises: Determine the received SINR according to the acquired time delay parameter; Determine the total received data rate according to the determined received SINR; Determine the effective capacity based on the determined total received data rate; Based on the determined effective capacity, the energy efficiency of the small base station is determined. 3. the power allocation method in the heterogeneous cell network according to claim 2, is characterized in that, described reception SINR is expressed as: in, Indicates the received signal-to-interference-noise ratio of user f on sub-channel n of small base station k; Indicates the channel power gain from small base station k to user f on subchannel n; Indicates the transmit power of small base station k on subchannel n; Indicates the transmit power of other small base stations except small base station k; is the delay parameter, which represents the interference of other small base stations and macro cells except small base station k on subchannel n to user f under the kth small base station; δ ² represents Gaussian white noise. 4. the power distribution method in the heterogeneous cell network according to claim 3, is characterized in that, described total receiving data rate is expressed as: in, Indicates the total received data rate of user f on subchannel n, K indicates the number of small base stations, Yes the shorthand form of Indicates the receiving data rate of user f on sub-channel n of small base station k, T _f represents the transmission time, B represents the bandwidth, Yes abbreviated form of . 5. the power distribution method in the heterogeneous cell network according to claim 4, is characterized in that, described effective capacity is expressed as: in, is the effective capacity, which means that the small base station k can support the maximum data rate of user f; θ _kf represents the service quality index of user f under small base station k; E( ) represents the error function; for abbreviated form of . 6. The power allocation method in the heterogeneous cell network according to claim 5, wherein the energy efficiency of the small base station is expressed as: in, Represents the energy efficiency of user f under small base station k, T _f represents the transmission time, P _k represents the total downlink transmission power, and P _c represents the circuit power. 7. The power allocation method in the heterogeneous cell network according to claim 6, wherein the power allocation problem is described as maximizing the energy efficiency of the small base station: in, Yes the shorthand form of Represents the preset SINR threshold, p _mask represents the preset hidden constraint power, K' represents the set of small base stations, F' represents the set of users contained in the small base station, N' represents the set of subchannels, denote the maximum and minimum transmit power of the small base station k, respectively, defined as 8. the power distribution method in the heterogeneous cell network according to claim 1, is characterized in that, the updating rule of Q value in Q learning algorithm for: in, s' _k and s _k represent the state of small base station k at time slot t and t+1 respectively, α _t represents the learning rate, Represents the reward value function, a _k represents the action set of small base station k, a _-k represents the action set of other small base stations except small base station k, and A _-k represents the actions of all small base stations except small base station k. The action set of the base station, β represents the cost factor in Q learning, β _k represents the action in the action set A _k of the small base station k, Denotes the policy set of small base station j, Indicates the transmit power of small base station k on sub-channel n under action α _k , and I _k represents the state of received SINR of user f in small base station k. 9. the power distribution method in the heterogeneous cell network according to claim 8, is characterized in that, to the update rule of Q value in the Q learning algorithm To update, the update rule of the guess-based Q value is: in, Represents a guess-based linear function. 10. The power allocation method in the heterogeneous cell network according to claim 9, wherein the linear function based on guess is expressed as: in, is a constant parameter, a positive scalar constant value; and Denote conjecture, strategy set and strategy set respectively, and τ represents the heat value.