JP2022018880A - Optimization method of wireless communication system, wireless communication system, and program for wireless communication system - Google Patents

Abstract

To provide an optimization method of a wireless communication system that calculates a setting parameter and an usage condition by using a computer in a situation in which a plurality of wireless communication systems with different requirements are mixed while interfering in the same environment, a wireless communication system, and a program.SOLUTION: In a reinforcement learning model, wireless environment information including a state related to wireless communication is detected from a wireless communication terminal belonging to a plurality of wireless communication systems. The wireless environment information is provided to agents 12-1, 12-2, and 12-3 prepared in response to a condition imposed on each of the plurality of wireless communication systems. A computer is caused to execute reinforcement learning to which the condition and the wireless environment information are applied for each of the agents 12-1, 12-2, and 12-3. For each of the plurality of wireless communication systems, the computer is caused to calculate an optimum control parameter under the wireless communication environment on the basis of the result of reinforcement learning.SELECTED DRAWING: Figure 4

Description

The present invention relates to a method for optimizing a wireless communication system, a wireless communication system, and a program for a wireless communication system, and in particular, a method for optimizing a wireless communication system for optimizing a communication state by using learning of multi-step evaluation, wireless. Related to communication systems and programs for wireless communication systems.

More specifically, the present invention aims to optimize communication in an environment in which different wireless communication systems coexist and interfere with each other. Here, when different requirements are imposed on each wireless communication system, optimization is performed in consideration of each requirement. This optimization is performed by computer-based learning such as machine learning and reinforcement learning for one or more wireless communication systems.

Wireless LAN is a wireless communication system that can be used inexpensively in the license-free band. For this reason, its spread is rapidly increasing, and a large number of wireless LAN terminals are mixed in the same area. As a result, it is a problem that wireless LAN terminals interfere with each other. In response to these problems, many technologies have been proposed for minimizing the influence of interference between wireless LAN terminals and expanding the individual or overall system capacity.

For example, FIG. 1 shows an example in which wireless communication terminals 1 to N are wireless LAN base stations (APs: Access Points) that interfere with each other. The wireless communication terminals N + 1 to N + M shown in the lower part of FIG. 1 are user terminals such as smartphones that establish communication with the above AP. In this example, each of the wireless communication terminals 1 to N functioning as an AP acquires interference information in the vicinity thereof and information on success / failure of connection with the wireless communication terminals N + 1 to N + M, and sends the control server 10 as wireless environment information. Send.

The control server 10 calculates the allocation of frequency channels and transmission power values so that the throughput of the AP group including the wireless communication terminals 1 to N is maximized, and returns the result as control information to each AP.

By the way, in applications and devices that use wireless LAN, the items that should be emphasized may differ depending on the usage scene. For example, in a wireless communication system including an IoT sensor, the communication speed is not important. On the other hand, when installing a large number of IoT sensors in an area, it is important to increase the number of communications that can be established in the area.

Therefore, it is necessary to select a frequency channel with less interference even in a narrow band, rather than aiming to expand the system capacity. Further, when it is desired to communicate in a wide range, it is necessary to control to maximize the transmission power within a range that does not affect other wireless communication systems, instead of emphasizing interference from other wireless communication systems. As described above, the required control policy differs depending on the application of the wireless communication system used.

In particular, a plurality of wireless communication systems coexist in the 920 MHz band, which is currently open for RFID and IoT in Japan. Specifically, this band is used in, for example, the following systems.
1. 1. Wireless communication system that periodically transmits sensor information such as position information and temperature 2. Wireless communication system that transmits video using surveillance cameras 3. Wireless communication system that requires network construction over a wide area such as mountains and the ocean

These wireless communication systems are expected to have different requirements and at the same time be mixed on the same frequency channel. Therefore, it is necessary to calculate the control information for these on the assumption that they are mixed on the same frequency resource.

In the configuration example of the wireless communication system shown in FIG. 1, a plurality of wireless communication terminals 1 to N are connected to the control server 10 in an environment in which they interfere with each other. Further, the wireless communication terminals 1 to N can perform data communication with other wireless communication terminals N + 1 to N + M by using wireless communication.

In the system configuration, each of the wireless communication terminals 1 to N transmits wireless environment information to the control server 10. For example, in the case of wireless LAN communication, the wireless environment information means frequency channel usage information such as SSID and channel usage rate, as well as parameters set in the wireless communication terminal. This parameter includes the frequency channel in use, channel bandwidth, transmit power value, and so on. The wireless environment information is acquired by the wireless communication terminals 1 to N by performing carrier sense in their respective surroundings.

In the conventional method, after collecting the wireless environment information, the control server 10 is optimized on the assumption that all the wireless communication terminals 1 to N have the same specifications and similarly require the communication capacity. Perform the calculation. The optimization calculation is performed, for example, on the position and width of the frequency channel and the transmission power. The result of the calculation is transmitted from the control server 10 to the wireless communication terminals 1 to N as control information. The wireless communication terminals 1 to N that have received the control information change the corresponding set value according to the control information.

If the control is executed periodically or due to some trigger, the optimum parameters are calculated again based on the updated information such as the frequency channel usage information in addition to the initial value calculation method. .. Then, the control information is determined by the calculation and transmitted to each of the wireless communication terminals 1 to N.

FIG. 2 is a flowchart of a conventional control example. In S100, wireless environment information of each of the wireless communication terminals 1 to N functioning as an AP is collected.

In S102, the optimum parameter is calculated based on the collected information. In the conventional control, it is determined that the same communication capacity is required for all of the wireless communication terminals 1 to N. Therefore, the same evaluation function is used for all the wireless communication terminals 1 to N, and the optimum wireless communication parameter is calculated by a heuristic method such as an iterative calculation or a genetic algorithm.

The calculated parameter information is transmitted to the wireless communication terminals 1 to N as control information (S104).

After that, until the occurrence of the control trigger is recognized (S106), the standby process is taken (S110) until the end of control is determined (S108). Then, when the control trigger is generated, the processing after S100 is executed again. However, since it is difficult to fully consider the environment and control information of multiple wireless communication systems using only the heuristic method, optimization using machine learning and reinforcement learning has also been proposed.

Liang, Le et al., “Multi-Agent Reinforcement Learning for Spectrum Sharing in Vehicular Networks”, 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 1-5, 2019. Cheng Wu, Kaushik Chowdhury, Marco Di Felice, and Waleed Meleis, “Spectrum management of cognitive radio using multi-agent reinforcement learning”, In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: Industry track (AAMAS '10), 1705-1712, 2010.

For services and applications that use various communication devices and wireless communication, methods incorporating iterative calculation and machine learning have been proposed in order to optimize wireless communication resources. These methods assume that all wireless communication systems have a unified purpose, such as expansion of communication capacity.

However, in reality, the same requirements are not always imposed on each system in an environment where a plurality of different wireless communication systems coexist. That is, in reality, the purpose or goal of optimization may differ from system to system. Traditional methods aimed at single optimization are inadequate for such situations. In this case, it is necessary to realize optimization that reflects the requirements for each wireless device or the requirements according to each usage scene.

The present invention provides setting parameters and usage conditions for a wireless communication system in a situation where a plurality of wireless communication systems imposed with a plurality of different constraints or different requirements coexist while interfering with each other in the same environment. , It is an object of the present invention to provide an optimization method of a wireless communication system calculated by using learning using a computer.

The first invention is a method for optimizing a wireless communication system executed in a wireless communication environment in which a plurality of different wireless communication systems coexist in order to achieve the above object, and is a radio belonging to the plurality of wireless communication systems. The wireless environment information is provided to each of the steps prepared for detecting the wireless environment information including the state related to the wireless communication from the communication terminal and the agents prepared in response to the conditions imposed on each of the plurality of wireless communication systems. Optimal under the wireless communication environment for each of the multiple wireless communication systems, a step of causing the computer to perform enhanced learning to which the conditions and the wireless environment information are applied to each of the agents. It is desirable to include a step of causing a computer to calculate the control parameter based on the result of the reinforcement learning, and a step of providing the control parameter to the wireless communication terminal belonging to the corresponding wireless communication system.

The second invention is a wireless communication system that operates in a wireless communication environment in which a plurality of different wireless communication systems coexist, and receives wireless environment information from the plurality of wireless communication systems and the plurality of wireless communication systems. The control server is provided with a control server that provides control information, and the control server detects wireless environment information including a state related to wireless communication from wireless communication terminals belonging to the plurality of wireless communication systems, and the plurality of wireless communication systems. The process of providing the radio environment information to each of the agents prepared in response to the conditions imposed on each of the agents and the reinforcement learning to apply the conditions and the radio environment information to each of the agents are carried out. The process, the process of calculating the optimum control parameters under the wireless communication environment for each of the plurality of wireless communication systems based on the result of the enhanced learning, and the process of calculating the control parameters according to the corresponding wireless communication system. It is desirable to execute the process provided to the wireless communication terminal belonging to the above.

The third invention is a program for a wireless communication system implemented in a control server that receives wireless environment information from a plurality of wireless communication systems and provides control information to the plurality of wireless communication systems, and the control server. In addition, the process of detecting wireless environment information including the state related to wireless communication from the wireless communication terminals belonging to the plurality of wireless communication systems and the conditions imposed on each of the plurality of wireless communication systems were prepared. The process of providing the wireless environment information to each of the agents, the process of performing enhanced learning to which the conditions and the wireless environment information are applied to each of the agents, and the process of performing the enhanced learning to each of the plurality of wireless communication systems are described. The process of calculating the optimum control parameters under the wireless communication environment in which a plurality of wireless communication systems are operating based on the result of the enhanced learning, and the control parameters are the wireless devices belonging to the corresponding wireless communication system. It is desirable to execute the process provided to the communication terminal.

According to the present invention, when a plurality of different wireless communication systems coexist in a wireless communication environment, an agent for reinforcement learning is prepared for each condition imposed on the wireless communication system. Then, reinforcement learning can be advanced by applying the conditions corresponding to each of the agents. Therefore, according to the present invention, in a situation where a plurality of wireless communication systems coexist while interfering with each other in the same environment, the setting parameters and usage conditions of the wireless communication system can be optimized for each condition. ..

It is a figure which shows the configuration example of a wireless communication system. It is a flowchart of the control example of a wireless communication system. It is a figure for demonstrating the model example of the conventional reinforcement learning. It is a figure for demonstrating an example of the model of reinforcement learning carried out in Embodiment 1 of this invention. It is a figure for demonstrating an example of the environment to which the model shown in FIG. 4 is applied. It is a figure for demonstrating the mode of the aggregation which is allowed in the environment shown in FIG. It is a figure which shows the example of the channel allocation determined by the conventional method. It is a figure which shows the example of the channel allocation determined by the method of Embodiment 1 of this invention. It is a figure for demonstrating an example of the model of reinforcement learning carried out in Embodiment 2 of this invention.

Embodiment 1.
[Structure of Embodiment 1]
The wireless communication system of the first embodiment of the present invention can be realized by the configuration example shown in FIG. In FIG. 1, the wireless communication terminals 1 to N shown in the middle stage each function as an access point (AP). These can communicate with the wireless communication terminals N + 1 to N + M shown in the lower part of FIG. The wireless communication terminals N + 1 to N + M are composed of a smartphone, a sensor for IoT, a smart meter, and the like. As described above, the configuration shown in FIG. 1 includes a plurality of wireless communication systems that share the same frequency resource but have different standards and specifications.

The wireless communication system of this embodiment includes a control server 10. The control server 10 includes hardware such as a communication interface, a processor unit, and a memory. The control server 10 realizes the functions described later by having these hardware proceed with processing according to a program stored in the memory.

The control server 10 can provide control information to the wireless communication terminals 1 to N that function as APs. The control information includes, for example, information such as available frequency resources and transmission power. On the other hand, the wireless communication terminals 1 to N can transmit wireless environment information to the control server 10. The wireless environment information includes interference information in the vicinity of each of the wireless communication terminals 1 to N and information on success or failure of connection with the wireless communication terminals N + 1 to N + M.

Further, the control server 10 is provided with a learning function for optimizing various parameters included in the control information based on wireless environment information and the like, and a function for determining these various parameters based on the learning result. ing.

[Outline of reinforcement learning]

In this embodiment, reinforcement learning is used for optimizing various parameters included in the control information. FIG. 3 shows a model diagram of general reinforcement learning. In the model shown in FIG. 3, an agent 12 exists as a learning target. The agent 12 calculates and executes the action A (t + 1) from the current state S (t) and the reward R (t) in the unique environment 14 with the observation timing of the event as t. As a result, the state S (t + 1) is realized. From this state S (t + 1), the reward R (t + 1) for evaluating the action is obtained, and the next action is calculated.

In the following description, s and S represent states, a and A represent actions, and r and R represent rewards, respectively. Here, lowercase letters mean parameters for individual agents (optimization targets), and uppercase letters mean parameters for their set (plurality of agents). Further, the subscript t of each parameter indicates that the parameter is a value at the observation timing t, and St, At, and Rt are the same as S (t), A (t), and R (t), respectively. It shall be.

The reinforcement learning shown in FIG. 3 is advanced by repeating the following steps.
1. 1. The agent 12 receives the state S (t) and the reward R (t) from the environment 14, and returns the action A (t) determined based on the policy π to the environment 14.
2. 2. The environment 14 changes to the next state S (t + 1) based on the action A (t) received from the agent 12 and the current state S (t), and the state S (t + 1) and the reward R (t + 1) after the transition. ) Is provided to the agent 12. The reward R is a scalar amount indicating the quality of the action A immediately before that.

Assuming that the action of the agent for a certain state S is A, the sum of the rewards R that can be obtained from the present time to the infinite future, that is, the profit G is as follows.

However, γ is 0 ≦ γ ≦ 1, and is a parameter for adjusting how much the influence of future rewards is evaluated as profit.

In Q-learning by reinforcement learning, the value of action a is evaluated by the following function.

However, E is a function indicating an expected value. Further, Q ^π is a value function (hereinafter referred to as “Q function”) representing an expected value when an agent taking action a from the state s takes an action according to the policy π.

The reinforcement learning shown in FIG. 3 is advanced so as to maximize this Q function. This learning can be advanced, for example, by obtaining a Q function for estimating the profit G when the action a is performed in the state s by the algorithm of the following equation.

Here, p is a parameter called the learning rate, which is an algebra determined by the machine learning designer. Usually set to a small value less than 1. Further, maxQ indicates the maximum value of the Q function that is considered to be ideally acquired. The learning of the Q function is to estimate all the Q values obtained by the action taken at the next time t + 1 for each time t, and update the Q value using the largest one among them.

[Characteristics of Embodiment 1]
FIG. 4 shows a model of reinforcement learning implemented in the wireless communication system of the present embodiment. In this embodiment, optimization is achieved for a plurality of wireless communication systems having different conditions. Multiple wireless communication systems can be grouped based on their respective conditions. In the model shown in FIG. 4, there are three groups, and an agent exists for each group.

Agents 12-1, 12-3, 12-3 shown in FIG. 4 evaluate the behavior of user i belonging to each group. For example, the agent 12-1 can calculate the reward R and the action A from the state S of the user i included in the group 1. Further, the agents 12-2 and 12-3 calculate the reward R from the state S of the user i belonging to each of them and determine the action A.

The three agents 12-1, 12-2, and 12-3 shown in FIG. 4 may output different actions A based on the reward R and the state S provided to each. Then, in the model shown in FIG. 4, different rewards R and different states S may be provided to agents 12-1, 12-2, and 12-3 from the same environment 14.

Hereinafter, the description will be continued with reference to FIGS. 5 and 6 by taking as an example the case where there are four different wireless communication terminals satisfying the same wireless communication standard. FIG. 5 is a diagram showing the requirements and the like for the four terminals to be controlled in this example in an organized manner. Further, FIG. 6 shows an example of the aggregation allowed in this example.

In this example, how to allocate frequency resources to all terminals (frequency channel position and frequency channel width) is controlled. As the frequency resource, there are four unit channels of channels 1 to 4. These channels can be used by aggregating two or four.

The requirements for the four wireless communication terminals are as follows.
1. 1. Since the wireless communication terminal 1 is used as a master unit in a sensor network, "maximization of the success rate of uplink transmission from a large number of terminals (sensors)" is a requirement.
2. 2. Since the wireless communication terminal 2 is used in a wide area sensor network, "maximization of transmission reach" is a requirement.
3. 3. Since the wireless communication terminals 3 and 4 are used as a master unit in data distribution, "maximization of downlink throughput to subordinate terminals" is a requirement.

In addition to the above requirements, each wireless communication terminal is naturally subject to some requirements. The above-mentioned requirements are the conditions that should be given the highest priority according to the nature of each of the several conditions required for each wireless communication terminal.

When the wireless communication terminal 1 is an agent 1, the wireless communication terminal 2 is an agent 2, and the wireless communication terminals 3 and 4 are agents 3, the calculation of each reward can be set as follows.

As described above, the requirement condition of the agent 1 is "maximization of the uplink transmission success rate from a large number of terminals". Therefore, the agent 1 determines the action A so as to maximize the transmission success rate of the transmission terminal that carries out uplink communication under the control of the wireless communication terminal 1. In this example, it is assumed that all terminals can carry out carrier sense without LAN. In this case, the transmission success rate can be expressed by the following equation.

However, N shown in the above equation is the total number of transmitting terminals existing in the same channel. Here, for uplink traffic, terminals under the control of all master units in the same channel are transmission terminals. In the case of downlink traffic, all master units in the same channel are transmission terminals. The above N is the sum of the number of upstream traffic transmitting terminals and the number of downlink traffic transmitting terminals. Further, τ in the above equation is the probability that each transmitting terminal performs transmission. This probability can be calculated based on the total number N above. Further, n'in the above equation is the number of transmission terminals connected to the master unit to be controlled.

The requirement of the agent 2 is "maximization of transmission reach". Therefore, the agent 2 considers the propagation characteristics, the power density, the frame error rate, and the like. When it is possible to select from a plurality of frequency channels, the reward considering the propagation characteristics, that is, the transmission reach distance D can be expressed by the following formula.

However, L in the above equation is a function for obtaining attenuation due to propagation, and L ^-1 is an inverse function thereof. Further, fc is the center frequency of the transmission signal, and B is the bandwidth.

For the agent 3, "maximization of downlink throughput to subordinate terminals" is a requirement. Therefore, the agent 3 evaluates the throughput when all the terminals can carry out carrier sense by wireless LAN, as in the case of the agent 1, for example. In this case, the throughput can be calculated by the following equation.

However, E [L] in the above equation is the average number of bits at the time of successful transmission, and E [I] is the average waiting time. Further, Ts in the equation is the average transmission frame time, and Tc is the average time wasted in the collision.

Optimization control by the conventional method is carried out with the aim of maximizing the throughput equally for all terminals. In this case, channel allocation is determined so that interference between terminals does not occur. More specifically, as shown in FIG. 7, one channel is assigned to each of the wireless communication terminals 1 to 4. That is, for the wireless communication terminals 3 and 4 mainly for downlink transmission, frequency resources are allocated with a narrow bandwidth even though there is almost no collision between the transmitting terminals. As a result, the throughput of the wireless communication terminals 3 and 4 is lower than the throughput that can be originally realized.

On the other hand, in this embodiment, the channel allocation is determined, for example, as shown in FIG. Here, the channel 2 is assigned to the wireless communication terminal 1 and the channel 1 is assigned to the wireless communication terminal 2. Since the wireless communication terminals 1 and 2 are components of the sensor network, they are easily affected by external interference. Therefore, these terminals 1 and 2 are assigned a unit channel having a narrow bandwidth. Further, the wireless communication terminal 2 is required to communicate over a wide area. The signal propagation loss decreases as the center frequency of communication decreases. The channel 1 assigned to the wireless communication terminal 2 has the lowest frequency and is considered to have the minimum signal propagation loss.

On the other hand, for the wireless communication terminals 3 and 4 where maximization of throughput is important, channels 3 + 4 in which two unit channels are aggregated are assigned. According to this allocation, the wireless communication terminals 3 and 4 interfere with each other. However, since all of them are mainly downlink traffic, the transmission terminals are mainly two master units. In this case, the probability of collision due to coexistence within the same frequency channel is low. Therefore, even if two terminals coexist in the same channel, the effect of increasing the instantaneous throughput can be expected by increasing the bandwidth by aggregation unless the scenario is such that the two terminals always compete with each other.

As described above, according to the wireless communication system of the present embodiment, different agents can be set for each of a plurality of wireless communication terminals having different requirements and the like, and reinforcement learning for optimization can be promoted. Then, the behavior of each terminal can be individually and independently optimized according to the requirements for each terminal and the like. Therefore, according to the wireless communication system of the present embodiment, in an area where different types of terminals having different requirements and the like coexist, each terminal can exert the maximum performance with respect to the required requirements. Can be done.

Since this example is a simple example, it can be modeled in a Markov process, for example, but the actual environment becomes complicated and difficult to model due to the influence of hidden terminals and the like. Therefore, when assuming a real environment, reinforcement learning is required. Further, when the environment is complicated, a method of using a simulation or a measurement result in a real space, a method of measuring a state or a reward using a database, or the like may be used instead of a mathematical model.

Further, in this example, an example of control for allocating frequency resources is shown, but the present invention is not limited thereto. In addition to the above examples, transmission power, transmission frequency (or average waiting time required for random access), parameters related to wireless communication systems such as RTS / CTS settings for wireless LAN, number of terminals that can be connected, power consumption value, etc. , Resources and settings used for wireless communication can be controlled.

Embodiment 2.
Next, a second embodiment of the present invention will be described with reference to FIG. The wireless communication system of the present embodiment can be realized by the configuration shown in FIG. 1 as in the case of the first embodiment.

FIG. 9 shows a model of reinforcement learning implemented in the wireless communication system of the present embodiment. The model shown in FIG. 9 includes a plurality of agents 12-1, 12-2, 12-3 as in the case of the first embodiment. And in this model, it is assumed that multiple different environments will be evaluated. More specifically, in the model shown in FIG. 9, a plurality of environments exist as environments for returning the behavior selected by each of the agents 12-1, 12-2, and 12-3. In this case, it becomes necessary to select the environment in order to evaluate the selected behavior.

When agents 12-1, 12-2, and 12-3 are defined by different requirements, the environment can be defined by different standards and communication systems. For example, as a wireless communication system for IoT, there are IEEE 802.11ah, Wi-SUN, or LoRa. These systems differ in the specified frequency bandwidth and modulation / demodulation method. Therefore, even if the received power value and the interference power value are the same, the probability that the transmission frame will cause an error will be different for each system.

Therefore, in an area where different wireless communication systems coexist and interfere with each other, it is necessary to estimate the influence of the interference on one system larger than the influence on the other system. Similar circumstances occur, for example, with respect to the evaluation of power consumption. That is, when comparing the actual machines, the hardware configurations of the wireless communication terminals are not always uniform, and those having a large battery capacity and those having a small capacity may coexist in the same area. The effect of power consumption needs to be estimated larger in a terminal having a small battery capacity than in a terminal having a large battery capacity.

Further, when a wireless communication terminal having a multi-RF function is a control target, it becomes necessary to evaluate the environment for each frequency band. For example, for a wireless communication terminal operating in a tri-band of 920MHz, 2.4GHz, or 5GHz, it is necessary to switch the environmental evaluation method depending on which frequency band the wireless communication terminal operates in.

In the model shown in FIG. 9, three environments 14-1, 14-2, and 14-3 are prepared. These environments 14-1, 14-2, and 14-3 are arranged so as to cover the environments that may be established for a plurality of wireless communication terminals to be controlled. Therefore, in the present embodiment, all the wireless communication terminals under the control of the control server 10 are operating under any of the environments 14-1, 14-2, and 14-3.

The model shown in FIG. 9 includes an environment selection unit 16. The environment selection unit 16 selects the environment in which the agent 12-1 is placed from the _three environments 14-1, 14-2, and 14-3, and provides the action A1 to the selected environment. As a result, even in a situation where a plurality of different environments coexist, the action A1 of the agent _12-1 is returned to the correct environment. The environment selection unit 16 also selects the same environment for agents 12-2 and 12-3. As a result, the actions A ₂ and A ₃ selected by the agents 12-2 and 12-3 are also returned to the appropriate environment, respectively. The function of the environment selection unit 16 is realized by the control server 10 determining the environment in which the wireless communication terminals 1 to N are placed based on the wireless environment information.

The model shown in FIG. 9 further includes an agent selection unit 18. The agent selection unit 18 provides the appropriate agents with the states S1, S2 _{, S3} and the rewards _{R1 , R2} , R3 provided by the environments 14-1, 14-2, and _14-3 , respectively. do. The function of the agent selection unit 18 is realized by the control server 10 providing control information to an appropriate one of the wireless communication terminals 1 to N.

As described above, according to the model shown in FIG. 9, when optimizing control is performed for a plurality of wireless communication systems coexisting within the same frequency resource, a plurality of environments are appropriately switched and selected actions are taken. A can be evaluated. Similarly, in the model shown in FIG. 9, even when a wireless communication terminal that operates by appropriately switching a plurality of frequency bands is included in the control target, the selected action A is useful by switching a plurality of environments. Gender can be evaluated appropriately.

10 Control server 12, 12-1, 12-2, 12-3 Agent 14, 14-1, 14-2, 14-3 Environment 16 Environment selection unit 18 Agent selection unit

Claims (7)

It is a method of optimizing a wireless communication system executed in a wireless communication environment in which a plurality of different wireless communication systems coexist.
A step of detecting wireless environment information including a state related to wireless communication from wireless communication terminals belonging to the plurality of wireless communication systems, and
A step of providing the wireless environment information to each of the agents prepared in response to the conditions imposed on each of the plurality of wireless communication systems.
A step of causing the computer to perform reinforcement learning to which the conditions and the wireless environment information are applied to each of the agents.
For each of the plurality of wireless communication systems, a step of causing a computer to calculate the optimum control parameters under the wireless communication environment based on the result of the reinforcement learning.
A step of providing the control parameter to the wireless communication terminal belonging to the corresponding wireless communication system, and
How to optimize wireless communication systems, including. The plurality of wireless communication systems include wireless communication systems that are subject to different conditions.
The optimization method according to claim 1, wherein in the reinforcement learning, the reward, state, and behavior of the agent are set for each of the same conditions and optimization is aimed at. The conditions imposed on each of the plurality of wireless communication systems include a plurality of requirements.
The optimization method according to claim 2, wherein in the reinforcement learning, the requirements that should be given the highest priority for each of the wireless communication systems are applied to the corresponding agents, and optimization is aimed at. In the wireless communication environment, a plurality of wireless communication systems conforming to different wireless communication standards are mixed.
The optimization method according to claim 2, wherein in the reinforcement learning, different conditions are set for each wireless communication standard and optimization is aimed at. The wireless communication environment is a mixture of different environments.
The optimization method according to claim 1, wherein in the reinforcement learning, an environmental evaluation corresponding to each of the agents is set for each of the plurality of environments, and optimization is aimed at. A wireless communication system that operates in a wireless communication environment in which a plurality of different wireless communication systems coexist.
A control server that receives wireless environment information from the plurality of wireless communication systems and provides control information to the plurality of wireless communication systems is provided.
The control server is
A process for detecting wireless environment information including a state related to wireless communication from wireless communication terminals belonging to the plurality of wireless communication systems.
A process of providing the wireless environment information to each of the agents prepared in response to the conditions imposed on each of the plurality of wireless communication systems.
A process of performing reinforcement learning by applying the conditions and the wireless environment information to each of the agents, and
For each of the plurality of wireless communication systems, a process of calculating the optimum control parameters under the wireless communication environment based on the result of the reinforcement learning, and
A process of providing the control parameter to the wireless communication terminal belonging to the corresponding wireless communication system, and
A wireless communication system that runs. A program for a wireless communication system implemented in a control server that receives wireless environment information from a plurality of wireless communication systems and provides control information to the plurality of wireless communication systems.
To the control server
A process for detecting wireless environment information including a state related to wireless communication from wireless communication terminals belonging to the plurality of wireless communication systems.
A process of providing the wireless environment information to each of the agents prepared in response to the conditions imposed on each of the plurality of wireless communication systems.
A process of performing reinforcement learning by applying the conditions and the wireless environment information to each of the agents, and
For each of the plurality of wireless communication systems, a process of calculating the optimum control parameters under the wireless communication environment in which the plurality of wireless communication systems are operating, and a process of calculating the optimum control parameters based on the result of the reinforcement learning.
A process of providing the control parameter to the wireless communication terminal belonging to the corresponding wireless communication system, and
A program for wireless communication systems to execute.

Images