CN118157143A - GA-MADRL-PPO combination-based distributed photovoltaic optimal scheduling strategy method, device and system - Google Patents

Abstract

The invention discloses a distributed photovoltaic optimal scheduling strategy method, device and system based on GA-MADRL-PPO combination. The method comprises the following steps: according to the modularity index and the active balance index of the distributed photovoltaic power distribution network, carrying out cluster division on a plurality of nodes in the distributed photovoltaic power distribution network to obtain a cluster division result; based on a cluster division result, constructing an initial scheduling model corresponding to the distributed photovoltaic power distribution network, wherein the initial scheduling model adopts a model framework of multi-agent deep reinforcement learning; performing near-end strategy optimization training on the initial scheduling model by using scheduling experience data of the distributed photovoltaic power distribution network to obtain a target scheduling model; and inputting real-time operation data of the distributed photovoltaic power distribution network into a target scheduling model to generate a scheduling strategy. The invention solves the technical problem of poor safety and reliability of the distributed photovoltaic system.

Description

A distributed photovoltaic optimization scheduling strategy method, device and system based on GA-MADRL-PPO combination

Technical Field

The present invention relates to the field of electric power technology, and in particular to a distributed photovoltaic optimization scheduling strategy method, device and system based on GA-MADRL-PPO combination, which can be specifically applied to the technical scenario of high-proportion distributed photovoltaic distribution network scheduling.

Background Art

The major decisions on carbon peak and carbon neutrality have provided policy support and a friendly environment for my country's energy structure adjustment. It has become a global consensus to promote clean energy substitution and renewable energy development. Distributed photovoltaics have the advantages of cleanliness and pollution-free, flexible scale, system safety, and peak-shaving performance, and have become an important part of my country's renewable energy system. However, the increasing penetration rate of distributed photovoltaics will affect the safety and stability of the system, such as local consumption problems, power reverse transmission problems, increased network losses, voltage over-limit, etc., while reducing the operational reliability and utilization rate of new energy. In this regard, how to dispatch distributed photovoltaic systems with higher penetration rates to improve system safety and stability has become one of the important technical issues in related technical fields.

To address the above-mentioned problems, no effective solution has been proposed yet.

Summary of the invention

The embodiments of the present invention provide a distributed photovoltaic optimization scheduling strategy method, device and system based on the combination of GA-MADRL-PPO, so as to at least solve the technical problem that the distributed photovoltaic scheduling scheme provided by the relevant technology is difficult to adapt to the situation of distributed photovoltaic output and load uncertainty, resulting in poor safety and reliability of the distributed photovoltaic system.

According to one aspect of an embodiment of the present invention, a distributed photovoltaic optimization scheduling strategy method based on the combination of GA-MADRL-PPO is provided, including: clustering multiple nodes in the distributed photovoltaic distribution network according to the modularity index and the active power balance index of the distributed photovoltaic distribution network to obtain clustering results; based on the clustering results, constructing an initial scheduling model corresponding to the distributed photovoltaic distribution network, wherein the initial scheduling model adopts a model framework of multi-agent deep reinforcement learning; using the scheduling experience data of the distributed photovoltaic distribution network, performing proximal strategy optimization training on the initial scheduling model to obtain a target scheduling model; inputting the real-time operation data of the distributed photovoltaic distribution network into the target scheduling model to generate a scheduling strategy, wherein the scheduling strategy is used to adjust the operating power of the distributed photovoltaic distribution network.

In the above-mentioned distributed photovoltaic optimization scheduling strategy method based on the combination of GA-MADRL-PPO, GA is the abbreviation of genetic algorithm (Genetic Algorithm), the above-mentioned MADRL is the abbreviation of the above-mentioned multi-agent deep reinforcement learning (Multi-Agent Deep Reinforcement Learning), and the above-mentioned PPO is the abbreviation of the above-mentioned proximal policy optimization (Proximal Policy Optimization).

Optionally, the distributed photovoltaic optimization scheduling strategy method based on the combination of GA-MADRL-PPO also includes: using the target sensitivity matrix corresponding to the distributed photovoltaic distribution network and the voltage amplitude change and reactive power change corresponding to multiple nodes to calculate the electrical distance between any two nodes among the multiple nodes, wherein the target sensitivity matrix is used to determine the sensitivity relationship between the node voltage and reactive power of the multiple nodes; based on the electrical distance and edge weight between any two nodes among the multiple nodes, constructing a modularity index; based on the net power data corresponding to the distributed photovoltaic distribution network, constructing an active power balance index.

Optionally, multiple nodes in the distributed photovoltaic distribution network are clustered according to the modularity index and the active power balance index of the distributed photovoltaic distribution network, and the clustering results include: weighted calculation of the modularity index and the active power balance index to obtain a comprehensive performance index; and clustering multiple nodes using the comprehensive performance index and the target genetic algorithm to obtain a clustering result.

Optionally, a plurality of nodes are clustered using a comprehensive performance indicator and a target genetic algorithm, and the clustering results include: encoding the adjacency matrix of the plurality of nodes to obtain an encoding result; performing fitness evaluation on the plurality of nodes based on the comprehensive performance indicator and the encoding result to obtain an evaluation result; in response to the evaluation result not meeting a preset fitness condition, adjusting the node crossover probability and the node compilation probability corresponding to the encoding result according to the evaluation result and a preset adjustment strategy, updating the encoding result and re-evaluating the fitness; in response to the evaluation result meeting the preset fitness condition, determining the clustering result based on the encoding result.

Optionally, based on the cluster division results, constructing an initial scheduling model corresponding to the distributed photovoltaic distribution network includes: determining multiple regional agents corresponding to multiple nodes according to the cluster division results, wherein the multiple regional agents have Markov decision control functions; constructing a deep reinforcement learning framework based on the multiple regional agents, wherein the deep reinforcement learning framework includes at least a communication layer, and the communication layer is used to realize collaborative decision-making of multiple regional agents; based on the deep reinforcement learning framework, constructing an initial scheduling model for the distributed photovoltaic distribution network.

Optionally, the initial scheduling model includes a state space; based on the deep reinforcement learning framework, constructing an initial scheduling model for the distributed photovoltaic distribution network includes: for any target area agent in the deep reinforcement learning framework, according to the target power generation power, target load power, interruptible load, electric energy storage capacity and scheduling period in the autonomous area corresponding to the target area agent, determining the state space corresponding to the target area agent in the initial scheduling model.

Optionally, the distributed photovoltaic optimization scheduling strategy method based on the combination of GA-MADRL-PPO also includes: obtaining the actual power generation power, actual load power, interruptible load and energy storage capacity of the autonomous area corresponding to the target area intelligent body during the scheduling period; superimposing a preset prediction deviation on the actual power generation power to obtain the target power generation power, and superimposing a preset prediction deviation on the actual load power to obtain the target load power, wherein the preset prediction deviation obeys a normal distribution.

Optionally, the initial scheduling model includes an action space, which includes multiple decision variables; based on the deep reinforcement learning framework, constructing an initial scheduling model for the distributed photovoltaic distribution network includes: for any target area intelligent agent in the deep reinforcement learning framework, using the state information and target constraints in the state space corresponding to the target area intelligent agent, calculate the multiple decision variables corresponding to the target area intelligent agent, wherein the target constraints include preset energy storage constraints and interruptible load constraints, and the multiple decision variables include: the interruptible load power within the autonomous area of the target area intelligent agent, the active power generated by the energy storage, and the reactive power generated by the energy storage.

Optionally, the initial scheduling model includes a reward function; based on the deep reinforcement learning framework, constructing an initial scheduling model for the distributed photovoltaic distribution network includes: for multiple regional agents in the deep reinforcement learning framework, determining a reward function based on the periodic operating cost constraints and the node voltage constraints, wherein the periodic operating cost is determined by the target operating costs of the multiple regional agents within the target scheduling cycle, the target operating cost includes the electric energy storage operating cost, the interruptible load cost and the electricity purchase cost, and the node voltage constraint is determined by the system rated voltage of the distributed photovoltaic distribution network and the target value range of the node voltage amplitude in the multiple regional agents.

Optionally, the dispatching experience data of the distributed photovoltaic distribution network is used to perform proximal strategy optimization training on the initial dispatching model to obtain the target dispatching model, including: sampling the historical dispatching data of the distributed photovoltaic distribution network in multiple historical dispatching cycles to obtain dispatching experience data; based on the dispatching experience data and the proximal strategy optimization algorithm, multiple rounds of offline training are performed on the initial dispatching model to update the strategy network parameters and value network parameters corresponding to multiple regional agents in the initial dispatching model to obtain the target dispatching model.

Optionally, the real-time operation data of the distributed photovoltaic distribution network is input into the target scheduling model, and the generation of the scheduling strategy includes: obtaining the real-time operation data, the real-time operation data at least including: the photovoltaic power generation power, real-time load power, interruptible load and electric energy storage power of the distributed photovoltaic distribution network in the current scheduling period; inputting the real-time operation data into the target scheduling model, and obtaining the decision information output by the target scheduling model, wherein the decision information is used to determine the target scheduling power corresponding to multiple autonomous areas of the distributed photovoltaic distribution network, the target scheduling power including: the target interruptible load power, the target active power emitted by the electric energy storage and the target reactive power emitted by the electric energy storage; generating the scheduling strategy according to the target scheduling power.

According to another aspect of an embodiment of the present invention, a distributed photovoltaic optimization scheduling strategy device based on the combination of GA-MADRL-PPO is also provided, including: a division module, used to cluster multiple nodes in a distributed photovoltaic distribution network according to the modularity index and active power balance index of the distributed photovoltaic distribution network, and obtain a cluster division result; a construction module, used to construct an initial scheduling model corresponding to the distributed photovoltaic distribution network based on the cluster division result, wherein the initial scheduling model adopts a model framework of multi-agent deep reinforcement learning; a training module, used to use the scheduling experience data of the distributed photovoltaic distribution network to perform proximal strategy optimization training on the initial scheduling model to obtain a target scheduling model; a scheduling module, used to input the real-time operation data of the distributed photovoltaic distribution network into the target scheduling model to generate a scheduling strategy, wherein the scheduling strategy is used to adjust the operating power of the distributed photovoltaic distribution network.

Optionally, the above-mentioned distributed photovoltaic optimization scheduling strategy device based on the combination of GA-MADRL-PPO also includes an indicator module, which is used to: use the target sensitivity matrix corresponding to the distributed photovoltaic distribution network and the voltage amplitude change and reactive power change corresponding to multiple nodes to calculate the electrical distance between any two nodes among the multiple nodes, wherein the target sensitivity matrix is used to determine the sensitivity relationship between the node voltage and reactive power of the multiple nodes; construct a modularity index based on the electrical distance and edge weight between any two nodes among the multiple nodes; and construct an active power balance index based on the net power data corresponding to the distributed photovoltaic distribution network.

Optionally, the above-mentioned division module is also used to: perform weighted calculation on the modularity index and the active power balance index to obtain a comprehensive performance index; and use the comprehensive performance index and the target genetic algorithm to cluster multiple nodes to obtain a cluster division result.

Optionally, the above-mentioned division module is also used to: encode the adjacency matrix of multiple nodes to obtain a coding result; based on the comprehensive performance index and the coding result, perform fitness evaluation on the multiple nodes to obtain an evaluation result; in response to the evaluation result not meeting the preset fitness condition, adjust the node crossover probability and node compilation probability corresponding to the coding result according to the evaluation result and the preset adjustment strategy, update the coding result and re-evaluate the fitness; in response to the evaluation result meeting the preset fitness condition, determine the cluster division result based on the coding result.

Optionally, the above-mentioned construction module is also used to: determine multiple regional agents corresponding to multiple nodes according to the cluster division results, wherein the multiple regional agents have Markov decision control functions; construct a deep reinforcement learning framework based on multiple regional agents, wherein the deep reinforcement learning framework includes at least a communication layer, and the communication layer is used to realize collaborative decision-making of multiple regional agents; based on the deep reinforcement learning framework, construct an initial scheduling model for the distributed photovoltaic distribution network.

Optionally, the initial scheduling model includes a state space; the above-mentioned building module is also used to: for any target area intelligent agent in the deep reinforcement learning framework, determine the state space corresponding to the target area intelligent agent in the initial scheduling model according to the target power generation power, target load power, interruptible load, electric energy storage capacity and scheduling period in the autonomous area corresponding to the target area intelligent agent.

Optionally, the above-mentioned distributed photovoltaic optimization scheduling strategy device based on the combination of GA-MADRL-PPO also includes a deviation module, which is used to: obtain the actual power generation power, actual load power, interruptible load and electric energy storage capacity of the autonomous area corresponding to the target area intelligent body during the scheduling period; superimpose a preset prediction deviation on the actual power generation power to obtain the target power generation power, and superimpose a preset prediction deviation on the actual load power to obtain the target load power, wherein the preset prediction deviation obeys a normal distribution.

Optionally, the initial scheduling model includes an action space, which includes multiple decision variables. The above-mentioned building module is also used to: for any target area intelligent agent in the deep reinforcement learning framework, use the state information and target constraints in the state space corresponding to the target area intelligent agent to calculate multiple decision variables corresponding to the target area intelligent agent, wherein the target constraints include preset energy storage constraints and interruptible load constraints, and the multiple decision variables include: the interruptible load power within the autonomous area of the target area intelligent agent, the active power generated by the energy storage, and the reactive power generated by the energy storage.

Optionally, the initial scheduling model includes a reward function; the above-mentioned building module is also used to: for multiple regional agents in the deep reinforcement learning framework, determine the reward function based on the periodic operating cost constraint and the node voltage constraint, wherein the periodic operating cost is determined by the target operating cost of the multiple regional agents within the target scheduling cycle, the target operating cost includes the electric energy storage operating cost, the interruptible load cost and the electricity purchase cost, and the node voltage constraint is determined by the system rated voltage of the distributed photovoltaic distribution network and the target value range of the node voltage amplitude in the multiple regional agents.

Optionally, the above-mentioned training module is also used to: sample historical scheduling data of the distributed photovoltaic distribution network in multiple historical scheduling cycles to obtain scheduling experience data; perform multiple rounds of offline training on the initial scheduling model based on the scheduling experience data and the proximal strategy optimization algorithm to update the strategy network parameters and value network parameters corresponding to multiple regional agents in the initial scheduling model to obtain the target scheduling model.

Optionally, the above-mentioned scheduling module is also used to: obtain real-time operation data, the real-time operation data at least includes: the photovoltaic power generation power, real-time load power, interruptible load and electric energy storage power of the distributed photovoltaic distribution network in the current scheduling period; input the real-time operation data into the target scheduling model, and obtain decision information output by the target scheduling model, wherein the decision information is used to determine the target scheduling power corresponding to multiple autonomous areas of the distributed photovoltaic distribution network, the target scheduling power includes: the target interruptible load power, the target active power emitted by the electric energy storage and the target reactive power emitted by the electric energy storage; generate a scheduling strategy according to the target scheduling power.

According to another aspect of an embodiment of the present invention, a distributed photovoltaic optimization scheduling strategy system based on the combination of GA-MADRL-PPO is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the above-mentioned distributed photovoltaic optimization scheduling strategy methods based on the combination of GA-MADRL-PPO.

In an embodiment of the present invention, according to the modularity index and active balance index of the distributed photovoltaic distribution network, multiple nodes in the distributed photovoltaic distribution network are clustered to obtain clustering results; based on the clustering results, an initial scheduling model corresponding to the distributed photovoltaic distribution network is constructed, wherein the initial scheduling model adopts a model framework of multi-agent deep reinforcement learning; using the scheduling experience data of the distributed photovoltaic distribution network, the initial scheduling model is trained for proximal strategy optimization to obtain a target scheduling model; the real-time operation data of the distributed photovoltaic distribution network is input into the target scheduling model to generate a scheduling strategy, wherein the scheduling strategy is used to adjust the operating power of the distributed photovoltaic distribution network. Thus, the present invention achieves the purpose of combining multi-agent deep reinforcement learning and proximal strategy optimization to construct a scheduling model to complete the scheduling of the distributed photovoltaic distribution network, thereby achieving the technical effect of enhancing the adaptability of the distributed photovoltaic system to uncertain situations and improving the safety and reliability of the distributed photovoltaic system, thereby solving the technical problem that the distributed photovoltaic scheduling scheme provided by the related technology is difficult to adapt to the situation of distributed photovoltaic output and load uncertainty, resulting in poor safety and reliability of the distributed photovoltaic system.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

1 is a hardware structure block diagram of a terminal device for an optional distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO combination according to an embodiment of the present invention;

2 is a flow chart of a distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO combination according to an embodiment of the present invention;

3 is a schematic diagram of an optional distributed photovoltaic optimization scheduling process based on GA-MADRL-PPO combination according to an embodiment of the present invention;

FIG4 is a schematic diagram of an optional MADRL-based distributed photovoltaic optimization scheduling framework according to an embodiment of the present invention;

FIG5 is a structural block diagram of a distributed photovoltaic optimization scheduling strategy device based on GA-MADRL-PPO combination according to an embodiment of the present invention.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

According to an embodiment of the present invention, an embodiment of a distributed photovoltaic optimization scheduling strategy method based on the combination of GA-MADRL-PPO is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

FIG. 1 is a hardware structure block diagram of a terminal device for a distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO combination according to an embodiment of the present invention. As shown in FIG. 1 , the terminal device 10 may include one or more processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor (Microcontroller Unit, referred to as MCU) or a programmable logic device (Field Programmable Gate Array, referred to as FPGA)), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display device 110, an input/output device 108 (i.e., an I/O device), a Universal Serial Bus (USB) port (which may be included as one of the ports of a computer bus, not shown in the figure), a network interface (not shown in the figure), a power supply (not shown in the figure) and/or a camera (not shown in the figure). It can be understood by those skilled in the art that the structure shown in FIG. 1 is only for illustration, and it does not limit the structure of the above-mentioned terminal device 10. For example, the terminal device 10 may also include more or fewer components than those shown in FIG. 1 , or have a configuration different from that shown in FIG. 1 .

It should be noted that the one or more processors 102 and/or other data processing circuits may be embodied in whole or in part as software, hardware, firmware or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be fully or partially integrated into any of the other components in the terminal device 10 (or mobile device).

The memory 104 can be used to store software programs and modules of application software, such as the program instructions/data storage device corresponding to the distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal device 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

Under the above operating environment, an embodiment of the present invention provides a distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO as shown in FIG2. FIG2 is a flow chart of a distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO according to an embodiment of the present invention. As shown in FIG2, the method includes the following implementation steps:

Step S201, clustering multiple nodes in the distributed photovoltaic distribution network according to the modularity index and active power balance index of the distributed photovoltaic distribution network to obtain a clustering result;

Step S202, based on the cluster division result, construct an initial scheduling model corresponding to the distributed photovoltaic distribution network, wherein the initial scheduling model adopts a model framework of multi-agent deep reinforcement learning;

Step S203, using the dispatching experience data of the distributed photovoltaic distribution network, performing proximal strategy optimization training on the initial dispatching model to obtain a target dispatching model;

Step S204: input the real-time operation data of the distributed photovoltaic distribution network into the target scheduling model to generate a scheduling strategy, wherein the scheduling strategy is used to adjust the operating power of the distributed photovoltaic distribution network.

In order to ensure the safe and reliable access of large-scale distributed photovoltaic power to the power grid, an effective solution is to configure energy storage. At present, some research results have been used to solve the problem of flexibility assessment of distribution networks, including the installation of energy storage, which can alleviate the voltage and power fluctuations caused by photovoltaic output fluctuations to a certain extent, and reduce the absorption problems caused by distributed photovoltaic power reverse transmission; considering the energy storage capacity optimization method for photovoltaic absorption, effectively improving the photovoltaic absorption rate; using energy storage to charge and discharge at different times to alleviate the volatility of distributed power output; using the reactive power regulation capability of energy storage to dynamically optimize the distribution network, which significantly improves the active power loss of the distribution network.

Another effective means is to divide the distributed photovoltaic power distribution network into clusters and conduct unified dispatching technology research to achieve friendly interaction between high-penetration distributed power sources and the power grid and maximize grid-connected consumption. The cluster management mode of high-penetration new energy helps to smooth out fluctuations in new energy output, increase the local absorption rate of new energy, and fundamentally solve various problems caused by the increase in distributed photovoltaic penetration. According to the above-mentioned method steps provided in the embodiment of the present invention, it is possible to have the ability to adapt to the uncertainty of distributed photovoltaic output and load during the dispatching process, and give optimized dispatching results in real time according to the random changes in source and load, thereby realizing the coordinated operation of multiple autonomous regions and improving economic efficiency.

In an embodiment of the present invention, according to the modularity index and active balance index of the distributed photovoltaic distribution network, multiple nodes in the distributed photovoltaic distribution network are clustered to obtain clustering results; based on the clustering results, an initial scheduling model corresponding to the distributed photovoltaic distribution network is constructed, wherein the initial scheduling model adopts a model framework of multi-agent deep reinforcement learning; using the scheduling experience data of the distributed photovoltaic distribution network, the initial scheduling model is trained for proximal strategy optimization to obtain a target scheduling model; the real-time operation data of the distributed photovoltaic distribution network is input into the target scheduling model to generate a scheduling strategy, wherein the scheduling strategy is used to adjust the operating power of the distributed photovoltaic distribution network. Thus, the present invention achieves the purpose of combining multi-agent deep reinforcement learning and proximal strategy optimization to construct a scheduling model to complete the scheduling of the distributed photovoltaic distribution network, thereby achieving the technical effect of enhancing the adaptability of the distributed photovoltaic system to uncertain situations and improving the safety and reliability of the distributed photovoltaic system, thereby solving the technical problem that the distributed photovoltaic scheduling scheme provided by the related technology is difficult to adapt to the situation of distributed photovoltaic output and load uncertainty, resulting in poor safety and reliability of the distributed photovoltaic system.

Some optional implementation methods of the above method in the embodiment of the present invention are further introduced below.

Optionally, the above-mentioned distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO combination may also include the following method steps:

Step S251, using the target sensitivity matrix corresponding to the distributed photovoltaic distribution network and the voltage amplitude change and reactive power change corresponding to the multiple nodes, the electrical distance between any two nodes among the multiple nodes is calculated, wherein the target sensitivity matrix is used to determine the sensitivity relationship between the node voltage and reactive power of the multiple nodes;

Step S252, constructing a modularity index based on the electrical distance and edge weight between any two nodes among the plurality of nodes;

Step S253: constructing an active power balance index based on the net power data corresponding to the distributed photovoltaic distribution network.

Optionally, in the above step S201, clustering is performed on multiple nodes in the distributed photovoltaic distribution network according to the modularity index and the active power balance index of the distributed photovoltaic distribution network to obtain a clustering result, and the following execution steps may also be included:

Step S211, performing weighted calculation on the modularity index and the active power balance index to obtain a comprehensive performance index;

Step S212, clustering multiple nodes using comprehensive performance indicators and target genetic algorithm to obtain clustering results.

Optionally, in the above step S212, clustering is performed on multiple nodes using comprehensive performance indicators and a target genetic algorithm to obtain a clustering result, and the following execution steps may also be included:

Step S2121, encoding the adjacency matrices of multiple nodes to obtain encoding results;

Step S2122, based on the comprehensive performance index and the encoding result, fitness evaluation is performed on multiple nodes to obtain evaluation results;

Step S2123, in response to the evaluation result not meeting the preset fitness condition, adjusting the node crossover probability and the node compilation probability corresponding to the coding result according to the evaluation result and the preset adjustment strategy, updating the coding result and re-evaluating the fitness;

Step S2124: in response to the evaluation result satisfying the preset fitness condition, determining a cluster division result based on the encoding result.

Optionally, in the above step S202, based on the cluster division result, constructing an initial scheduling model corresponding to the distributed photovoltaic distribution network may further include the following execution steps:

Step S221, determining multiple regional agents corresponding to multiple nodes according to the cluster division result, wherein the multiple regional agents have a Markov decision control function;

Step S222, constructing a deep reinforcement learning framework based on the multiple regional agents, wherein the deep reinforcement learning framework includes at least a communication layer, and the communication layer is used to realize collaborative decision-making of the multiple regional agents;

Step S223: construct an initial scheduling model for the distributed photovoltaic distribution network based on a deep reinforcement learning framework.

Optionally, the initial scheduling model includes a state space; in the above step S223, based on the deep reinforcement learning framework, constructing an initial scheduling model for the distributed photovoltaic distribution network may also include the following execution steps:

Step S2231, for any target area agent in the deep reinforcement learning framework, determine the state space corresponding to the target area agent in the initial scheduling model according to the target power generation power, target load power, interruptible load, energy storage capacity and scheduling period in the autonomous area corresponding to the target area agent.

Optionally, the above-mentioned distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO combination may also include the following method steps:

Step S261, obtaining the actual power generation power, actual load power, interruptible load and electric energy storage capacity of the autonomous area corresponding to the target area agent during the scheduling period;

Step S262, superimposing the preset prediction deviation on the actual power generation power to obtain the target power generation power, and superimposing the preset prediction deviation on the actual load power to obtain the target load power, wherein the preset prediction deviation obeys a normal distribution.

Optionally, the initial scheduling model includes an action space, and the action space includes multiple decision variables; in the above step S223, based on the deep reinforcement learning framework, the initial scheduling model is constructed for the distributed photovoltaic distribution network, and the following execution steps may also be included:

Step S2232, for any target area intelligent agent in the deep reinforcement learning framework, multiple decision variables corresponding to the target area intelligent agent are calculated using the state information and target constraints in the state space corresponding to the target area intelligent agent, wherein the target constraints include preset energy storage constraints and interruptible load constraints, and the multiple decision variables include: the interruptible load power within the autonomous area of the target area intelligent agent, the active power generated by the energy storage, and the reactive power generated by the energy storage.

Optionally, the initial scheduling model includes a reward function; in the above step S223, based on the deep reinforcement learning framework, constructing an initial scheduling model for the distributed photovoltaic distribution network may also include the following execution steps:

Step S2233, for multiple regional agents in the deep reinforcement learning framework, determine the reward function based on the periodic operating cost constraint and the node voltage constraint, wherein the periodic operating cost is determined by the target operating cost of the multiple regional agents within the target scheduling cycle, and the target operating cost includes the electric energy storage operating cost, the interruptible load cost and the electricity purchase cost, and the node voltage constraint is determined by the system rated voltage of the distributed photovoltaic distribution network and the target value range of the node voltage amplitude in the multiple regional agents.

Optionally, in the above step S203, the dispatching experience data of the distributed photovoltaic distribution network is used to perform proximal strategy optimization training on the initial dispatching model to obtain the target dispatching model, and the following execution steps may also be included:

Step S231, sampling historical dispatching data of the distributed photovoltaic distribution network in multiple historical dispatching cycles to obtain dispatching experience data;

Step S232, based on the scheduling experience data and the proximal strategy optimization algorithm, the initial scheduling model is trained offline for multiple rounds to update the strategy network parameters and value network parameters corresponding to multiple regional agents in the initial scheduling model to obtain the target scheduling model.

Optionally, in the above step S204, the real-time operation data of the distributed photovoltaic distribution network is input into the target scheduling model to generate a scheduling strategy, which may also include the following execution steps:

Step S241, obtaining real-time operation data, the real-time operation data at least includes: photovoltaic power generation power, real-time load power, interruptible load and electric energy storage power of the distributed photovoltaic distribution network in the current scheduling period;

Step S242, inputting the real-time operation data into the target scheduling model, and obtaining the decision information output by the target scheduling model, wherein the decision information is used to determine the target scheduling power corresponding to the multiple autonomous areas of the distributed photovoltaic distribution network, and the target scheduling power includes: the target interruptible load power, the target active power generated by the electric energy storage, and the target reactive power generated by the electric energy storage;

Step S243: generating a scheduling strategy according to the target scheduling power.

Based on the above method steps, a more specific implementation method is further described below in combination with application scenarios.

In order to solve the scheduling problem of distributed photovoltaic distribution network with high penetration, the present invention proposes a distributed photovoltaic optimization scheduling strategy based on GA-MADRL-PPO combination. Specifically, in the application scenario, the online optimization scheduling process shown in Figure 3 is performed according to the above distributed photovoltaic optimization scheduling strategy. As shown in Figure 3, first, according to the cluster division principle, the distributed photovoltaic in the distribution network is clustered by using a genetic algorithm, and a comprehensive performance index based on the modularity index of the electrical distance and the active balance index is constructed as a comprehensive division basis. Secondly, according to the cluster division results, the distribution network with high penetration of distributed photovoltaic is divided into several regional intelligent agents to realize autonomous control, and a partitioned distributed photovoltaic optimization scheduling framework of multi-agent deep reinforcement learning is constructed according to the CommNet architecture. Then, with the lowest daily operating cost as the goal, and considering the corresponding constraints, a day-ahead optimization scheduling model containing state space, action space and reward function is constructed. Finally, the model is trained offline using the proximal strategy optimization algorithm, and the trained model is used for online optimization scheduling decisions. This technical solution solves the problem that the fluctuation of distributed photovoltaic and load output affects the normal operation of the distribution network, and improves the economy of system operation.

The method for constructing a comprehensive performance index based on the modularity index and the active power balance index of the electrical distance includes the following steps A01 to A06.

Step A01: Modularity is used to measure whether the cluster division result is reasonable. The larger the value, the more reasonable the cluster division. When the entire network is divided into one cluster, the modularity is 0; when each node is divided into different clusters, the modularity is negative, and there is no community structure at this time.

Specifically, the above modularity It is defined by the following formulas (1) to (3).

In the above formulas (1) to (3), _Aij represents the edge weight connecting nodes i and j, which is 1 when nodes i and j are directly connected and 0 when they are not connected; _kij represents the sum of the edge weights of all edges connected to node i; m represents the sum of the edge weights of the entire network; δ is a 0-1 matrix, if nodes i and j are in the same cluster, then δ(i,j)=1, if they are not in the same cluster, then δ(i,j)=0.

Step A02, using sensitivity to calculate the electrical distance, the specific calculation method is shown in the following formula (4) and formula (5).

ΔV＝ _SVQΔQ Formula (4)

In the above formulas (4) and (5), ΔV represents the change in voltage amplitude; ΔQ represents the change in reactive power; _SVQ represents the voltage-reactive sensitivity matrix of the system; _dij represents the ratio of the change in voltage amplitude of node j to that of node i when the reactive power of node j changes. The smaller the ratio, the greater the influence of node j on node i, that is, the closer the distance.

Assuming that there are n nodes in the network, the electrical distance e _ij between node j and node i is calculated using the Euclidean distance according to the following formula (6).

Furthermore, the edge weight A _ij based on the electrical distance between the node i and the node j is defined according to the following formula (7).

In the above formula (7), e represents the electrical distance matrix composed of the electrical distances e _ij between any two nodes in the network. Substituting this formula (7) into formula (1) can obtain the modularity index based on electrical distance:

Step A03, define the active power balance according to the following formula (8) and formula (9).

In the above formula (8) and formula (9), N _c represents the number of cluster divisions; M represents the set of all clusters; represents the active balance of cluster c; P _c,t represents the net power of cluster c at time t; T is the time length of the specified scenario, Indicates the overall active balance of the network.

Step A04, define the comprehensive performance index according to the following formula (10).

In the above formula (10), w ₁ and w ₂ represent the weights of the modularity index and the active power balance index respectively.

Step A05, taking the comprehensive performance index as the fitness function, the index size measures the quality of the division result, and using the adjacency matrix between the distribution network nodes for encoding, firstly obtain the "0-1" adjacency matrix according to the connection status of the distribution network nodes. During encoding, the genetic algorithm forms a new adjacency matrix by searching and modifying the element 1 in the matrix, 0 represents disconnection, 1 represents connection, and the modified matrix represents a division result of the network cluster, that is, the new individual encoded.

Step A06, an adaptive method is used to adjust the crossover probability and mutation probability. The adjustment principle is: if the fitness is less than the average fitness, a larger crossover probability and mutation probability are assigned; if the fitness is greater than the average fitness, a crossover probability and mutation probability are assigned according to the iteration state. The above adjustment of the crossover probability and mutation probability can be performed using the adjustment formulas shown in the following formulas (11) and (12).

In the above formulas (11) and (12), _Pc represents the crossover probability; _Pc,max and _Pc,min represent the maximum and minimum values of the crossover probability respectively; _Pm represents the mutation probability; _Pm,max and Pm _,min represent the maximum and minimum values of the mutation probability respectively; I represents the number of current iterations; _Imax represents the set maximum number of iterations; _fc represents the larger fitness of the two individuals performing the crossover operation; _fm represents the fitness of the individual performing the mutation operation; and _favg represents the average fitness of the population.

According to the above method steps provided by an embodiment of the present invention, the construction of a partitioned distributed photovoltaic optimization scheduling framework for multi-agent deep reinforcement learning includes the following steps B01 to B02.

Step B01, MADRL uses the multi-agent Markov decision process as the basic framework, which can be expressed as a tuple <N,{S ⁿ },{A ⁿ },P,R,γ>. Where N is the number of agents; S ⁿ is the state set of agent n, represents the state of the agent in time period t. The states of all agents are combined to form a joint state vector S = S ¹ ×S ² ×... ×S ^N , and s _t ∈S; A ^N is the action set of agent n, represents the action selected by the agent in time period t. The actions of all agents are combined to form a joint action vector A = A ¹ × A ² × ... × A ^N , and a _t ∈ A; P: S × A × S → [0, 1] is the state transition probability, which represents the probability that the state of the environment is transferred to s _{t + 1} after executing action a _t under s _t ; R is the reward function, which represents the reward given by the environment after executing action a _t under s _t , and r _t ∈ R, γ is the discount factor. In time period t, each agent performs actions on the environment according to the observed state, obtains the reward r _t of the environment, and changes the state of the environment to s _{t + 1} , so as to interact with the environment cyclically and use the generated data to modify the joint action strategy, that is, the mapping relationship between joint action and joint state π: S → A = a ~ π (a | s) to maximize the cumulative return.

Step B02, CommNet is used as the multi-agent deep reinforcement learning neural network architecture. According to the network structure, each agent with distributed photovoltaics has a neural network with the same structure to make decisions. In time period t, for the nth agent, in order to protect privacy, its network input is its own observation state Output is decision action In the encoding layer, the encoding function is The input information is converted into hidden state information and input into the communication layer. The communication layer is the key to realize the cooperation of each intelligent agent. Each intelligent agent sends the hidden state information h _t,m-1 (m represents the mth communication) to the communication layer network f _m , and performs mean pooling on the output h _t,m of the communication layer network of the adjacent intelligent agent. The obtained result and h _t,m are used as the input of the next layer of neural network of each adjacent intelligent agent.

Specifically, the iterative relationship between the input and output of each layer of the above-mentioned intelligent agent communication layer can be expressed by the following formula (13) and formula (14).

In the above formulas (13) and (14), _Hm and _Cm represent the parameters to be updated in the mth communication layer network, σ represents the nonlinear activation function, and N(n) represents the set of agents adjacent to agent n. The last layer of the network is the decoding layer, which converts the hidden state information into actual actions.

According to the above method steps provided by the embodiment of the present invention, the day-ahead optimization scheduling model involved includes a state space, an action space and a reward function.

Specifically, the state space is the environmental information perceived by the agent. The state space of each agent is established to include the photovoltaic power generation, load power, interruptible load, energy storage capacity and the scheduling period t in this space. Based on this, the state of the agent in the nth area in period t is expressed as the following formula (15).

In the above formula (15), P _t ^n,PV and P _t ^n,load represent the photovoltaic power generation and load power in autonomous area n in period t, respectively; They represent the amount of energy storage and interruptible load in the autonomous region n during period t-1 respectively.

In order to deal with the uncertainty of distributed photovoltaic output and load, the historical forecast data of photovoltaic and load are superimposed with the randomness of the forecast deviation as the input of photovoltaic and load in formula (15), and it is assumed that the forecast deviation obeys the normal distribution shown in the following formulas (16) and (17).

σ＝εP _t ⁿ Formula (17)

In the above formulas (16) and (17), ΔP _t ⁿ represents the prediction deviation between the photovoltaic and load in autonomous area n during period t; μ and σ represent the expectation and standard deviation of the prediction deviation; ε is the percentage of the prediction deviation in the prediction value.

Specifically, the action space is a relevant decision variable. In the action space, the action space of each regional agent is the output of the controllable devices in its control area, which can be specifically expressed as the following formula (18).

In the above formula (18), _Ptn ^,L represents the interruptible load power in autonomous area n during period t; _Ptn ^,es , They respectively represent the active power and reactive power generated by the energy storage in the autonomous region n during time period t.

Furthermore, the electric energy storage constraints considered in the action space can be expressed as the following formulas (19) to (22).

In the above formulas (19) to (22), and They represent the maximum charging power and the maximum discharging power of the electric energy storage respectively; and They represent the maximum storage capacity and the minimum storage capacity respectively; η ^n,ch and η ^n,dis represent the charging efficiency and discharging efficiency of the energy storage respectively; Sn ^,es represents the maximum apparent power of the inverter connected to the energy storage in area n.

Furthermore, the interruptible load constraint considered in the action space is expressed as the following formula (23).

_Ptn ^,L ＜ _Ptn ^,Lmax Formula (23)

Specifically, for the reward function, the training goal of multi-agent deep reinforcement learning is to find the optimal strategy π ^* so that the expected value of the cumulative return is maximized, which is specifically expressed as the following formula (24).

In the above formula (24), _ρπ represents the trajectory formed by strategy π; T represents the total number of scheduling periods in a day.

Furthermore, in the above method steps provided by the embodiment of the present invention, the operating cost is considered based on the following concept: the optimization goal is to minimize the operating cost of the active distribution network, and the objective function is expressed as the following formula (25).

In the above formula (25), N represents the number of divided areas; They represent the energy storage operation cost and interruptible load cost in autonomous region n during period t respectively; c _grid (t) represents the cost of purchasing electricity from the upper-level power grid during period t.

For electric energy storage, considering its cost per kilowatt-hour, the cost per kilowatt-hour coefficient is defined as ρ, then the operating cost of the electric energy storage can be expressed as the following formula (26).

The cost of purchasing electricity from the upper power grid can be expressed as the following formula (27) and formula (28).

In the above formulas (27) and (28), λ _buy (t) and λ _sell (t) represent the prices of electricity purchased from the superior power grid and electricity sold to the superior power grid in period t, respectively; P _t ^grid represents the electric power purchased from the superior power grid, P _t ^grid >0 represents purchasing electricity, and P _t ^grid <0 represents selling electricity.

The interruptible load cost can be expressed as the following formula (29).

In the above formula (29), c _cu is the unit interruptible load compensation cost.

In summary, the total operating cost considered can be expressed as the following formula (30).

Considering the node voltage constraint, the voltage amplitude of each node needs to be limited to a safe range, which can be specifically expressed as the following formula (31).

0.95u _N ≤u _j,t ≤1.05u _N Formula (31)

In the above formula (31), u _j,t is the per unit value of the voltage at node j at time t; u _N is the rated voltage of the system.

Calculate the system power flow distribution after the scheduling decision for each intelligent agent. The reward obtained by the intelligent agent based on the voltage of each node can be expressed as the following formula (32).

In the above formula (32), k represents the penalty function.

The regional agents are in a cooperative relationship, so the environment feeds back the same global reward to the regional agents in each period, which can be expressed as the following formula (33).

r(t)＝F ₁ (t)+F ₂ (t) Formula (33)

Furthermore, the step of using the PPO algorithm to train the scheduling model offline includes the following steps D01 to D05.

Step D01, determine the total number of time periods T and the number of training rounds M of the regional agent neural network in each scheduling cycle, and randomly initialize the strategy network parameters θ _a and value network parameters θ _c of each regional agent.

Step D02, initialize the environment, take 00:00 every day as the initial scheduling time, and initialize the device status before scheduling at this time.

Step D03: Each regional agent interacts with the environment and collects regional status information As network input, output action information As a scheduling instruction, the global reward r _t is calculated after the scheduling of this period ends; the sampled experience (s _t , a _t , r _t , st ₊₁ ) of each period is stored in the experience pool for network parameter update.

Step D04: Regional agent training. After a scheduling cycle sampling is completed, the T experience in the experience pool is used to update the strategy network and value network of each agent using the gradient descent method. The update goal is to maximize the cumulative global reward of a scheduling cycle. The learning rates of the two types of networks are denoted as _la and l _c respectively.

Step D05, determining whether the set maximum number of training rounds M is reached, if so, the training is terminated; if not, the next round of network parameter update is performed.

According to the above method steps provided by an embodiment of the present invention, in a specific application scenario, a scheduling model is established for a distributed photovoltaic system according to the MADRL-based distributed power optimization scheduling framework as shown in FIG4. As shown in FIG4, an environmental model is constructed to divide the devices on the main network of the distributed photovoltaic system distribution network into n areas, and the devices in each area include: photovoltaic devices, load devices, energy storage devices, and interruptible devices.

Further, according to the above step D03, as shown in FIG4 , N regional agents interact with the environment model, and each region in the environment model sends regional state information Input the network, and the network returns the regional action information corresponding to the regional state information of each region to each region After each scheduling period (e.g., denoted as t) ends, the environment model calculates the global reward r _t after the scheduling of the period ends, and stores the sampled experience (s _t , a _t , r _t , s _t+1 ) corresponding to the current scheduling period in the experience pool for network parameter update.

Further, according to the above-mentioned step D04 and step D05, as shown in FIG4 , based on the data stored in the experience pool, the gradient descent method is used to continuously update the strategy network and value network of each agent until the training is completed.

In summary, the present invention achieves the purpose of combining multi-agent deep reinforcement learning and proximal strategy optimization to construct a scheduling model to complete the scheduling of distributed photovoltaic distribution networks, thereby achieving the technical effect of enhancing the adaptability of distributed photovoltaic systems to uncertain situations and improving the safety and reliability of distributed photovoltaic systems, and further solves the technical problem that the distributed photovoltaic scheduling scheme provided by the relevant technology is difficult to adapt to the situation of uncertain distributed photovoltaic output and load, resulting in poor safety and reliability of distributed photovoltaic systems.

In this embodiment, a distributed photovoltaic optimization scheduling strategy device based on GA-MADRL-PPO is also provided, which is used to implement the above embodiments and preferred implementations, and will not be repeated for what has been described. As used below, a "module" is a combination of software and/or hardware that can implement a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

FIG5 is a structural block diagram of a distributed photovoltaic optimization scheduling strategy device based on GA-MADRL-PPO combination according to an embodiment of the present invention. As shown in FIG5 , the device includes: a division module 501, which is used to cluster multiple nodes in a distributed photovoltaic distribution network according to a modularity index and an active power balance index of the distributed photovoltaic distribution network to obtain a cluster division result;

A construction module 502 is used to construct an initial scheduling model corresponding to the distributed photovoltaic distribution network based on the cluster division result, wherein the initial scheduling model adopts a model framework of multi-agent deep reinforcement learning;

The training module 503 is used to use the dispatching experience data of the distributed photovoltaic distribution network to perform proximal strategy optimization training on the initial dispatching model to obtain the target dispatching model;

The scheduling module 504 is used to input the real-time operation data of the distributed photovoltaic distribution network into the target scheduling model to generate a scheduling strategy, wherein the scheduling strategy is used to adjust the operating power of the distributed photovoltaic distribution network.

Optionally, the distributed photovoltaic optimization scheduling strategy device based on the GA-MADRL-PPO combination includes, in addition to all the above modules, an indicator module 505 (not shown in the figure), which is used to: use the target sensitivity matrix corresponding to the distributed photovoltaic distribution network and the voltage amplitude change and reactive power change corresponding to multiple nodes to calculate the electrical distance between any two nodes among the multiple nodes, wherein the target sensitivity matrix is used to determine the sensitivity relationship between the node voltage and reactive power of the multiple nodes; construct a modularity index based on the electrical distance and edge weight between any two nodes among the multiple nodes; and construct an active balance index based on the net power data corresponding to the distributed photovoltaic distribution network.

Optionally, the above-mentioned division module 501 is also used to: perform weighted calculation on the modularity index and the active power balance index to obtain a comprehensive performance index; and perform cluster division on multiple nodes using the comprehensive performance index and the target genetic algorithm to obtain a cluster division result.

Optionally, the above-mentioned division module 501 is also used to: encode the adjacency matrix of multiple nodes to obtain a coding result; based on the comprehensive performance index and the coding result, perform fitness evaluation on the multiple nodes to obtain an evaluation result; in response to the evaluation result not meeting the preset fitness condition, adjust the node crossover probability and node compilation probability corresponding to the coding result according to the evaluation result and the preset adjustment strategy, update the coding result and re-evaluate the fitness; in response to the evaluation result meeting the preset fitness condition, determine the cluster division result based on the coding result.

Optionally, the above-mentioned construction module 502 is also used to: determine multiple regional agents corresponding to multiple nodes according to the cluster division results, wherein the multiple regional agents have Markov decision control functions; construct a deep reinforcement learning framework based on the multiple regional agents, wherein the deep reinforcement learning framework includes at least a communication layer, and the communication layer is used to realize collaborative decision-making of multiple regional agents; based on the deep reinforcement learning framework, construct an initial scheduling model for the distributed photovoltaic distribution network.

Optionally, the initial scheduling model includes a state space; the above-mentioned construction module 502 is also used to: for any target area intelligent agent in the deep reinforcement learning framework, determine the state space corresponding to the target area intelligent agent in the initial scheduling model according to the target power generation power, target load power, interruptible load, electric energy storage capacity and scheduling period in the autonomous area corresponding to the target area intelligent agent.

Optionally, the distributed photovoltaic optimization scheduling strategy device based on the GA-MADRL-PPO combination includes, in addition to all the above modules, a deviation module 506 (not shown in the figure), which is used to: obtain the actual power generation power, actual load power, interruptible load and energy storage capacity of the autonomous area corresponding to the target area intelligent body during the scheduling period; superimpose a preset prediction deviation on the actual power generation power to obtain the target power generation power, and superimpose a preset prediction deviation on the actual load power to obtain the target load power, wherein the preset prediction deviation obeys a normal distribution.

Optionally, the initial scheduling model includes an action space, which includes multiple decision variables. The above-mentioned construction module 502 is also used to: for any target area intelligent agent in the deep reinforcement learning framework, use the state information and target constraints in the state space corresponding to the target area intelligent agent to calculate multiple decision variables corresponding to the target area intelligent agent, wherein the target constraints include preset electric energy storage constraints and interruptible load constraints, and the multiple decision variables include: the interruptible load power within the autonomous area of the target area intelligent agent, the active power generated by the electric energy storage, and the reactive power generated by the electric energy storage.

Optionally, the initial scheduling model includes a reward function; the above-mentioned building module 502 is also used to: for multiple regional agents in the deep reinforcement learning framework, determine the reward function based on the periodic operating cost constraint and the node voltage constraint, wherein the periodic operating cost is determined by the target operating cost of the multiple regional agents within the target scheduling cycle, the target operating cost includes the electric energy storage operating cost, the interruptible load cost and the electricity purchase cost, and the node voltage constraint is determined by the system rated voltage of the distributed photovoltaic distribution network and the target value range of the node voltage amplitude in the multiple regional agents.

Optionally, the above-mentioned training module 503 is also used to: sample historical scheduling data of the distributed photovoltaic distribution network in multiple historical scheduling cycles to obtain scheduling experience data; perform multiple rounds of offline training on the initial scheduling model based on the scheduling experience data and the proximal strategy optimization algorithm to update the strategy network parameters and value network parameters corresponding to multiple regional intelligent agents in the initial scheduling model to obtain the target scheduling model.

Optionally, the above-mentioned scheduling module 504 is also used to: obtain real-time operation data, the real-time operation data at least includes: the photovoltaic power generation power, real-time load power, interruptible load and electric energy storage power of the distributed photovoltaic distribution network in the current scheduling period; input the real-time operation data into the target scheduling model, and obtain decision information output by the target scheduling model, wherein the decision information is used to determine the target scheduling power corresponding to multiple autonomous areas of the distributed photovoltaic distribution network, the target scheduling power includes: the target interruptible load power, the target active power emitted by the electric energy storage and the target reactive power emitted by the electric energy storage; generate a scheduling strategy according to the target scheduling power.

It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

According to another aspect of an embodiment of the present invention, a distributed photovoltaic optimization scheduling strategy system based on the combination of GA-MADRL-PPO is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the aforementioned distributed photovoltaic optimization scheduling strategy methods based on the combination of GA-MADRL-PPO.

Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: clustering multiple nodes in the distributed photovoltaic distribution network according to the modularity index and active power balance index of the distributed photovoltaic distribution network to obtain clustering results; constructing an initial scheduling model corresponding to the distributed photovoltaic distribution network based on the clustering results, wherein the initial scheduling model adopts a model framework of multi-agent deep reinforcement learning; performing proximal strategy optimization training on the initial scheduling model using the scheduling experience data of the distributed photovoltaic distribution network to obtain a target scheduling model; inputting the real-time operation data of the distributed photovoltaic distribution network into the target scheduling model to generate a scheduling strategy, wherein the scheduling strategy is used to adjust the operating power of the distributed photovoltaic distribution network.

Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiment and its optional implementation manners, which will not be described in detail here.

The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, ROM, RAM, mobile hard disks, magnetic disks or optical disks.

The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims (13)

1. A distributed photovoltaic optimization scheduling strategy method based on GA-MADRL-PPO combination is characterized by comprising the following steps:

According to the modularity index and the active balance index of the distributed photovoltaic power distribution network, performing cluster division on a plurality of nodes in the distributed photovoltaic power distribution network to obtain a cluster division result;

Based on the cluster division result, constructing an initial scheduling model corresponding to the distributed photovoltaic power distribution network, wherein the initial scheduling model adopts a model framework of multi-agent deep reinforcement learning;

Performing near-end strategy optimization training on the initial scheduling model by using scheduling experience data of the distributed photovoltaic power distribution network to obtain a target scheduling model;

And inputting the real-time operation data of the distributed photovoltaic power distribution network to the target scheduling model to generate a scheduling strategy, wherein the scheduling strategy is used for adjusting the operation power of the distributed photovoltaic power distribution network.

2. The GA-MADRL-PPO-combination-based distributed photovoltaic optimal scheduling policy method of claim 1, further comprising:

calculating to obtain the electrical distance between any two nodes in the plurality of nodes by using a target sensitivity matrix corresponding to the distributed photovoltaic power distribution network and voltage amplitude variation and reactive power variation corresponding to the plurality of nodes, wherein the target sensitivity matrix is used for determining the sensitivity relationship between node voltages and reactive power of the plurality of nodes;

Constructing the modularity index based on the electrical distance and the side weight between any two nodes in the plurality of nodes;

And constructing the active balance index based on the net power data corresponding to the distributed photovoltaic power distribution network.

3. The distributed photovoltaic optimization scheduling policy method based on GA-MADRL-PPO combination according to claim 1, wherein performing cluster division on the plurality of nodes in the distributed photovoltaic power distribution network according to the modularity index and the active balance index of the distributed photovoltaic power distribution network, to obtain the cluster division result includes:

Weighting and calculating the module degree index and the active balance degree index to obtain a comprehensive performance index;

and carrying out cluster division on the plurality of nodes by utilizing the comprehensive performance index and the target genetic algorithm to obtain a cluster division result.

4. The GA-MADRL-PPO-combination-based distributed photovoltaic optimization scheduling policy method of claim 3, wherein performing cluster division on the plurality of nodes by using the comprehensive performance index and the target genetic algorithm to obtain the cluster division result comprises:

Encoding the adjacent matrixes of the plurality of nodes to obtain an encoding result;

based on the comprehensive performance index and the coding result, carrying out fitness evaluation on the plurality of nodes to obtain an evaluation result;

Responding to the evaluation result that the preset fitness condition is not met, adjusting the node crossing probability and the node compiling probability corresponding to the coding result according to the evaluation result and a preset adjustment strategy, updating the coding result and carrying out the fitness evaluation again;

and responding to the evaluation result to meet the preset fitness condition, and determining the cluster division result based on the coding result.

5. The distributed photovoltaic optimization scheduling policy method based on GA-MADRL-PPO combination according to claim 1, wherein constructing the initial scheduling model corresponding to the distributed photovoltaic power distribution network based on the cluster division result includes:

determining a plurality of regional agents corresponding to the plurality of nodes according to the cluster division result, wherein the regional agents have a Markov decision control function;

Constructing a deep reinforcement learning framework based on the plurality of regional agents, wherein the deep reinforcement learning framework comprises at least a communication layer for implementing collaborative decisions of the plurality of regional agents;

And constructing the initial scheduling model for the distributed photovoltaic power distribution network based on the deep reinforcement learning framework.

6. The GA-MADRL-PPO-combination-based distributed photovoltaic optimization scheduling policy method of claim 5, wherein the initial scheduling model comprises a state space; based on the deep reinforcement learning framework, constructing the initial scheduling model for the distributed photovoltaic power distribution network includes:

And for any one target area intelligent agent in the deep reinforcement learning framework, determining the state space corresponding to the target area intelligent agent in the initial scheduling model according to the target power generation, the target load power, the interruptible load quantity, the electric energy storage electric quantity and the scheduling period in the autonomous area corresponding to the target area intelligent agent.

7. The GA-MADRL-PPO-combination-based distributed photovoltaic optimal scheduling policy method of claim 6, further comprising:

Acquiring actual power generation, actual load power, the interruptible load quantity and the electric energy storage electric quantity of an autonomous region corresponding to the target region intelligent agent in the scheduling period;

and superposing a preset prediction deviation on the actual power to obtain the target power, and superposing the preset prediction deviation on the actual load power to obtain the target load power, wherein the preset prediction deviation obeys normal distribution.

8. The GA-MADRL-PPO-combination-based distributed photovoltaic optimization scheduling policy method of claim 6, wherein the initial scheduling model comprises an action space comprising a plurality of decision variables therein; based on the deep reinforcement learning framework, constructing the initial scheduling model for the distributed photovoltaic power distribution network includes:

For any one target area agent in the deep reinforcement learning framework, calculating to obtain the plurality of decision variables corresponding to the target area agent by using state information and target constraints in the state space corresponding to the target area agent, wherein the target constraints comprise preset electric energy storage constraints and interruptible load constraints, and the plurality of decision variables comprise: and the interruptible load power, the active power emitted by the electric energy storage and the reactive power emitted by the electric energy storage in the autonomous region of the target region intelligent agent.

9. The GA-MADRL-PPO-combination-based distributed photovoltaic optimization scheduling policy method of claim 6, wherein the initial scheduling model comprises a reward function; based on the deep reinforcement learning framework, constructing the initial scheduling model for the distributed photovoltaic power distribution network includes:

Determining the rewarding function for the plurality of regional agents in the deep reinforcement learning framework based on a periodic operation cost constraint and a node voltage constraint, wherein the periodic operation cost is determined by a target operation cost of the plurality of regional agents in a target scheduling period, the target operation cost comprises an electric energy storage operation cost, an interruptible load cost and a power purchase cost, and the node voltage constraint is determined by a system rated voltage of the distributed photovoltaic power distribution network and a target value range of node voltage amplitude values in the plurality of regional agents.

10. The GA-MADRL-PPO-combination-based distributed photovoltaic optimization scheduling policy method of claim 1, wherein performing near-end policy optimization training on the initial scheduling model using the scheduling experience data of the distributed photovoltaic power distribution network, obtaining the target scheduling model comprises:

Sampling historical scheduling data of the distributed photovoltaic power distribution network in a plurality of historical scheduling periods to obtain scheduling experience data;

and performing offline training on the initial scheduling model for a plurality of rounds based on the scheduling experience data and a near-end strategy optimization algorithm so as to update strategy network parameters and value network parameters corresponding to a plurality of regional agents in the initial scheduling model and obtain a target scheduling model.

11. The GA-MADRL-PPO-combination-based distributed photovoltaic optimization scheduling policy method of claim 1, wherein inputting the real-time operational data of the distributed photovoltaic power distribution network to the target scheduling model, generating the scheduling policy comprises:

Acquiring the real-time operation data, wherein the real-time operation data at least comprises: photovoltaic power generation power, real-time load power, interruptible load quantity and electric energy storage electric quantity of the distributed photovoltaic power distribution network in the current scheduling period;

Inputting the real-time operation data into the target scheduling model, and acquiring decision information output by the target scheduling model, wherein the decision information is used for determining target scheduling power corresponding to a plurality of autonomous areas of the distributed photovoltaic power distribution network, and the target scheduling power comprises: the target can interrupt load power, target active power sent by the electric energy storage and target reactive power sent by the electric energy storage;

and generating the scheduling strategy according to the target scheduling power.

12. A distributed photovoltaic optimal scheduling strategy device based on GA-MADRL-PPO combination is characterized by comprising:

the dividing module is used for carrying out cluster division on a plurality of nodes in the distributed photovoltaic power distribution network according to the modularity index and the active balance index of the distributed photovoltaic power distribution network to obtain a cluster division result;

the construction module is used for constructing an initial scheduling model corresponding to the distributed photovoltaic power distribution network based on the cluster division result, wherein the initial scheduling model adopts a model frame of multi-agent deep reinforcement learning;

The training module is used for performing near-end strategy optimization training on the initial scheduling model by using scheduling experience data of the distributed photovoltaic power distribution network to obtain a target scheduling model;

the scheduling module is used for inputting the real-time operation data of the distributed photovoltaic power distribution network to the target scheduling model to generate a scheduling strategy, wherein the scheduling strategy is used for adjusting the operation power of the distributed photovoltaic power distribution network.

13. A GA-MADRL-PPO-combination based distributed photovoltaic optimal scheduling policy system comprising a memory and a processor, the memory having stored therein a computer program, the processor being arranged to run the computer program to perform the GA-MADRL-PPO-combination based distributed photovoltaic optimal scheduling policy method of any of the claims 1 to 11.