CN114064997B - An artificial intelligence power dispatch decision-making system based on big data - Google Patents

Abstract

The present invention provides an artificial intelligence power dispatching decision system based on big data. The artificial intelligence power dispatching decision system based on big data includes: a data crawling module: developed using Focused Crawler software, searching and acquiring data sources related to the power industry from domestic and foreign industry websites and multimedia. The artificial intelligence power dispatching decision system based on big data can collect professional information about the power industry on the Internet through the cooperation of the data crawling module and the database storage module, and can manage and classify the collected information data through the database management module and the data processing and analysis module, so as to establish a professional data information library. At the same time, through the cooperation of the information crawling time identification module and the latest information push module, the effective information related to the power industry can be retrieved from the professional data information library using a Web page, and the intelligent push of the latest effective information can be realized, so as to realize the intelligent decision of power dispatching.

Description

An artificial intelligence power dispatch decision-making system based on big data

Technical Field

The present invention relates to the technical field of electric power dispatching, and in particular to an artificial intelligence electric power dispatching decision system based on big data.

Background Art

Power dispatching is an effective management method to ensure the safe and stable operation of the power grid, reliable power supply, and orderly power production. The replacement, improvement, and development of the power dispatching professional support system are closely related to national policies and industry development trends. At this stage, a large number of policy-oriented documents, technical innovation papers, and the latest industry standards are the basis for power dispatching decisions.

With the rapid development of the Internet era, modern information technologies such as big data and artificial intelligence can be used to integrate and analyze power technology and economic big data, expert big data, regional power industry big data, and Internet power information resources. However, in daily work, the existing power dispatching technical support system lacks the means to effectively retrieve, intelligently classify and push, and extract key and effective information on documents from higher-level authorities, the latest industry developments, etc., which will affect the effective decision-making of power dispatching.

Summary of the invention

In order to achieve the above purpose of the artificial intelligence power dispatching decision system based on big data, the present invention is implemented through the following technical solutions: an artificial intelligence power dispatching decision system based on big data, comprising:

Data crawling module: Developed using the Focused Crawler software, it searches for data sources related to the power industry from domestic and foreign industry websites and multimedia;

Database storage module: used to store the data source obtained by the data crawling module;

Database management module: Utilizes the intelligent management of MySQL database to process the data source in the database storage module, including filtering useless information, deduplication and error correction, label extraction, and data security management;

Data processing and analysis module: Analyze and process the data in the database storage module based on the Python network, and combine the Sharksearch algorithm to analyze the relevance of data information and topics, and complete the classification of data sources in the database storage module;

Professional data information database: Classify the data through the data processing and analysis module, and match the information data through regular expressions to establish a professional data information database related to the power industry to store and manage the data sources after classification;

Index generation unit: used to generate indexes according to data sources classified in the professional data information database;

Web page: used to input keyword information to be retrieved into the professional data information database;

Retrieval optimization module: used to optimize the confidence of Web page retrieval and recommend display;

Data statistics module: used to perform statistical analysis on data sources collected in the professional data information database;

Electric power dispatching information visualization platform: Based on the statistics of the data sources in the professional data module by the data statistics module, an electric power dispatching information visualization platform is established. The electric power dispatching information visualization platform includes a regional distribution module, a data classification proportion module, and a data download record module;

Regional distribution module: used for statistics of data sources in each region;

Data classification proportion module: used to classify and count data sources according to the types of data sources;

Data download record module: used to record and count the data sources downloaded from the professional information database;

Information crawling time identification module: used to identify the network access time of the crawled data source;

Latest information push module: used to automatically push the latest information related to the power industry to the Web page.

Furthermore, in the data crawling module, the specific process of data crawling includes the following process:

S1. Initialize the URL sequence to be crawled through Focused Crawler software development;

S2, using the URL sequence to be crawled to search for power industry websites and multimedia pages;

S3. When the search page is empty, the data crawling ends. When a page related to the power industry is found, the page content corresponding to the URL is downloaded.

S4. When the page is downloaded successfully, add the URL to the crawled list and continue to analyze and process the document. When the page fails to download, choose whether to retry. If retry is not required, add the URL to the crawled list and add relevant exception handling and log records. If retry is required, repeat S2-S3.

S5. When adding the URL to the crawled list, choose whether to collect the page. If the page needs to be collected, the page will be taken as valid data and stored in the database module;

S6. When collection is not needed, determine that the URL is a valuable URL, and continue to repeat S2-S5.

Furthermore, the data sources obtained in the data crawling module include power industry policies, papers, achievements, technologies, research and development directions, products, and planning information.

Furthermore, the data classification in the data classification proportion module includes standards, documents, software copyrights, and patents.

Furthermore, the regional distribution module, the data classification proportion module and the data download record module are connected in pairs.

Furthermore, there are multiple Web pages, and the Web pages are connected by a network.

Compared with the prior art, the present invention has the following beneficial effects:

1. This artificial intelligence power dispatching decision-making system based on big data can collect professional information about the power industry on the Internet through the cooperation of the data crawling module and the database storage module, and can manage and classify the collected information data through the database management module and the data processing and analysis module, so as to obtain a more professional data source, thereby establishing a professional data information database. At the same time, through the cooperation of the information crawling time identification module and the latest information push module, the Web page can be used to retrieve valid information related to the power industry in the professional data information database, and the intelligent push of the latest valid information can be realized, so as to realize intelligent decision-making of power dispatching.

2. The artificial intelligence power dispatching decision-making system based on big data can automatically and in real time collect professional information related to the power industry on the Internet through the data collection method of the data crawling module, which is conducive to obtaining the latest information on policies, industry standards and other relevant information of the power industry. The cooperation of the data statistics module and the professional data information database can conduct statistical analysis on the information within the professional information database, thereby establishing a power dispatching information visualization analysis platform. Through the cooperation between the regional distribution module, the data classification proportion module and the data download record module in the power dispatching information visualization analysis platform, a professional database prediction system can be formed, thereby making predictive analysis and intelligent decision-making on power dispatching.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of establishing a professional data information database and a power dispatching information visualization platform according to the present invention;

FIG2 is a flow chart of a data crawling method of a data crawling module of the present invention;

FIG3 is a schematic diagram of a power dispatch information visualization analysis platform according to the present invention;

FIG. 4 is a schematic diagram of the professional data information database and the latest information push of the Web page of the present invention.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

The implementation example of the artificial intelligence power dispatch decision system based on big data is as follows:

Please refer to Figures 1 to 4, an artificial intelligence power dispatch decision system based on big data, including:

Data crawling module: Developed using the Focused Crawler software, it searches for data sources related to the power industry from domestic and foreign industry websites and multimedia. Through network channels such as the State Grid Library, China National Knowledge Infrastructure, universities, governments, and self-media public accounts, it comprehensively collects effective information on policies, methods, and technologies related to power dispatching, power network security, power grid development, technological innovation, and products;

Database storage module: used to store the data source obtained by the data crawling module;

Database management module: Utilize the intelligent management of MySQL database to process the data source in the database storage module, including filtering useless information, deduplication and error correction, label extraction, and data security management, to ensure the safe storage and reasonable and effective use of data sources;

Data processing and analysis module: Based on the Python network, the large amount of professional information crawled by the data crawling module is used for data analysis and big data research. At the same time, combined with the Sharksearch algorithm, the relevance of data information and topics is analyzed to complete the classification of data sources in the database storage module;

Professional data information database: The data processing and analysis module is used to classify the data, and the information data is matched through regular expressions to establish a professional data information database related to the power industry, which is used to store and manage the data sources after classification, so that the professional data information database can classify and systematically store and manage the collected data sources;

Index generation unit: used to generate indexes according to the data sources classified in the professional data information database, so as to quickly retrieve specific information in the professional data information database;

Web page: used to input keyword information to be retrieved into the professional data information database. There are multiple Web pages, which are connected by a network. Effective information on the advantages of the power industry can be pushed and shared through the Web pages.

Retrieval optimization module: used to optimize the confidence of Web page retrieval and recommend display;

Data statistics module: used to conduct statistical analysis on the data sources collected in the professional data information database, and establish a power dispatch information visualization platform through real-time monitoring and statistical analysis of the data sources in the professional data information database;

Electric power dispatching information visualization platform: Based on the statistics of the data sources in the professional data module by the data statistics module, an electric power dispatching information visualization platform is established. The electric power dispatching information visualization platform includes a regional distribution module, a data classification proportion module, and a data download record module;

Regional distribution module: used for statistics of data sources in each region;

Data classification proportion module: used to classify and count data sources according to the types of data sources;

Data download record module: used to record and count the data sources downloaded from the professional information database;

Through the regional distribution module and the data classification proportion module, the number of data sources related to the power industry in a certain area and the proportion of each category in the power-related information of the data source can be analyzed. Through the regional distribution module, the data download record module and the data classification proportion module, the number of data sources related to the power industry downloaded in a certain area and the proportion of each category in the power-related information of the data source can be analyzed. According to the statistics of data sources in the regional distribution module, the data classification proportion module and the data download record module, a professional database prediction system can be formed, thereby making predictive analysis and intelligent decision-making for power dispatching.

Information crawling time identification module: used to identify the network access time of the crawled data source, so as to detect the initial effective time of the data source crawled by the data crawling module;

The latest information push module is used to automatically push the latest information related to the power industry to the Web page. The information crawling time identification module cooperates with the professional data information database to intelligently push the latest power industry information to the Web page through the latest information push module.

In the data crawling module, the specific process of data crawling includes the following processes:

S1. Initialize the URL sequence to be crawled through Focused Crawler software development;

S2, using the URL sequence to be crawled to search for power industry websites and multimedia pages;

S3. When the search page is empty, the data crawling ends. When a page related to the power industry is found, the page content corresponding to the URL is downloaded.

S4. When the page is downloaded successfully, add the URL to the crawled list and continue to analyze and process the document. When the page fails to download, choose whether to retry. If retry is not required, add the URL to the crawled list and add relevant exception handling and log records. If retry is required, repeat S2-S3.

S5. When adding the URL to the crawled list, choose whether to collect the page. If the page needs to be collected, the page will be taken as valid data and stored in the database module;

S6. When collection is not needed, determine that the URL is a valuable URL, and continue to repeat S2-S5.

The data sources obtained in the data crawling module include power industry policies, papers, achievements, technologies, R&D directions, products, and planning information.

The data categories in the data classification proportion module include standards, documents, software copyrights, and patents.

The regional distribution module, the data classification proportion module and the data download record module are connected in pairs.

There are multiple web pages, and the web pages are connected by a network.

When in use, first use the Focused Crawler software development, and use the data crawling module to obtain and collect professional information related to the power industry through power industry websites, multimedia platforms and other network channels, including pictures, text, links, etc., and collect effective information on policies, methods, technologies, power network security, power grid development, technological innovation, products, etc. related to power dispatching.

The information collected by the data crawling module is stored and managed, and a database storage module is established. At the same time, through the intelligent management of the MySQL database, the database management module is used to process the data source in the database storage module, filter out invalid information in the data source, remove duplications and correct errors, extract labels, and safely store the data source in the database storage module and manage its use reasonably and effectively.

Based on the Python network, a large amount of professional information in the database storage module is used for data analysis and big data research. At the same time, the Sharksearch algorithm is used to effectively enhance the relevance of the collected information and the subject, so as to classify the data sources in the database storage module.

According to the data sources in the database storage module after classification, and by matching the information data through regular expressions, a powerful internal professional system information base, namely the professional data information base, is established, and the index is generated by the index generation unit.

The data sources in the professional data information library can be searched through the web page, and the search optimization module can be used to optimize the display of the retrieved information to facilitate retrieval and download application.

The data statistics module establishes a power dispatching information visualization analysis platform based on the data source in the professional data information library. The power dispatching information visualization platform includes a regional distribution module, a data classification proportion module, and a data download record module. Through the regional distribution module and the data classification proportion module, the number of data sources related to the power industry in a certain region and the proportion of each category in the power-related information of the data source can be analyzed. Through the regional distribution module, the data download record module and the data classification proportion module, the number of data sources related to the power industry downloaded in a certain region and the proportion of each category in the power-related information of the data source can be analyzed.

According to the statistics of data sources in the regional distribution module, data classification proportion module and data download record module, a professional database prediction system can be formed to make predictive analysis and intelligent decision-making for power dispatching.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims (6)

1. An artificial intelligence power scheduling decision system based on big data, which is characterized in that: comprising the following steps:

And the data crawling module: utilizing Focused Crawler software development to search and obtain data sources related to the power industry from domestic and foreign industry websites and multimedia;

And a database storage module: the data source is used for storing the data acquired by the data crawling module;

And a database management module: processing the data source in the database storage module by utilizing intelligent management of the MySQL database, wherein the processing comprises useless information filtering, duplicate removal and error correction, tag extraction and data security management;

And the data processing and analyzing module is used for: analyzing and processing the data in the database storage module based on the Python network, and simultaneously combining SHARKSEARCH algorithm to analyze the correlation between the data information and the subject, so as to complete the classification processing of the data sources in the database storage module;

Professional data information base: classifying the data through a data processing analysis module, matching information data through a regular expression, and establishing a professional data information base related to the power industry for storing and managing the classified data sources;

an index generation unit: the index is generated according to the data sources classified in the professional data information base;

Web page: the key word information to be searched is input into the professional data information base;

and (5) a retrieval optimization module: the method is used for optimizing and recommending the information retrieved by the Web page;

and a data statistics module: the system is used for carrying out statistical analysis on data sources collected in the professional data information base;

And the power scheduling information visualization platform comprises: according to statistics of data sources in the professional data module by the data statistics module, a power scheduling information visualization platform is established, wherein the power scheduling information visualization platform internally comprises a region distribution module, a data classification duty ratio module and a data downloading recording module;

Region distribution module: statistics of data sources for each region;

And the data classification duty ratio module is used for: the data source classifying and counting device is used for classifying and counting the data sources according to the types of the data source classification;

and the data downloading and recording module is used for: the method is used for recording and counting the situation of the data source downloaded from the professional information base;

information crawling time identification module: the method comprises the steps of identifying the network access time of a crawled data source;

the latest information pushing module: for automatically pushing newly released power industry related information to the Web page.

2. The big data based artificial intelligence power scheduling decision system of claim 1, wherein: in the data crawling module, the specific process of data crawling comprises the following steps:

S1, initializing a URL sequence to be crawled through Focused Crawler software development;

S2, searching a website and a multimedia page of the power industry by utilizing the URL sequence to be crawled;

S3, when the search page is empty, finishing data crawling, and when the page related to the power industry is searched, downloading page content corresponding to the URL;

S4, when the page downloading is successful, adding the URL into the crawled list, continuously analyzing and processing the document, when the page downloading fails, selecting whether to retry, when the retry is not needed, adding the URL into the crawled list, adding related exception handling and log records, and when the retry is needed, repeating S2-S3;

S5, selecting whether to collect pages when the URL is added into the crawled list, and taking the pages as effective data and storing the effective data into a database module when the pages need to be collected;

and S6, when the collection is not needed, judging the URL to be a valuable URL, and continuously repeating the steps S2-S5.

3. The big data based artificial intelligence power scheduling decision system of claim 1, wherein: the data sources acquired in the data crawling module comprise policies, papers, achievements, technologies, research and development directions, products and planning information of the power industry.

4. The big data based artificial intelligence power scheduling decision system of claim 1, wherein: the data classification duty ratio module is used for classifying data, wherein the data classification duty ratio module comprises standards, documents, books and patents.

5. The big data based artificial intelligence power scheduling decision system of claim 1, wherein: the region distribution module, the data classification duty ratio module and the data downloading recording module are connected in pairs.

6. The big data based artificial intelligence power scheduling decision system of claim 1, wherein: the number of the Web pages is multiple, and network connection is arranged among the Web pages.