CN114706924A - Data acquisition master station system of loop state inspection instrument for special transformer user - Google Patents

Abstract

The invention relates to a main station system for data acquisition of a loop state inspection instrument for dedicated users, comprising: a collection file management module, a task scheduling module, a pre-collection module, a Mqtt Broker module, a loop state inspection instrument module and data Inbound module. The invention realizes the bidirectional transmission based on the MQTT network protocol and the data of the loop state inspection instrument by transforming the original acquisition master station system and adding the Mqtt Broker program on the server side, and then realizes the collection, analysis and input of the special user data. library.

Description

Data acquisition master station system of loop state inspection instrument for special transformer user

Technical Field

The invention belongs to the technical field of loop state inspection instruments and acquisition master stations, relates to a data acquisition master station system, and particularly relates to a data acquisition master station system for a special transformer user loop state inspection instrument.

Background

The loop state inspection instrument is modified on the basis of the traditional inspection instrument, can realize high-frequency acquisition of load identification data of a special transformer user, and reports the load identification data to an acquisition master station system. The loop state inspection instrument is improved on the basis of the traditional inspection instrument, integrates an MQTT module and realizes data transmission based on an MQTT protocol. Therefore, in order to realize data acquisition of the loop state inspection instrument, the original acquisition communication scheme needs to be modified.

The current power consumption information acquisition system is because the framework system that adopts traditional monomer formula, the simple operation is experienced poorly, the computing power is not enough, the business demand response is poor, can't support the high-speed mass data of growing of special transformer user and put the requirement in storage in real time, can't be for satisfying the requirement scheduling problem of special transformer user data acquisition, the current collection system of deep analysis is not enough to await urgent, upgrade the transformation to current power consumption information acquisition system, realize special transformer user power consumption information acquisition system overall structure, application property, data processing efficiency promotes.

The data overall structure who is used for special change user's return circuit state to patrol and examine appearance at present adopts Mqtt +698 (towards object oriented protocol) mode, does not pass through any collection equipment, can directly communicate with main website system, and network protocol, communication mode all change, consequently need reform transform original collection main website system, just can realize the collection of novel return circuit state patrol and examine appearance to special change user data.

Upon search, no prior art documents that are the same or similar to the present invention have been found.

Disclosure of Invention

The invention aims to overcome the defects of the prior art, and provides a data acquisition master station system of a loop state polling instrument for a special transformer user.

The invention solves the practical problem by adopting the following technical scheme:

a loop state patrols and examines appearance data acquisition master station system for special transformer user, includes: the system comprises an acquisition file management module, a task scheduling module, an acquisition front module, an Mqtt breaker module, a loop state patrol instrument module and a data entry module;

the acquisition file management module is used for storing the acquisition files of the special transformer users, and the output end of the acquisition file management module is connected with the task scheduling module and used for sending the information of the acquisition files of the special transformer users to the task scheduling module; the input end of the special transformer is connected with the data warehousing module and used for receiving and verifying the acquisition result data sent by the data warehousing module and storing the acquisition result data as a special transformer user acquisition file; the task scheduling module is used for generating, scheduling and sending acquisition tasks, the output end of the task scheduling module is connected with the acquisition front-end module, and the task scheduling module is used for generating acquisition tasks based on the file information acquired by the special transformer user and sent by the acquisition file management module and sending acquisition task strings of the scheduled acquisition tasks to the acquisition front-end module; the acquisition preposition module is used for assembling and sending an acquisition task 698 message and receiving and forwarding acquisition result data, and the output end of the acquisition preposition module is connected with the data warehousing module and used for forwarding the received acquisition result data to the data warehousing module for warehousing; the acquisition preposing module is also connected with the Mqtt Broker module and is used for sending an acquisition task 698 message to the Mqtt Broker module and receiving acquisition result data sent by the Mqtt Broker module; and the Mqtt Broker module and the loop state polling instrument module are used for acquiring the electricity utilization information of the special transformer user and returning an acquired data result.

Moreover, the acquisition file management module comprises an acquisition file checking module and an acquisition file storage module; the acquisition file checking module is used for checking the correctness of the acquisition result data sent by the data warehousing module, and the output end of the acquisition file checking module is connected with the acquisition file storage module and used for sending the acquired files of the special transformer users passing the checking to the acquisition file storage module for storage.

Moreover, the task scheduling module comprises an acquisition task generating module and an acquisition task scheduling module; and the acquisition task generating module is used for generating a power utilization information acquisition task for the special transformer user, and the output end of the acquisition task generating module is connected with the acquisition task scheduling module and is used for scheduling and sending the acquisition task to the acquisition front-end module.

The data warehousing module comprises a data analysis module and a warehousing module and is used for analyzing the acquired data result received by the acquisition front-end module and warehousing the acquired data result into the acquisition file management module for storage; the data analysis module is used for analyzing the acquired data result sent to the data storage module by the acquisition front-end module into a standard data format from a 698 message format; the output end of the storage module is connected with the storage module, the storage module is used for storing the analyzed acquired data results and generating a data storage queue, and the data storage queue is stored in an acquisition system real-time library of the acquisition file storage module after the data storage queue is verified by the acquisition file verification module.

The invention has the advantages and beneficial effects that:

1. the invention provides a loop state polling instrument data acquisition master station system for a special transformer user, which is characterized in that the original acquisition master station system is modified, a Server end Mqtt Broker program is additionally arranged, the bidirectional transmission of loop state polling instrument data for the special transformer user based on an MQTT network protocol is realized, and the acquisition, analysis and storage of the special transformer user data are further realized. The invention realizes high-frequency acquisition, analysis and high-speed warehousing of the special user data of the circuit state-based inspection instrument through a plurality of links such as file synchronization, task generation, task scheduling and sending, message assembly, message issuing (main station) -inspection instrument), message uploading (inspection instrument) -main station), message analysis, message warehousing and the like.

2. The invention can realize the high-frequency data acquisition of the circuit state polling instrument of the special transformer user. Based on new technologies such as 'big cloud thing moves intelligence', a set of 'safe, controllable and flexibly expanded' loop state inspection instrument data acquisition master station system for the special transformer user is built to meet the data acquisition requirements of the special transformer user. An Mqtt Broker transfer module is added to realize the decoupling connection of the acquisition master station system and the loop state inspection instrument, and the acquisition task and the acquisition result data are quickly transmitted in a packaged 698 message format based on an MQTT network protocol, so that the high-frequency acquisition of the loop state inspection instrument data of a special transformer user is realized.

3. The invention can realize high-speed data storage. The data warehousing module develops functions of data warehousing diversification processing, data batch writing and directional writing, customizes personalized warehousing strategies for different data warehousing, realizes data batch warehousing and directional writing, and greatly improves the data warehousing speed.

Drawings

Fig. 1 is a schematic diagram of a system for acquiring the load identification data of a specific transformer user based on a loop state inspection instrument.

Detailed Description

The embodiments of the invention will be described in further detail below with reference to the accompanying drawings:

a loop status data collection master station system for a dedicated transformer user, as shown in fig. 1, comprising: the system comprises a collection file management module, a task scheduling module, a collection preposition module, an Mqtt Broker module, a loop state inspection instrument module, a data warehousing module and the like;

the system can realize high-frequency acquisition, analysis and high-speed warehousing of the special user data of the circuit state inspection instrument through a plurality of links such as file synchronization, task generation, task scheduling and sending, message assembly, message issuing (main station) -inspection instrument), message uploading (inspection instrument) -main station), message analysis and message warehousing.

The acquisition file management module is used for storing the acquisition files of the special transformer users, and the output end of the acquisition file management module is connected with the task scheduling module and used for sending the information of the acquisition files of the special transformer users to the task scheduling module; the input end of the special transformer is connected with the data warehousing module and used for receiving and verifying the acquisition result data sent by the data warehousing module and storing the acquisition result data as a special transformer user acquisition file; the task scheduling module is used for generating, scheduling and sending acquisition tasks, the output end of the task scheduling module is connected with the acquisition front-end module, and the task scheduling module is used for generating acquisition tasks based on the file information acquired by the special transformer user and sent by the acquisition file management module and sending acquisition task strings of the scheduled acquisition tasks to the acquisition front-end module; the acquisition preposition module is used for assembling and sending an acquisition task 698 message and receiving and forwarding acquisition result data, and the output end of the acquisition preposition module is connected with the data warehousing module and used for forwarding the received acquisition result data to the data warehousing module for warehousing; the acquisition preposing module is also connected with the Mqtt Broker module and is used for sending an acquisition task 698 message to the Mqtt Broker module and receiving acquisition result data sent by the Mqtt Broker module; and the Mqtt Broker module and the loop state polling instrument module are used for acquiring the electricity utilization information of the special transformer user and returning an acquired data result.

In this embodiment, the collected archive management module includes a collected archive verification module and a collected archive storage module; the acquisition file checking module is used for checking the correctness of the acquisition result data sent by the data warehousing module, and the output end of the acquisition file checking module is connected with the acquisition file storage module and used for sending the acquired files of the special transformer users passing the checking to the acquisition file storage module for storage.

In this embodiment, the task scheduling module includes an acquisition task generating module and an acquisition task scheduling module; and the acquisition task generating module is used for generating a power utilization information acquisition task for the special transformer user, and the output end of the acquisition task generating module is connected with the acquisition task scheduling module and is used for scheduling and sending the acquisition task to the acquisition front-end module.

In this embodiment, the data warehousing module includes a data analysis module and a warehousing module, and is configured to analyze the data acquisition result received by the acquisition front-end module and warehouse the data acquisition result into the acquisition archive management module for storage; the data analysis module is used for analyzing the acquired data result sent to the data storage module by the acquisition front-end module into a standard data format from a 698 message format; the output end of the storage module is connected with the storage module, the storage module is used for storing the analyzed acquired data results and generating a data storage queue, and the data storage queue is stored in an acquisition system real-time library of the acquisition file storage module after the data storage queue is verified by the acquisition file verification module.

The functions and functions of the various modules in the system are further described below:

the acquisition file management module is used for storing the acquisition files of the special transformer users, and comprises two sub-modules of acquisition file verification and acquisition file storage; the output end of the acquisition file management module is connected with the task scheduling module and used for sending the file information acquired by the special transformer user to the task scheduling module. The input end of the acquisition file management module is connected with the data warehousing module and used for receiving and verifying acquisition result data sent by the data warehousing module and storing the acquisition result data into a special transformer user acquisition file.

The acquisition file checking module is used for checking the correctness and logicality of the acquisition files of the special transformer user, such as non-null, key field values, logic accuracy and the like; the input end of the acquisition file checking module is connected with the data warehousing module and is used for checking the correctness of the acquisition result data sent by the data warehousing module; the output end of the acquisition file checking module is connected with the acquisition file storage module and used for sending the acquisition files of the special transformer users passing the checking to the acquisition file storage module for storage.

The acquisition file storage module is used for storing the acquisition file information of the special transformer user; the input end of the acquisition file storage module is connected with the acquisition file calibration module and used for storing the acquisition file information of the special transformer user calibrated by the acquisition file calibration module. The output end of the acquisition file storage module is connected with the task scheduling module and used for sending the corresponding special transformer user acquisition files to the task scheduling module.

The task scheduling module is used for generating, scheduling and sending the acquisition tasks and comprises two sub-modules of acquisition task generation and acquisition task scheduling; the input end of the task scheduling module is connected with the acquisition file management module and is used for generating an acquisition task based on file information acquired by a special transformer user and sent by the acquisition file management module; the output end of the task scheduling module is connected with the acquisition front module and used for sending the acquisition task string of the acquisition task which is scheduled to be finished to the acquisition front module.

The acquisition task generating module is used for generating a power utilization information acquisition task for the special transformer user; the input end of the acquisition task generation module is connected with the acquisition file management module and is used for generating an acquisition task based on file information acquired by a special transformer user and sent by the acquisition file management module; the output end of the collection task generation module is connected with the collection task scheduling module and used for sending the generated collection tasks to the collection task scheduling module;

the acquisition task scheduling module is used for scheduling and sending acquisition tasks; the input end of the acquisition task scheduling module is connected with the acquisition task generating module and used for scheduling the acquisition tasks generated by the acquisition task generating module and generating an acquisition task string; the output end of the collection task scheduling module is connected with the collection front module and used for sending the collection task string to the collection front module.

The acquisition front-end module is used for assembling and sending an acquisition task 698 message and receiving and forwarding acquisition result data; the input end of the acquisition front module is connected with the task scheduling module and is used for assembling the acquisition task strings sent by the acquisition front module into 698 messages; the output end of the acquisition preposition module is connected with the Mqtt Broker module and is used for sending an acquisition task 698 message to the Mqtt Broker module; the input end of the acquisition preposing module is connected with the Mqtt Broker module and is used for receiving acquisition result data sent by the Mqtt Broker module; the output end of the acquisition preposing module is connected with the data warehousing module and used for forwarding the received acquisition result data to the data warehousing module for warehousing.

The Mqtt Broker module is a transfer module and is used for acquiring data connection between the front module and the loop state inspection instrument; the input end of the Mqtt Broker module is connected with the acquisition front module and is used for receiving an acquisition task 698 message sent by the acquisition front module; the output end of the Mqtt Broker module is connected with the loop state inspection instrument module and is used for forwarding the collection task 698 message to the corresponding loop state inspection instrument module; the input end of the Mqtt Broker module is connected with the loop state polling instrument module and is used for receiving the acquisition result data returned by the loop state polling instrument module; the output end of the Mqtt Broker module is connected with the acquisition front module and used for forwarding the received acquisition result data to the acquisition front module.

The loop state inspection instrument module is used for acquiring the power utilization information of the special transformer user and returning an acquired data result; the input end of the loop state polling instrument module is connected with the Mqtt Broker module and is used for receiving a task 698 message sent by the Mqtt Broker module and acquiring power consumption information data of a corresponding special transformer user according to the 698 message; the output end of the loop state patrol instrument module is connected with the Mqtt Broker module and used for transmitting the power utilization information data acquisition result data of the special transformer user back to the Mqtt Broker module.

The data warehousing module is used for analyzing the acquired data result received by the acquisition front-end module and warehousing the acquired data result into the acquisition file management module for storage, and comprises two sub-modules of data analysis and warehousing; the input end of the data storage module is connected with the acquisition front module and used for analyzing the received acquisition result data sent by the acquisition front module; the output end of the data warehousing module is connected with the acquisition file management module and used for sending the analyzed acquisition result data to the acquisition file management module for storage according to the warehousing logic of the data.

The acquisition file management module is used for checking and storing the acquisition files of the special transformer users and comprises two sub-modules of acquisition file checking and acquisition file storage; the output end of the acquisition file management module is connected with the task scheduling module and used for sending the file information acquired by the special transformer user to the task scheduling module.

The collection file checking module is used for checking the integrity, logic accuracy and service accuracy of the collection system files, and can check the collection files in the aspects of non-null, key field values, logic accuracy and the like through a file checking and checking function. The output end of the acquisition file checking module is connected with the acquisition file storage module and is used for acquiring file information of the special transformer user passing the checking

The acquisition file storage module is mainly used for storing the acquisition files of the special transformer users which are verified by the acquisition file verification module to be correct. And respectively storing the acquired archive information to two databases, namely a historical database of the acquisition system and a real-time database of the acquisition system, by setting a Redis cache mechanism. And caching the file content, including a terminal file, a main acquisition and supplementary acquisition task file, a protocol data item uplink and downlink mapping file, a task template file, an acquisition strategy file, a data dictionary file and the like. The above is the acquisition file management module.

And the task scheduling module. The method is used for generating, scheduling and sending the collection tasks. The method comprises two submodules of data acquisition task generation and task scheduling.

And the acquisition task generation module is used for configuring the data acquisition task of the loop state inspection instrument by the acquisition master station system and generating the acquisition task. The method comprises the steps of acquisition task configuration, acquisition task template generation, matching and issuing, personalized data acquisition strategy customization and the like. Generating an acquisition task queue based on the acquired file information in the acquisition system historical library, and flexibly configuring the acquisition period, frequency and reporting reference time of the data acquisition task by an acquisition system master station; the acquisition task template generation module has a visualization function, and can generate and automatically match acquisition task templates to set and issue according to different terminal types, local communication modes and acquisition meter types; and customizing a personalized data acquisition strategy aiming at each type of acquisition object, realizing multi-source heterogeneous data acquisition and hierarchical acquisition according to needs, and supporting a user to respectively perform online configuration on a terminal acquisition scheme and a system acquisition scheme.

The acquisition task scheduling module is used for flexibly configuring the generated acquisition tasks, triggering the acquisition tasks in a timing and real-time manner, generating acquisition task strings aiming at the triggered acquisition tasks and sending the acquisition task strings to the acquisition front-end module. The task scheduling module provides a friendly operation mode for acquisition, operation and maintenance and manual triggering and additional acquisition; in order to save memory occupation, the method supports the completion of the execution of a special task, namely quitting, and automatically schedules and starts at regular time; and generating a collection task string for the triggered collection task, and sending the collection task string to a collection front module. The above is a task scheduling module.

The acquisition preposing module is used for assembling 698 messages with the received acquisition task strings, sending the messages to the Mqtt Broker module, receiving acquisition result data returned by the Mqtt Broker module, and sending the data to the data storage module. A task integration mechanism (698 protocol is adopted to assemble the collection task strings into 698 messages according to the same collection object, the same time dimension and the same freezing data collection task integration mechanism) in the communication protocol, so that the number of issued tasks and the communication flow are saved; and receiving the acquisition result data returned by the Mqtt Broker module, and sending the acquisition result data to the data storage module. The above is the acquisition front module.

The Mqtt Broker module is a transfer module and is used for data connection between the acquisition front module and the loop state patrol instrument. The Mqtt Broker server program receives the collection task 698 message sent by the collection preposed module, forwards the message to the loop state inspection instrument, and sends the collection result data returned by the loop state inspection instrument to the collection preposed module, so that the decoupling connection between the collection master station system and the loop state inspection instrument is realized. The above is the Mqtt Broker module.

And the loop state inspection instrument module is used for acquiring the electricity utilization information of the special transformer user and returning an acquired data result. And 4, the loop state patrol instrument acquires a collection task 698 message sent by the collection front module through the Mqtt Broker module in the step 4, executes a collection task after verification, and returns collection result data to the Mqtt Broker module. The above is a loop state inspection instrument module.

And the data warehousing module is used for analyzing the acquired data result received by the acquisition preposing module and warehousing the acquired data result into the acquisition file management module for storage, and comprises two sub-modules of data analysis and warehousing.

And the data analysis module is used for analyzing the acquired data result sent to the data storage module by the acquisition front-end module into a standard data format from a 698 message format.

And the warehousing module is used for warehousing the analyzed acquired data results and generating a data warehousing queue, and storing the data warehousing queue to an acquisition system real-time warehouse of the acquisition file storage module after the data warehousing queue is verified by the acquisition file verification module. The storage module comprises functions of data storage diversification processing, data batch writing and directional writing.

The data storage diversified processing function is used for ensuring the timeliness of data storage. The event data and the general data are stripped in a storage queue and are directly accessed into a historical library, so that the query and use of each application function at an application side are facilitated, and the event data storage simply adopts insert operation. Other data are unified to be put in storage along with the real-time storage and storage program of the acquisition system, so that the condition that the data writing table is locked is avoided. And writing the consumption data into a database side data warehousing advanced queue according to a standard data format, configuring advanced queue data by the database, and taking charge of warehousing data in the consumption queue.

The data batch writing function is used for achieving batch data storage and improving data storage speed. A transaction-level memory Temp table is additionally designed for writing data in batches, and the data storage speed is increased.

And the oriented writing function is used for analyzing the data submitted in batches and directionally writing the data into the data table. And developing a service code for consuming Temp table data, analyzing the data submitted in batches and writing the data into a specified data table.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Claims (4)

1. a loop state inspection instrument data acquisition master station system for changing users, it is characterized in that: comprising: collection file management module, task scheduling module, collection front module, Mqtt Broker module, loop state inspection instrument modules and data warehousing modules; The collection file management module is used to store the special change user collection file, and its output end is connected with the task scheduling module, and is used for sending the special change user collection file information to the task scheduling module; its input end is connected with the data storage module. The connection is used to receive and verify the collection result data sent by the data storage module, and store it as a dedicated user collection file; the task scheduling module is used to generate, schedule and send collection tasks, and its output is connected to the collection pre- The modules are connected to each other, and are used to generate collection tasks based on the special-changing user collection file information sent by the collection file management module, and send the collection task string of the scheduled collection tasks to the collection pre-module; the collection pre-module is used for assembling, Send the collection task 698 message and receive and forward the collection result data, and its output end is connected to the data storage module, and is used to forward the received collection result data to the data storage module for storage; the collection pre-module is also connected with the data storage module. The Mqtt Broker module is connected to send the collection task 698 message to the Mqtt Broker module, and receives the collection result data sent by the Mqtt Broker module; the Mqtt Broker module is also connected to the loop status inspection instrument module for collecting special changes User electricity consumption information, and return the collected data results. 2. a kind of loop state inspection instrument data acquisition master station system according to claim 1, is characterized in that: described acquisition archives management module comprises acquisition archives verification module and acquisition archives storage module; The collection file verification module is used for verifying the correctness of the collection result data sent by the data storage module, and its output end is connected with the collection file storage module, and is used for sending the user collection files that have passed the verification to the collection file. File storage module for storage. 3. The data collection master station system of a loop state inspection instrument for dedicated users according to claim 1, wherein the task scheduling module comprises a collection task generation module and an collection task scheduling module; the collection task The generation module is used to generate the electricity consumption information collection task of the specific change user, and its output end is connected with the collection task scheduling module, and is used for scheduling and sending the collection task to the collection front-end module. 4. The data acquisition master station system of a loop state inspection instrument for dedicated users according to claim 1, wherein the data storage module comprises a data analysis module and a storage module, which are used for The collection data results received by the collection pre-module are analyzed and stored in the collection file management module for storage; the data analysis module is used to send the collection pre-module to the collection data results of the data storage module, The 698 message format is parsed into a standard data format; its output end is connected to the warehousing module, which is used to store the parsed collection data results and generate a data warehousing queue, which is checked by the collection file verification module. After the verification is completed, it is stored in the real-time library of the acquisition system of the acquisition file storage module.

Images