CN114286203A - A system and method for shunting and processing electricity meter check and receipt data based on big data - Google Patents
A system and method for shunting and processing electricity meter check and receipt data based on big data Download PDFInfo
- Publication number
- CN114286203A CN114286203A CN202111581964.6A CN202111581964A CN114286203A CN 114286203 A CN114286203 A CN 114286203A CN 202111581964 A CN202111581964 A CN 202111581964A CN 114286203 A CN114286203 A CN 114286203A
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- response rate
- real
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
本发明提供一种基于大数据的电表核收数据分流处理方法，包括：步骤S10，周期性地采集各类电表相关设备的运行数据；步骤S11，对采集的运行数据进行预处理，并将每一运行数据标准化；步骤S12，将预处理后的所有数据进行数据类别划分，分别导入多个数据处理通道中；步骤S13，根据系统实时响应率，确定是否需要对各通道的数据进行分流处理；步骤S14，在需要分流时，则将需要分流的数据分流至相应的并行服务器中；否则，将数据传送给主服务器中；步骤S15，主服务器或并行服务器对数据进行整合分析以及核对处理，形成各用户的用户数据包。本发明还公开了相应的系统。实施本发明，可以提升抄表效率，缩短电表数据的采集核收的整体时间。
The present invention provides a big data-based method for shunting and processing electric meter check and receipt data, comprising: step S10, periodically collecting operation data of various types of electric meter related equipment; step S11, preprocessing the collected operation data, and distributing each 1. Standardization of running data; Step S12, classify all the preprocessed data into data categories and import them into multiple data processing channels respectively; Step S13, according to the real-time response rate of the system, determine whether it is necessary to perform shunting processing on the data of each channel; Step S14, when it is necessary to split the data, the data that needs to be split is split to the corresponding parallel server; otherwise, the data is transmitted to the main server; Step S15, the main server or the parallel server performs integrated analysis and verification processing on the data to form User data package for each user. The invention also discloses a corresponding system. By implementing the present invention, the efficiency of meter reading can be improved, and the overall time for collecting and checking the meter data can be shortened.
Description
Technical Field
The invention relates to the technical field of electric meter verification and collection, in particular to an electric meter verification and collection data shunting processing system and method based on big data.
Background
With the vigorous development of digital technologies and applications such as 'cloud big thing moving intelligence', power enterprises in recent years continuously perfect a power marketing system on the premise of ensuring power stability. The electricity charge reading, checking and receiving work is the key part work of an electric power enterprise, the meter reading is the data acquisition link of a reading, checking and receiving system, and the data acquisition link is the basis of the following two works. In the prior art, meter reading in a reading-checking-receiving system is realized by a task uploading and data complementary collection mode, but a hundred thousand-level terminal device is simultaneously connected with a main station front-end processor, so that pressure is brought to the connection efficiency of the front-end processor, and in the transmission process after data collection, data need to pass through a safety access area and reach a subsequent device through two isolation devices. The existing integral meter reading time is long, the data acquisition and data transmission have an optimized space, and the meter reading efficiency is low.
Disclosure of Invention
The technical problem to be solved by the invention is to provide the ammeter data collecting and receiving shunting processing system and method based on the big data, which can improve the meter reading efficiency and shorten the overall time for collecting and receiving ammeter data.
In order to solve the above technical problems, an aspect of the present invention provides a method for shunting data received by an electric meter core based on big data, which includes the following steps:
step S10, periodically collecting the operation data of the related equipment of various electric meters through the front-end processor;
step S11, preprocessing the collected operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data;
step S12, determining the data type of each preprocessed data, dividing all the preprocessed data according to the data type, and respectively importing the data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
step S13, determining whether the data of each data processing channel needs to be distributed according to the real-time response rate of the system;
step S14, if the judgment result is that the data needs to be distributed, distributing the data of the data processing channel needing to be distributed to one of a plurality of parallel servers connected with the main server; otherwise, transmitting the data of the data processing channel to a main server;
step S15, after receiving the data of the data processing channel, the main server or the parallel server carries out integration analysis and check processing on the data to form a user data packet of each user; and pushing the user data packet to a corresponding user side.
Preferably, further comprising: the method comprises the steps of configuring the corresponding relation between parallel servers and data categories in advance, wherein each parallel server corresponds to one data category.
Preferably, the step S13 further includes:
comparing the real-time response rate of the system with a preset response rate threshold, and if the real-time response rate of the system is smaller than the response threshold, determining that shunting processing is required; otherwise, determining that the shunting processing is not needed;
if the data needs to be distributed, inquiring a corresponding table of response rate and distribution quantity according to the real-time response rate, determining the quantity of the data processing channels needing to be distributed, and obtaining the parallel server information corresponding to each data processing channel so as to distribute the data of each data processing channel needing to be distributed to the corresponding parallel server.
Correspondingly, in another aspect of the present invention, a big data based electric meter nuclear receiving data shunting processing system is further provided, which includes:
the data acquisition unit is used for periodically acquiring the operation data of various types of electric meter related equipment through the front-end processor;
the preprocessing unit is used for preprocessing the acquired operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data;
the data dividing unit is used for determining the data type of each preprocessed data, dividing all the preprocessed data according to the data type and respectively importing the data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
the shunt judgment unit is used for determining whether the data of each data processing channel needs to be shunted or not according to the real-time response rate of the system;
the distribution processing unit is used for distributing the data of the data processing channel needing distribution to one of a plurality of parallel servers connected with the main server if the judgment result of the distribution judging unit is that the distribution is needed; otherwise, transmitting the data of the data processing channel to a main server;
the checking processing unit is used for performing integration analysis and checking processing on the data after the main server or the parallel server receives the data of the data processing channel to form a user data packet of each user; and pushing the user data packet to a corresponding user side.
Preferably, further comprising:
the configuration unit is used for configuring the corresponding relation between the parallel servers and the data categories in advance, wherein each parallel server corresponds to one data category.
Preferably, the shunt judgment unit further includes:
the comparison unit is used for comparing the real-time response rate of the system with a preset response rate threshold value, and if the real-time response rate of the system is smaller than the response threshold value, the shunt processing is determined to be required; otherwise, determining that the shunting processing is not needed;
and the flow dividing channel number determining unit is used for inquiring a corresponding table of response rate and flow dividing number according to the real-time response rate when the comparing unit determines that flow dividing processing is needed, determining the number of the data processing channels needing flow dividing, and acquiring the parallel server information corresponding to each data processing channel.
The embodiment of the invention has the following beneficial effects:
the invention provides a big data-based ammeter nuclear receiving data shunting processing system and method, which can enable different servers to process different types of ammeter data in parallel by shunting the collected massive ammeter data, thereby being capable of rapidly performing nuclear receiving processing on the ammeter data and greatly improving the overall data processing efficiency; the whole time for collecting and checking the ammeter data is greatly shortened.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is within the scope of the present invention for those skilled in the art to obtain other drawings based on the drawings without inventive exercise.
Fig. 1 is a schematic main flow chart of an embodiment of a split-flow processing method for electric meter acceptance data based on big data according to the present invention;
fig. 2 is a schematic structural diagram of a big data-based electric meter verification data distribution processing system provided by the present invention;
fig. 3 is a schematic structural diagram of the flow determination unit in fig. 2.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings.
Fig. 1 is a main flow diagram illustrating an embodiment of a method for shunting data received by an electric meter based on big data according to the present invention; in the present embodiment, it is preferred that,
in order to solve the above technical problems, an aspect of the present invention provides a method for shunting data received by an electric meter core based on big data, which includes the following steps:
step S10, periodically collecting the operation data of the related equipment of various electric meters through the front-end processor; it can be understood that, due to different types of the related devices of the electric meter, the data formats acquired by the electric meter are different;
step S11, preprocessing the collected operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data; and controlling the length of the acquired data to realize data standardization. To facilitate processing in subsequent steps;
step S12, determining the data type of each preprocessed data, dividing all the preprocessed data according to the data type, and respectively importing the data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
step S13, determining whether the data of each data processing channel needs to be distributed according to the real-time response rate of the system;
specifically, in a specific example, preferably, the step S13 further includes:
comparing the real-time response rate of the system with a preset response rate threshold, and if the real-time response rate of the system is smaller than the response threshold (such as 50%), determining that shunting processing is required; otherwise, determining that the shunting processing is not needed;
if the data needs to be distributed, inquiring a corresponding table of response rate and distribution quantity according to the real-time response rate, determining the quantity of the data processing channels needing to be distributed, and obtaining the parallel server information corresponding to each data processing channel so as to distribute the data of each data processing channel needing to be distributed to the corresponding parallel server. Specifically, the lower the real-time response rate is, which indicates that the real-time processing efficiency of the existing system is lower, the more data of the data processing channels need to be distributed to the parallel server for processing; the relationship (including specific channels) between the real-time response rate and the shunt number in the response rate and shunt number correspondence table can be calibrated or configured in advance.
It can be understood that, in the method of the present invention, the correspondence between the parallel servers and the data categories needs to be configured in advance, wherein each parallel server corresponds to one data category.
Step S14, if the judgment result is that the data needs to be distributed, distributing the data of the data processing channel needing to be distributed to one of a plurality of parallel servers connected with the main server; otherwise, transmitting the data of the data processing channel to a main server;
step S15, after receiving the data of the data processing channel, the main server or the parallel server carries out integration analysis and check processing on the data to form a user data packet of each user; and pushing the user data packet to a corresponding user side. Specifically, in this step, at least the resources in the database may be configured and integrated according to a preset distribution scheme to obtain a data packet, and the data packet is distributed and pushed to the user side.
Fig. 2 is a schematic structural diagram illustrating an embodiment of a big data-based electric meter core data distribution processing system according to the present invention. Referring to fig. 3 together, in this embodiment, the system 1 at least includes:
the data acquisition unit 10 is used for periodically acquiring the operation data of various electric meter related devices through the front-end processor;
the preprocessing unit 11 is used for preprocessing the acquired operation data and standardizing each operation data; the pre-treatment at least comprises: repairing defective data, deleting useless data, and removing messy codes and repeated parts contained in the data;
the data dividing unit 12 is configured to determine a data type of each preprocessed data, divide all the preprocessed data according to the data type, and import the divided data into a plurality of data processing channels; each data processing channel corresponds to one category of data respectively;
the shunt judgment unit 13 is configured to determine whether data of each data processing channel needs to be shunted according to the system real-time response rate;
the distribution processing unit 14 is configured to distribute data of the data processing channel to be distributed to one of the multiple parallel servers connected to the main server if the determination result of the distribution determining unit is that distribution is required; otherwise, transmitting the data of the data processing channel to a main server;
a checking processing unit 15, configured to perform, after the main server or the parallel server receives data of the data processing channel, integration analysis and checking processing on the data to form a user data packet of each user; pushing the user data packet to a corresponding user side;
the configuration unit 16 is configured to pre-configure a corresponding relationship between parallel servers and data categories, where each parallel server corresponds to one data category.
As shown in fig. 3, the shunt determination unit 14 further includes:
the comparing unit 140 is configured to compare the system real-time response rate with a preset response rate threshold, and if the system real-time response rate is smaller than the response threshold, determine that the shunting processing is required; otherwise, determining that the shunting processing is not needed;
the shunting channel number determining unit 141 is configured to, when the comparing unit determines that shunting processing is required, query a response rate and shunting number correspondence table according to the real-time response rate, determine the number of data processing channels that need to be shunted, and obtain parallel server information corresponding to each data processing channel.
For more details, reference may be made to and combined with the foregoing description of fig. 1, which is not repeated herein.
The embodiment of the invention has the following beneficial effects:
the invention provides a big data-based ammeter nuclear receiving data shunting processing system and method, which can enable different servers to process different types of ammeter data in parallel by shunting the collected massive ammeter data, thereby being capable of rapidly performing nuclear receiving processing on the ammeter data and greatly improving the overall data processing efficiency; the whole time for collecting and checking the ammeter data is greatly shortened.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (6)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111581964.6A CN114286203B (en) | 2021-12-22 | 2021-12-22 | Electric meter nuclear data distribution processing system and method based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111581964.6A CN114286203B (en) | 2021-12-22 | 2021-12-22 | Electric meter nuclear data distribution processing system and method based on big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114286203A true CN114286203A (en) | 2022-04-05 |
| CN114286203B CN114286203B (en) | 2024-03-29 |
Family
ID=80873814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111581964.6A Active CN114286203B (en) | 2021-12-22 | 2021-12-22 | Electric meter nuclear data distribution processing system and method based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114286203B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673457A (en) * | 2009-08-14 | 2010-03-17 | 深圳市科陆电子科技股份有限公司 | Method and system of data acquisition from data acquisition terminal |
| CN102510568A (en) * | 2011-11-22 | 2012-06-20 | 联通宽带业务应用国家工程实验室有限公司 | Internet access data processing system and method for mobile terminal |
| CN103886735A (en) * | 2014-03-07 | 2014-06-25 | 北京国电通网络技术有限公司 | Acquisition method and acquisition system of power utilization information, and meter reading equipment |
| CN104112345A (en) * | 2014-07-29 | 2014-10-22 | 深圳华越南方电子技术有限公司 | Data monitoring method of remote wireless meter reading system |
| CN108563664A (en) * | 2018-01-04 | 2018-09-21 | 上海电气集团股份有限公司 | A kind of real-time data processing method at industrial equipment end |
| CN112650788A (en) * | 2020-12-25 | 2021-04-13 | 青岛鼎信通讯股份有限公司 | Power consumption behavior monitoring system based on non-invasive ammeter |
-
2021
- 2021-12-22 CN CN202111581964.6A patent/CN114286203B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673457A (en) * | 2009-08-14 | 2010-03-17 | 深圳市科陆电子科技股份有限公司 | Method and system of data acquisition from data acquisition terminal |
| CN102510568A (en) * | 2011-11-22 | 2012-06-20 | 联通宽带业务应用国家工程实验室有限公司 | Internet access data processing system and method for mobile terminal |
| CN103886735A (en) * | 2014-03-07 | 2014-06-25 | 北京国电通网络技术有限公司 | Acquisition method and acquisition system of power utilization information, and meter reading equipment |
| CN104112345A (en) * | 2014-07-29 | 2014-10-22 | 深圳华越南方电子技术有限公司 | Data monitoring method of remote wireless meter reading system |
| CN108563664A (en) * | 2018-01-04 | 2018-09-21 | 上海电气集团股份有限公司 | A kind of real-time data processing method at industrial equipment end |
| CN112650788A (en) * | 2020-12-25 | 2021-04-13 | 青岛鼎信通讯股份有限公司 | Power consumption behavior monitoring system based on non-invasive ammeter |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114286203B (en) | 2024-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6308211B1 (en) | Method and apparatus for identifying informational packets | |
| CN104283918B (en) | A kind of WLAN terminal type acquisition methods and system | |
| CN112327100B (en) | Power failure detection method and system based on Internet of things | |
| CN110046073B (en) | Log collection method and device, equipment and storage medium | |
| WO2020155508A1 (en) | Suspicious user screening method and apparatus, computer device and storage medium | |
| CN108712433A (en) | A kind of network security detection method and system | |
| CN106997487B (en) | An automatic interpretation method of aircraft test data based on knowledge base | |
| CN114979186B (en) | Flow link analysis method and system based on Flink component | |
| CN116074215A (en) | Network quality detection method, device, equipment and storage medium | |
| CN102055620B (en) | Method and system for monitoring user experience | |
| CN114286203A (en) | A system and method for shunting and processing electricity meter check and receipt data based on big data | |
| CN114172744A (en) | Data transmission system and data transmission method used among multiple servers | |
| CN112800140A (en) | A method for obtaining high-credibility data based on blockchain oracles | |
| CN108989116B (en) | Big data collection and analysis system and method | |
| CN107204892B (en) | Power communication network operation data processing method and device | |
| CN116578911A (en) | Data processing method, device, electronic device and computer storage medium | |
| CN115344633A (en) | Data processing method, device, equipment and storage medium | |
| CN111444156B (en) | Fault diagnosis method based on cloud computing | |
| CN113076355A (en) | Method for sensing data security flow situation | |
| CN114124509A (en) | Spark-based network abnormal flow detection method and system | |
| CN114385699B (en) | Power grid user quantitative price and cost anomaly analysis method | |
| CN110868732B (en) | VoLTE radio access failure problem positioning method, system and equipment | |
| CN112671593A (en) | Server management method and related equipment | |
| CN112765010A (en) | Method, device, equipment and storage medium for centralized management of service parameters | |
| CN104980750A (en) | Collection method, device and system for video transcoding logs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |