CN103532975A - Dynamically and smoothly expandable data acquisition system and method - Google Patents

Abstract

The invention discloses a data collection system and method capable of dynamic and smooth expansion, including: a data collection center system and a data collection terminal; the data collection center system includes a gateway routing node, a data collection management node, a data collection node, a security authentication node, Collect data storage nodes, monitoring nodes, time synchronization service nodes, security certificate storage nodes, log management nodes, log storage nodes, and the data collection center system communicates with the data collection terminal through the communication network, and the data collection terminal will be used to collect the data of the equipment Transfer to the data collection node, and submit the data collection application form to the gateway routing node. According to the data collection node routing information form returned by the gateway routing node, initiate a business report block transmission request with each data collection node for encrypted data transmission. The invention solves the problems of high expansion cost, poor expansion ability and safety risk of the existing data acquisition system under new application requirements.

Description

A data acquisition system and method capable of dynamic and smooth expansion

technical field

The invention relates to the field of data acquisition, in particular to a data acquisition system and method capable of dynamic and smooth expansion.

Background technique

With the development of computer technology, Internet of Things technology and the improvement of data communication infrastructure, society has entered the era of Internet of Things from the computer information age, and more and more on-site data acquisition devices appear in production and life. These devices include both traditional applications. Professional data acquisition equipment for industrial monitoring fields such as smart grids and automated production lines, as well as portable intelligent equipment with data acquisition functions for smart homes, environmental monitoring, and personal mobile applications. The traditional data acquisition system in the computer information age has developed and grown since the 1980s. It consists of computer equipment installed with data acquisition master station software, field bus and data acquisition equipment. The design goal of this type of system is to pass professional data from industrial sites Acquisition equipment realizes industrial monitoring data collection. In the era of the Internet of Things, the traditional mode of data acquisition system has shown more and more limitations in the face of the data transmission requirements of a large number and variety of Internet of Things acquisition devices, mainly reflected in:

1) In the traditional data acquisition system, the acquisition equipment is expanded by writing and configuring the protocol in the data acquisition master station software. It is only applicable to the field of industrial automation where the type and quantity of acquisition equipment are fixed and specific operations are performed.

When expanding access to new equipment in the traditional data acquisition system, first write the acquisition protocol for this type of equipment in the data acquisition master station software, and then perform communication configuration for each equipment of this type in the software system, including routing information , protocol information, and data point table information. This configuration process must be carried out by professionals who have a good understanding of the system, otherwise these configuration changes may affect the stability of the original system. This process needs to be repeated every time a new device is connected to the production site. Fortunately, the types and quantities of data acquisition equipment in the industrial production environment are strictly designed. Although the configuration workload of such equipment expansion will be very heavy during the system debugging stage, after the system is debugged and put into stable operation, Except for expansion or maintenance, there is less demand for new equipment. In this application, the traditional data acquisition system meets the requirements.

In application scenarios where the types and quantities of collection devices change frequently, such as multi-user-oriented smart home public data collection services, wide-area environmental dynamic monitoring, and electric vehicle mobile data collection, etc., due to the diversity of monitoring objects Due to the irregularity of operating conditions, the type and quantity of data acquisition equipment will change frequently. If a traditional data acquisition system is used, frequent configuration changes need to be made on the system that has been running stably. A huge amount of maintenance work is required, leaving hidden dangers to the stable operation of the system.

2) In the traditional data acquisition system, the software system of the data acquisition master station maintains the routing information from the central system to all the collected equipment, occupies a dominant position in the data collection process, and is responsible for establishing the transmission route with the collected equipment. A data collection scenario where there is fixed routing information from the collection center system to the collected device.

When collecting data on key equipment in the production process in the industrial automation production process, it is necessary to establish a dedicated data collection channel for the collected equipment. Establish routing record information. When the data acquisition center system starts to run, the software system of the data acquisition master station establishes routing connections with all data acquisition devices one by one according to the routing information stored by itself. This method of exclusive channel and route establishment by the data acquisition center system meets the requirements of large data transmission volume, high real-time and high stability of data transmission in industrial production.

In the field of Internet of Things data collection, a large number of Internet of Things data devices present discrete distribution characteristics and location mobility in geographical space. In reality, there is no condition for each device to be equipped with a dedicated and fixed data communication channel, that is, No fixed routing information. In the absence of a fixed route, the data acquisition center system cannot pre-establish a route matching information table for the collected device and initiate a route connection, that is, a new route establishment method needs to be adopted; Belonging to the attributes of the collected devices, the collection system needs to configure the same number of data routes according to the maximum number of collected devices that may be connected. In the traditional data collection system, it is a scene with a small number of collected devices. This method It is feasible, but in the field of the Internet of Things, it is necessary to face millions of devices. At the same time, the online status of the collection devices will change frequently, and the collection devices will frequently go online and offline according to the actual requirements or restrictions of the collection site. The static routing resource configuration method in the traditional data collection system cannot meet this frequently changing demand, and requires a new routing management method and dynamic configuration technology for available routing.

3) Traditional data acquisition systems are mostly used in industrial production environments, and their network environment is a private industrial field data acquisition network. This kind of industrial field network, which is physically isolated from public transmission channels and has an independent network, reduces the impact of data acquisition systems on external security. The risk of attack, in order to ensure the real-time performance of industrial applications, data is usually transmitted in plain text. Different from the independent network environment of professional acquisition equipment on the industrial site, the smart acquisition equipment of the Internet of Things is distributed in a large geographical range, and the data transmission network must rely on the public transmission network. The risk of security attacks on the system is very high. Traditional data acquisition systems and The method is not suitable for such scenarios with high security risks.

China's No. 200410021983.3 invention patent mainly relates to a design method for storing data according to a set threshold in an industrial site data acquisition system, which is suitable for achieving an effective balance between effective data volume and data restoration when storing dynamic environment data at the bottom. It can solve the problems of cumbersome configuration of new data acquisition equipment, routing dynamic management and on-demand configuration, and security protection proposed in this solution.

China's invention patent No. 200810198817.9 proposes a management system that supports different data collection services. The system adopts different terminals for different services and can support and manage different data collection services in a unified manner. However, the system is still traditional in nature. The data acquisition system only isolates different business data acquisition systems through the terminal to solve the problem of protocol expansion. If it is applied to the Internet of Things acquisition scene, there are still cumbersome equipment expansion, routing management and security restrictions, and it cannot solve the problems of cumbersome configuration of new data acquisition equipment, routing management and dynamic configuration, security protection and other problems proposed in this solution.

China's invention patent No. 200910111103.4 proposes a distributed data acquisition control platform system. This invention realizes the distributed acquisition of data through the integration of various communication methods based on the traditional data acquisition system. The scope of protocol expansion of the traditional acquisition system mentioned above also failed to solve the problems of cumbersome configuration of new data acquisition equipment, dynamic routing management and configuration, and security protection proposed in this solution.

Contents of the invention

In view of the above-mentioned deficiencies, the present invention provides a data collection system and method that can be dynamically and smoothly expanded, and realizes that the data collection center system can realize intelligent identification, access and data classification and collection of newly added equipment without manual configuration. Automatic on-demand allocation and dynamic adjustment of collection routes, security authentication of collection equipment access and data transmission, solve the problems of high expansion costs, poor expansion capabilities, and poor security of existing data collection systems under new application requirements.

In order to achieve the above object, the present invention adopts the following technical solutions:

A data acquisition system that can be dynamically and smoothly expanded, including: a data acquisition center system and a data acquisition terminal;

The data collection center system communicates with the data collection terminal through a communication network,

The data collection center system is the core of the data collection system, responsible for data collection, management, security certification, storage and log services;

The data collection terminal transmits the data of the collected equipment to the data collection node, and submits the data collection application form to the gateway routing node, and initiates business reports with each data collection node respectively according to the data collection node routing information form returned by the gateway routing node Block transfer request for encrypted data transfer;

The communication network is responsible for providing the data collection terminal with a physical link to the data collection center system, and is composed of one or more of Ethernet, GPRS, 3G, 4G, and industrial bus; and, it can also provide data for other Wired or wireless physical links for transmission; in order to ensure the versatility of routing information customization, the TCP/IP protocol is selected as the transport layer protocol.

The data collection center system includes:

The gateway routing node communicates with the data collection terminal, several data collection management nodes, security authentication nodes, monitoring nodes and log management nodes. It is the first entrance for the data collection terminal to enter the data collection center system through the communication network. The routing information of the gateway routing node is fixed. , generally one, open to all data collection terminals, responsible for processing the terminal data collection application form initiated by the data collection terminal to the data collection center system; and submitting the identity authentication information in the terminal data collection application form to the security authentication node for identity authentication , send the identity authentication result returned by the security authentication node to the data collection terminal; and, after the security authentication is passed, reorganize the summary information of the business report block in the terminal data collection application form according to the business type of the business report block to form a or Multiple classified business report block collection application forms; submit the classified business report block collection application form to the data collection management node that handles the business type according to the business type, and send the data collection node routing information form returned by each data collection management node to Data collection terminal.

The data collection center system also includes:

The security authentication node communicates with the gateway routing node, monitoring node, and security certificate storage node for security authentication of the data collection terminal, receives the identity authentication information in the terminal data collection application form forwarded by the gateway routing node, and compares the identity authentication information with the The certificate in the security certificate storage node is compared and authenticated, and the authentication result is sent to the gateway routing node. And, the identity authentication information is generated according to the unique identity of the data collection terminal, and the identity authentication information can be downloaded and stored in the key module of the data collection terminal.

The security certificate storage node communicates with the security authentication node and is used to store the security certificate of the data acquisition terminal;

The monitoring node communicates with the gateway routing node, security authentication node and log management node, and is used to monitor and control the running status of the gateway routing node, security authentication node, and log management node to ensure that each node works online.

The log management node communicates with the gateway routing node, security authentication node, data acquisition node, data acquisition management node, and log storage node, and is used for the recording, maintenance and management of log information generated by each node of the system;

The log storage node communicates with the log management node and is responsible for log storage;

The data acquisition management node communicates with the gateway routing node, the log management node, and several data acquisition nodes, receives the classification service report block collection application form sent by the gateway routing node, and collects each of the application forms for the classification service report block according to the routing selection strategy. The business report block entry selects the data collection node to form the data collection node routing information form; and, sends the data collection node routing information form to the gateway routing node, and pushes the communication information form to the gateway routing node according to the matching information in the data collection node routing information form. Corresponding data collection node;

The data acquisition node communicates with the data acquisition management node, the acquisition data storage node, the data acquisition terminal, and the log management node, is responsible for the data acquisition of the data acquisition terminal, establishes a communication connection for the transmission request of the data acquisition terminal, and is responsible for the data acquisition and analysis process;

Collect data storage nodes, communicate with data collection nodes, store data collected by each data collection node, and provide data access functions;

The time synchronization node is used to provide time synchronization services for all nodes in the system.

The gateway routing node, data acquisition management node and data acquisition node are logically organized according to the tree structure; the gateway routing node is the root node of the tree structure, and one or more data acquisition management nodes are connected thereunder; each data There are one or more data collection nodes under the collection management node.

Gateway routing nodes, data collection management nodes, data collection nodes, security authentication nodes, collection data storage nodes, monitoring nodes, time synchronization service nodes, security certificate storage nodes, log management nodes, log storage nodes, all the above application nodes are allowed to run on On the same computer server, multiple application nodes are also allowed to run on the same computer server; and, the above nodes are allowed to run on the same or multiple different cloud computing service platforms.

The terminal data collection application form includes identity authentication information, transmission key information, and business report block summary information.

The data acquisition terminal includes a main control board, and the main control board is respectively connected with a power supply module, a CPU module, a storage module, a communication module, a display module, a key module, a protocol conversion module, an input and output module, a time synchronization module, The keyboard module communicates bidirectionally.

The data collection terminal is allowed to communicate with several collected devices through RS232, RS485, CAN bus, Ethernet and wireless. The collected devices include traditional industrial field devices such as sensors, operation terminals, controllers, device communication modules, and PLCs, or Internet of Things smart devices in the fields of smart homes, smart buildings, and smart logistics, especially smart meters and advanced measurements. , smart electricity, electric vehicles and charging and other smart devices in the Internet of Things in the smart grid.

The communication between the data collection terminal and the gateway routing node and the data collection node is allowed to be connected through optical fiber Ethernet and wireless with TCP/IP protocol as the transport layer protocol.

Before the data acquisition terminal is connected to the above-mentioned system, it needs to complete the configuration of identity authentication, transmission key, and business report block summary information.

The data acquisition method used by the above-mentioned data acquisition system includes steps:

Step (1): The data collection terminal sends the terminal data collection application form to the gateway routing node;

Step (2): The gateway routing node generates a processing flow for the terminal data collection application form, each processing flow is an Erlang process, and the gateway routing node sends the identity authentication in the terminal data collection application form to the security authentication node in the Erlang process Information, waiting for the security authentication node to return the security authentication result of the data collection terminal; at the same time, the gateway routing node is waiting for a new collection application;

Step (3): The security authentication node performs security authentication on the data collection terminal according to the identity authentication information in the terminal data collection application form, and sends the authentication result to the gateway routing node;

Step (4): If the authentication fails, the gateway routing node returns the authentication result to the data collection terminal, ends the processing flow, and ends the Erlang process responsible for this processing;

Step (5): If the authentication is passed, the gateway routing node reorganizes the summary information of the business report block in the terminal data collection application form according to the business type of the business report block to form a business application form for the classified business report block; further, the gateway routing The node submits the business application form of the classified business report block to several data collection management nodes that handle the business type;

Step (6): After several data collection management nodes receive the business application form, they will match data collection nodes for each business report entry in the business application form according to the optimal strategy, organize and form the data collection node routing information form, and send the routing information form Return to the gateway routing node; the data acquisition management node generates a communication information form according to the matching information between the data acquisition node and the business report entry in the routing information form, and pushes the communication information form to the data acquisition node that completes the matching;

Step (7): The gateway routing node receives the routing information form returned by the data acquisition management node. At this time, the routing information form is composed of multiple specific business report block entries and a combination of routing information matching the business report block entry. The gateway routing node sends the routing information form to the data collection terminal that submitted the collection application; the Erlang processing process generated by the gateway routing node for the collection application has been waiting for the return of all application results submitted to the data collection management node. The terminal sends a timeout message;

Step (8): If the data collection terminal receives the timeout information, or the data collection node in the received routing information form is an empty data collection node, the data collection terminal ends the application, and after a period of delay, it sends the gateway routing node Initiate a new collection application;

Step (9): After receiving all the returned routing information forms, the data collection terminal initiates a business report block transmission request to the data collection node specified in each routing information form, and the data collection node establishes a transmission channel for each transmission request. Each transmission channel is an Erlang process, which completes the decryption, analysis, storage, framing, encryption and transmission of transmission messages according to the data transmission key information and data analysis method corresponding to the business report block;

Step (10): The data acquisition node stores the collected data in the collected data storage node according to the storage policy;

During the execution of the step (1)-step (10), the log management node stores the work logs generated by the gateway routing node, security authentication node, data acquisition management node, and data acquisition node in real time; the monitoring node monitors and maintains the gateway route in real time The running status of nodes, security authentication nodes, and data collection nodes. When the gateway routing node, security authentication node, or data collection node is abnormal, the monitoring node tries to restart the abnormal node and reminds the system maintenance personnel to handle the fault in time; time The synchronization node regularly synchronizes the time of the gateway routing node, security authentication node, security certificate storage node, monitoring node, log management node, log storage node, data collection management node, data collection node, and collection data storage node.

The terminal data collection application form in the step (1) includes identity authentication information, transmission key information, and business report block summary information.

The identity authentication information is generated by the security certificate management node according to the unique identity of the data collection terminal during the initialization phase of the collection terminal device, and is downloaded and stored in the key module of the data collection terminal. The downloading process cannot be repeated carried out and is irreversible.

The transmission key information is generated by the key module of the data collection terminal, and the transmission key information in each new data collection application form will be different, and the data collection node can use the key to analyze the data collection terminal in this transmission. The data encrypted by the key module is sent during the application transaction cycle, and the command sent to the data collection terminal is allowed to be encrypted with the key to ensure that the command can be recognized by the data collection terminal.

The business report block summary information is the collectable data entry information initialized by the data acquisition terminal, including the business report block set and its attributes divided according to the business type, the report block entries included in the report block set, and the report block entries included. Measurement points, data storage strategies and other attributes of measurement points.

The security authentication in the step (3) is used to ensure that only legally authenticated data collection terminals can be automatically connected to the system. The security authentication node verifies and matches the identity authentication information of the data collection terminal with the security certificate in the security certificate storage node, and returns the authentication passing result if the authentication is passed, otherwise the authentication fails and returns the authentication failure information.

The specific content of the step (5) is: the gateway routing node parses the terminal data collection application form, and forms one or more business type-based Classified business application forms, each business application form includes data collection terminal identification, data collection terminal transmission key information, business collection application type, supported data collection node commands and collection report block set of the service, where the collection report block The collection contains one or more business report block entries, and each entry contains the measurement point, data storage strategy and other descriptive information of the entry. The gateway routing node needs to send the business application form to the data collection management node that handles this type of business according to the business collection application type.

The preferred strategy in the step (6), the preferred strategy for the data acquisition management node to select among multiple data acquisition nodes under the management of the node includes the following factors: the IO load of the data acquisition node, the CPU load, and the physical Performance parameters, Qos from data collection terminal to data collection node.

The IO load is judged by the average number of Erlang processes within 30 minutes of the data collection node;

The CPU load is judged by the average CPU occupancy rate within 30 minutes;

The physical performance parameters of core components include CPU temperature, hard disk temperature, and CPU fan speed;

The Qos calculation method from the designated data acquisition terminal to the designated data acquisition node is as follows: select the data acquisition terminal located in the same and similar communication topology position as the data acquisition terminal from all the data acquisition terminals that maintain communication connections with the designated data acquisition node, and use these The average value of the Qos from the data collection terminal to the designated data collection node is the Qos from the data collection terminal to the designated node. If there is no data collection terminal with the same or similar network topology position, it is allowed not to use this policy factor.

The specific principles in the data acquisition node selection strategy are: low IO load, low CPU load, normal physical performance parameters of core components, and low Qos.

The routing information form in the step (6) is organized and formed by the data collection management node according to the optimal strategy for each business report entry in the business application form to match the data collection node, including the data collection terminal identifier and the business report block Corresponding communication protocol type, collection node IP address, collection node port number. The communication information form in the step (6) is generated by the data acquisition management node according to the data acquisition node matching information in the routing information form, including data acquisition terminal identification, data acquisition terminal transmission key information, business acquisition application type, support The data acquisition node command and the corresponding service report block entry information.

The steps of the step (9) are: the data acquisition management node generates a routing information form for each business application form, and each report block entry in the business application form corresponds to a data acquisition node routing information in the routing information form, and the routing information Including the type of communication protocol, IP address and port number of the data collection node; the data collection terminal initiates a connection request to each data collection node, and the data collection node provides a data collection channel for the data collection terminal in the form of an Erlang process; the data collection terminal communicates with the data collection node When the acquisition node establishes a communication connection for the first time, the data acquisition node requests the detailed description information of the measurement point from the data acquisition terminal, and the data acquisition terminal uploads the measurement point information under the corresponding report block entry of the data acquisition node; after that, the data acquisition terminal and the data acquisition node exchange data The key provided by the collection terminal performs data encryption and transmission, and the data collection node completes data analysis and data storage according to the analysis method specified in the self-description information of the measurement point provided by the data collection terminal.

For the data storage in the step (10), the data acquisition node transmits the data to the acquisition data storage node, and the acquisition data storage node completes the data cache according to the preset data storage strategy such as cache deviation and time interval, and provides real-time data section access function and persist the data to the database, and the database provides the persistent data access function; the data storage strategies of different business data report blocks also allow customized descriptions in the report block.

Beneficial effects of the present invention:

1. The data acquisition terminal is configured with identity authentication information, transmission key information, data points and business report block summary information during initialization, so that the data acquisition terminal has self-description ability, and realizes the self-description data access of various field devices . The data collection center system accepts the collection application form of the data collection terminal, queries the self-description information of the data collection terminal, and parses and stores the self-description information of the data collection terminal. In the case of changes, the data collection of the new data collection terminal is realized.

2. The system sends the service collection application form to the data collection management node through the gateway routing node, and the data collection management node dynamically allocates data collection nodes according to the optimal strategy to form a routing information form, and finally the gateway routing node returns the routing information to the collection terminal. With the cooperation of the gateway routing node, the dynamic automatic configuration and access of the routing information of the data collection system is realized through the data collection management node and data collection node organized according to the business and the technical implementation method of using one Erlang process corresponding to one collection channel Smooth adjustment of channel capacity.

3. In the present invention, the gateway routing node is the first entrance to the data collection system, and service requests without security authentication or encryption can only enter the gateway routing node, and the gateway routing node performs business forwarding or discarding; to the data collection node All services are securely authenticated and encrypted data. The combination of gateway routing and security authentication reduces the risk of the overall data collection system being attacked in various network environments; the technical solution of separating the gateway routing node from the data collection node and shielding the collection node improves the overall security of the data collection system. Security and availability, even if the gateway routing node is attacked, it only affects the access of new data collection terminals, and does not affect the normal data collection of the original connected data collection terminals.

4. The monitoring node monitors the normal operation of the gateway routing node, security authentication node, and log management node to ensure the stable operation of these nodes; at the same time, the monitoring node itself runs in hot standby mode to ensure the normal operation of the monitoring nodes. The overall monitoring ensures the stability of the system operation.

5. The optimal selection of data collection nodes by the data collection management node ensures the load balance of each data collection node. The data acquisition node provides data acquisition channels for data acquisition terminals in the form of Erlang processes to achieve smooth expansion of the acquisition channels; and the data acquisition terminals provide data analysis methods and measurement point information configuration-free methods to the acquisition nodes, avoiding tedious and troublesome manual configuration. The possibility of error; at the same time, the data communication between the data acquisition node and the data acquisition terminal adopts the key encryption method, which enhances the security of the communication.

6. During the operation of the system, except for the initialization operation of the data collection terminal newly connected to the system, other nodes of the data center system do not need to be manually involved in configuration changes, which reduces the number of operation and maintenance personnel during the use of the system The workload ensures the safe and stable operation of the system.

Description of drawings

Fig. 1 is a structural representation of the present invention;

Fig. 2 is a schematic diagram of the data acquisition node management hierarchy of the present invention;

Fig. 3 is a schematic diagram of the working structure of the monitoring node of the present invention;

Fig. 4 is a schematic structural diagram of a data acquisition terminal of the present invention;

Fig. 5 is a schematic diagram of the connection between the data collection terminal and the collected equipment of the present invention;

Fig. 6 is a schematic diagram of the data acquisition work interaction process of the present invention;

Fig. 7 is a schematic diagram of the acquisition application process of the present invention;

Fig. 8 is a schematic diagram of initial configuration information of the data acquisition terminal of the present invention;

Fig. 9 is a schematic diagram of a collection application form of the present invention;

Fig. 10 is a schematic diagram of the service application form of the present invention;

Fig. 11 is a schematic diagram of the routing information form of the present invention;

Fig. 12 is a schematic diagram of a communication information form of the present invention;

Among them, 1. Data collection terminal, 2. Gateway routing node, 3. Security authentication node, 4. Security certificate storage node, 5. Monitoring node, 6. Log management node, 7. Log storage node, 8. Data collection management node , 9. Data acquisition node, 10. Acquisition data storage node, 11. Time synchronization node; 21. Power supply module, 22. CPU module, 23. Storage module, 24. Communication module, 25. Display module, 26. Key module , 27. Protocol conversion module, 28. Input and output module, 29. Time synchronization module, 30. Keyboard module, 31. Acquired equipment, 40. Main control board.

Detailed ways

Below in conjunction with accompanying drawing and embodiment the present invention will be further described:

As shown in Figure 1, a data acquisition system that can be dynamically and smoothly expanded includes a data acquisition terminal 1, a gateway routing node 2, a security authentication node 3, a data acquisition management node 8, a data acquisition node 9, an acquisition data storage node 10, Monitoring node 5, time synchronization node 11, log management node 6, security certificate storage node 4, log storage node 7; the data collection terminal 1 communicates bidirectionally with the gateway routing node 2 and the data collection node 9 respectively; the gateway The routing node 2 communicates with the log management node 6, the monitoring node 5, the safety authentication node 3, and the data collection management node 8 respectively; , safety authentication node 3 and monitoring node 5 two-way communication; described safety authentication node 3 communicates with monitoring node 5, safety certificate storage node 4 two-way communication respectively; described data acquisition node 9 communicates with acquisition data storage node 10 and data acquisition management node respectively 8 two-way communication.

Gateway routing node, data collection management node, data collection node, security authentication node, collection data storage node, monitoring node, time synchronization service node, security certificate storage node, log management node, log storage node, all the above application nodes can be each The node runs on one computer server, or multiple nodes can run on the same computer server, each computer server is connected by a dedicated fiber channel or a dedicated wireless channel, and can be fixedly installed in the same room geographically , can be installed in many different cities, and can also be in a high-speed mobile state. Further, each of the above nodes can run on the same or multiple different cloud servers.

The gateway routing node 2 is the first entrance of the data acquisition terminal 1 into the data acquisition system, the routing information of the gateway routing node 2 is fixed and disclosed to all data acquisition terminals 1, and is responsible for processing the terminal data initiated by the data acquisition terminal 1 to the data acquisition system Collect application forms, including identity authentication information, transmission key information, and business report block summary information.

Further, the gateway routing node 2 submits the identity authentication information in the terminal data collection application form to the security authentication node 3 for identity authentication, waits for the security authentication node 3 to process, and sends the identity authentication result returned by the security authentication node 3 to the data Collect terminal 1.

Further, the gateway routing node 2, when the security authentication is passed, reorganizes the summary information of the business report block in the terminal data collection application form according to the business type of the business report block to form one or more classified business report block collection applications form, and submit the classified business report block collection application form to the data collection management node 8 that handles the corresponding business type by type. Wait for the data collection management node 8 to process, and send the routing information form of the data collection node 9 returned by each data collection management node 8 to the data collection terminal 1 .

The security authentication node 3 performs security authentication on the data collection terminal 1, receives the identity authentication information in the terminal data collection application form forwarded by the gateway routing node 2, and performs security authentication on the identity authentication information and the certificate in the security certificate storage node 4, Send the security authentication result to the gateway routing node 2. Further, the security certificate storage node 4 can generate identity authentication information according to the unique identity of the data collection terminal 1 , and the identity authentication information can be downloaded and stored in the key module of the data collection terminal 1 .

The data collection management node 8 manages one or more data collection nodes 9 that process similar business data, and receives the classification service report block collection application form sent by the gateway routing node 2, and the data collection management node 8 reports the classification service according to the routing strategy. Each business report block in the block collection application form selects a data collection node 9 to form a data collection node 9 routing information form. Further, the routing information form of the data collection node 9 is sent to the gateway routing node 2, and the communication information form is pushed to the corresponding data collection node 9 according to the matching information in the routing information form of the data collection node 9.

The data acquisition node 9 provides a data acquisition channel for the data acquisition terminal 1, and when receiving the service report block transmission request of the data acquisition terminal 1, a transmission channel is established for each transmission request, and each transmission channel is an Erlang process, According to the data transmission key information and data analysis method corresponding to the business report block, the decryption, analysis, storage, framing, encryption and transmission of the transmission message are completed.

Described time synchronization server node 11, with one or more in GPS signal, Beidou, B code, CDMA, atomic clock standard clock source as clock source, is data collection terminal 1, gateway routing node 2, security authentication node in the system 3. The data acquisition management node 8, the data acquisition node 9, the acquisition data storage node 10, the monitoring node 5, the log management node 6, the security certificate management node 4, and the log storage node 7 perform time synchronization, and the way of the time synchronization signal can be: NTP/SNTP signal, pulse signal, IRIG-B signal, time message.

The collected data storage node 10 stores the data collected by each data collection node 9 and provides a data access function.

The log management node 6 is responsible for the recording, maintenance and management of log information generated by each node in the system, and the log storage node 7 is responsible for the storage of logs.

The monitoring node 8 monitors and controls the running status of the gateway routing node 2, the security authentication node 3, and the log management node 6 to ensure that each node works online, and can be operated in a dual-node hot standby mode or in a cluster mode.

The data collection terminal 1 can communicate with the collected equipment through RS232, RS485, CAN bus, Ethernet, or wireless.

Further, the data collection terminal 1 can communicate with the gateway routing node 2 and the data collection node 9 through fiber optic Ethernet or wirelessly.

Further, the data collection terminal 1 submits the data collection application form to the gateway routing node 2 after completing identity authentication, transmission key, and business report block summary information configuration, and according to the routing information of the data collection node 9 returned by the gateway routing node 2 The form is connected with each data collection node 9 respectively, and the encrypted data transmission is carried out.

As shown in Figure 2, the system may include one or more data collection management nodes 8, the data collection management node 8 manages one or more data collection nodes 9 that process similar business data, and receives the classified service report block sent by the gateway routing node 2 To collect the application form, the data collection management node 8 selects the data collection node 9 for each business report block in the collection application form of the classified business report block according to the routing selection strategy, and forms the routing information form of the data collection node 9. Further, the routing information form of the data collection node 9 is sent to the gateway routing node 2, and the communication information form is pushed to the corresponding data collection node 9 according to the matching information in the routing information form of the data collection node 9.

As shown in Figure 3, monitoring node 5 communicates bidirectionally with gateway routing node 2, security authentication node 3, and log management node 6 respectively, and between the monitoring node 5 and other monitoring nodes 5 is also bidirectional communication; monitoring node 5 uses node Running in hot standby mode or cluster mode, gateway routing node 2, security authentication node 3, and log management node 6 register with monitoring node 5 and keep logging in. If a node runs abnormally, monitoring node 5 tries to restart the faulty node , and remind the system maintenance personnel to deal with the fault in time.

As shown in Figure 4, the data acquisition terminal 1 includes a main control board 40, and the main control board is respectively connected with a power supply module 21, a CPU module 22, a storage module 23, a communication module 24, a display module 25, a key module 26, a protocol conversion The module 27, the input and output module 28, the time synchronization module 29, and the keyboard module 30 communicate bidirectionally.

As shown in FIG. 5 , the data collection terminal 1 allows data interaction with several collected devices 31 in different communication methods, and the available communication methods include RS232, RS485, CAN bus, Ethernet, and wireless methods.

As shown in Figure 6, the schematic diagram of the data collection process:

S1. The data collection terminal sends the terminal data collection application form to the gateway routing node to obtain the routing information of the data collection node;

S2. Data collection terminal security authentication request: the gateway routing node sends the identity authentication information in the terminal data collection application form to the security authentication node, and performs security authentication of the data collection terminal;

S3. The security authentication node performs security authentication on the data collection terminal, and returns the authentication result to the gateway routing node. If the authentication fails, execute S6, and end this processing flow, and wait for a new collection application; if the authentication passes, execute S4;

S4. The gateway routing node reorganizes the summary information of the business report block in the terminal data collection application form according to the business type of the business report block to form a classification business report block collection application form, and submits the classification business report block collection application form to the processing The data collection management node of the service type to obtain the routing information of the data collection node;

S5. The data acquisition management node selects a data acquisition node for each business report entry in the application form according to the optimal strategy, organizes and forms a data acquisition node routing information form, returns the routing information form to the gateway routing node, and follows the data acquisition node routing information The matching information in the form pushes the communication information form to the corresponding data collection node;

S6. The gateway routing node sends the received data collection node routing information form to the data collection terminal;

S7. The data acquisition terminal initiates a service report block transmission request to the data acquisition node specified in each routing information form, and the data acquisition node establishes a transmission channel for each transmission request in the form of an Erlang process, and transmits key information according to the data corresponding to the service report block And data analysis method, decryption, analysis, storage and framing, encryption and transmission of transmission messages;

S8. Data storage. The data collection node will collect the data and store the data in the collection data storage node according to the storage policy.

As shown in Figure 7, the schematic diagram of the collection application process:

S01. Data collection terminal configuration, forming a data collection terminal configuration document.

S02. The data collection terminal sends a collection application form to the security routing node.

S03. The gateway routing node authenticates the identity of the data collection terminal to the security authentication node.

S04. The security authentication node returns the authentication result to the gateway routing node. If the authentication fails, execute S05, and if it succeeds, execute S06.

S05. The gateway routing node returns the application result to the data collection terminal: authentication failed.

S06. The gateway routing node forms a service application form and sends it to the data collection management node.

S07. The gateway routing node judges the application timeout, and returns the application result to the data collection terminal when the timeout expires: the application timeout.

S08. The data collection management node returns the data collection node routing information form to the gateway routing node, and pushes the communication information form to the data collection node.

S09. The gateway routing node sends the data collection node routing information form to the data collection terminal.

S10. After the data collection terminal receives all routing information forms, the collection application is successful.

As shown in Figure 8, the configuration document of the data acquisition terminal includes the file header, business report block information, and measurement point information. The file header is the basic information of the data collection terminal, including the name of the data collection terminal, the unique identifier of the data collection terminal, the model of the data collection terminal, the manufacturer, the creator, and the creation time. The security authentication node generates the data collection terminal according to the unique identifier of the data collection terminal. The identity authentication information of the terminal; the business report block information provides the basis for the data collection management node to select the data collection node. Each device can configure multiple report block sets according to the business type. The report block set attributes include business type and supported data collection node commands. And data parsing method, each report block set can configure one or more report block entries according to the number of measurement points or other factors, configure data storage attributes such as storage interval, storage deviation, and the ID of the measurement point included in the report block entry; The measuring point information configures the ID and specific information of each measuring point, such as measuring point name, measuring point type, measuring point control type and other required information.

As shown in Figure 9, the data collection application form includes identity authentication information, transmission key information, and business report block summary information. The identity authentication information is generated by the security certificate management node according to the unique identity of the data collection terminal in the equipment configuration stage, and downloaded and stored in the key module of the data collection terminal; the transmission key information is provided by the key module of the data collection terminal Generated; the business report block summary information is the business report block information in the data acquisition terminal configuration document, including the business report block set and its attributes divided according to the business type, the report block entries contained in the report block set, the measurement points contained in the report block entries, Data storage strategy and other properties.

As shown in Figure 10, the business application form is generated by the gateway routing node according to the business type of the report block set in the summary information of the collection application form business report block, and each business application form includes a data collection terminal identifier and a data collection terminal transmission key Information, business collection application type, supported data collection node commands, and business collection report set, where the business collection report set contains one or more business report block entries, and each entry contains the measurement point, data storage strategy and other descriptions of the entry information.

As shown in Figure 11, the routing information form, the data acquisition management node generates a routing information form for each business application form, and each report block entry in the business application form corresponds to a data acquisition node routing information in the routing information form, routing information Including communication protocol type, data acquisition node IP address and port number.

As shown in Figure 12, the communication information form, the data collection management node is generated according to the data collection node matching information in the routing information form, including the data collection terminal identification, data collection terminal transmission key information, business collection application type, and supported data collection Node commands and corresponding service report block entry information.

Although the specific implementation of the present invention has been described above in conjunction with the accompanying drawings, it does not limit the protection scope of the present invention. Those skilled in the art should understand that on the basis of the technical solution of the present invention, those skilled in the art do not need to pay creative work Various modifications or variations that can be made are still within the protection scope of the present invention.

Claims (10)

1. a data acquisition system for the level and smooth expansion of capable of dynamic, is characterized in that, comprising: data acquisition center system and data collection station;

Described data acquisition center system is communicated by letter with described data collection station by communication network,

Described data acquisition center system is the core of data acquisition system, is responsible for data acquisition, management, safety certification, storage and log services;

Described data collection station by the transfer of data of collected equipment to data acquisition node, and by data acquisition application submission of sheet gateway routing node, the data acquisition node routing iinformation list returning according to gateway routing node, initiate business report piece transmission request with each data acquisition node respectively, carry out Data Encryption Transmission;

Described communication network is responsible for the physical link that data collection station is provided to data acquisition center system;

Described data acquisition center system comprises:

Gateway routing node, with data collection station, some data capture management nodes, safety certification node, monitor node and log management node communication, that data collection station enters data acquisition center system the first entrance by communication network, the routing iinformation of gateway routing node is fixed, open to all data collection stations, be responsible for the terminal data collection application list that deal with data acquisition terminal is initiated to data acquisition center system; And, submit to safety certification node to carry out authentication the authentication information in terminal data collection application list, the identity authentication result that safety certification node is returned is sent to data collection station; And, after safety certification is passed through, business report piece summary information in terminal data collection application list is reorganized according to the type of service of business report piece, form one or more classify traffic report blocks collection application lists; By type of service, classify traffic report blocks collection application list is submitted to the data capture management node of processing this type of service, the data acquisition node route information table single transmit that each data capture management node is returned is to data collection station.

2. the data acquisition system of the level and smooth expansion of a kind of capable of dynamic as claimed in claim 1, is characterized in that,

Described data acquisition center system also comprises:

Safety certification node, communicate by letter with gateway routing node, monitor node, safety certificate memory node, for carrying out the safety certification of data collection station, authentication information in the terminal data collection application list that reception gateway routing node forwards, by the authentication of comparing of the certificate in authentication information and safety certificate memory node, authentication result is sent to gateway routing node; And, according to the uniqueness identify label of data collection station, generating authentication information, authentication information allows lower dress to store in the cipher key module of data collection station;

Safety certificate memory node, with safety certification node communication, for storing the safety certificate of data collection station;

Monitor node, with gateway routing node, safety certification node and log management node communication, for monitoring and control the running status of gateway routing node, safety certification node, log management node, guarantees that each node works online;

Log management node, with gateway routing node, safety certification node, data acquisition node, data capture management node, log store node communication, the record, the maintenance and management work that for each node of system, produce log information;

Log store node, with log management node communication, is responsible for the storage of daily record;

Data capture management node, communicate by letter with gateway routing node, log management node, some data acquisition nodes, receive the classify traffic report blocks collection application list that gateway routing node sends, according to routing strategy, be that each business report piece entry in classify traffic report blocks collection application list is chosen data acquisition node, form data acquisition node routing iinformation list; And, data acquisition node route information table single transmit, to gateway routing node, is pushed to corresponding data acquisition node according to the match information in data acquisition node routing iinformation list by communication information list simultaneously;

Data acquisition node, with data capture management node, acquired data storage node, data collection station, log management node communication, be responsible for the data acquisition of data collection station, for the transmission request of data collection station, set up communication and connect, be responsible for data acquisition resolving;

Acquired data storage node, communicates by letter with data acquisition node, the data that gather for storing each data acquisition node, and data access function is provided;

Time synchronization node, is used to all nodes in system that time synchronized service is provided.

3. the collecting method that the data acquisition system of the level and smooth expansion of a kind of capable of dynamic as described in above-mentioned arbitrary claim is used, is characterized in that, comprises step:

Step (1): data collection station is to gateway routing node transmitting terminal data acquisition application list;

Step (2): gateway routing node is that terminal data collection application list generates handling process, each handling process is an Erlang process, gateway routing node to the authentication information in safety certification node transmitting terminal data acquisition application list, waits for that safety certification node returns to the security certification result of this data collection station in this Erlang process; Gateway routing node is waited for new collection application simultaneously;

Step (3): safety certification node carries out safety certification according to the authentication information in terminal data collection application list to data collection station, and authentication result is sent to gateway routing node;

Step (4): if authentification failure, gateway routing node returns to data collection station by authentication result, finishes this handling process finishes to be responsible for the Erlang process of this processing simultaneously;

Step (5): if authentication is passed through, gateway routing node reorganizes business report piece summary information in terminal data collection application list according to the type of service of business report piece, forms the service order form list of classify traffic report blocks; Further, gateway routing node is submitted to the service order form list of classify traffic report blocks respectively some data capture management nodes of processing this type of service;

Step (6): after some data capture management nodes receive service order form list, according to preference policy, it is each the business report entries match data acquisition node in service order form list, tissue forms data acquisition node routing iinformation list, and routing iinformation list is back to gateway routing node; Data capture management node generates communication information list according to the match information of the data acquisition node in routing iinformation list and business report entry simultaneously, and communication information list has been pushed to the data acquisition node of coupling;

Step (7): gateway routing node receives the routing iinformation list that data capture management node returns, now routing iinformation list forms by a plurality of concrete business report piece entries with the combination entry of the routing iinformation of this business report piece entries match, and gateway routing node is delivered to routing iinformation list the data collection station of submitting to this collections to apply for; Gateway routing node is waited for returning of oriented data capture management node submit applications result by the Erlang treatment progress that this collection application generates always, if overtime, to data collection station, sends time-out information;

Step (8): if data collection station receives time-out information, or the data acquisition node in the routing iinformation list receiving is empty data acquisition node, data collection station finishes this application, and postpones after a period of time, to gateway routing node, initiates new collection application;

Step (9): data collection station is after receiving all routing iinformation lists that return, to the fixed data acquisition node of each routing iinformation list middle finger, initiate business report piece transmission request, data acquisition node transmits request for each sets up transmission channel, each transmission channel is an Erlang process, according to transfer of data key information corresponding to business report piece and Data Analysis mode, complete deciphering, parsing, storage and framing, encryption and the transmission of message transmission;

Step (10): data acquisition node stores image data into acquired data storage node according to storage policy by data; The daily record of work that log management node real-time storage gateway routing node, safety certification node, data capture management node, data acquisition node produce in described step (1)-step (10) implementation; The running status of gateway routing node, safety certification node, data acquisition node is monitored, safeguarded to monitor node in real time, when gateway routing node, safety certification node or data acquisition node occur when abnormal, monitor node attempts abnormal nodes to carry out reboot operation, and system for prompting attendant carries out troubleshooting in time; The time of time synchronization node Timing Synchronization gateway routing node, safety certification node, safety certificate memory node, monitor node, log management node, log store node, data capture management node, data acquisition node, acquired data storage node.

4. method as claimed in claim 3, is characterized in that, the terminal data collection application list in described step (1), comprises authentication information, transmission security key information, business report piece summary information;

Described authentication information is that this acquisition terminal equipment is at initial phase, by safety certificate management node, according to the uniqueness identify label of this data collection station, generated, and lower dress stores the cipher key module of data collection station into, this lower process of assembling does not allow to repeat, and irreversible;

Described transmission security key information is produced by data collection station cipher key module, transmission security key information in each new data acquisition application list can be different, data acquisition node can apply in this transmission the data after cipher key module is encrypted that send in transaction cycles by resolution data acquisition terminal by this key, and allow to send to the order of data collection station to be encrypted with this key, guarantee that this order can be identified by data collection station;

Described business report piece summary information is the data items for information that the initialized permission of this image data acquisition terminal gathers, and comprises the business report set of blocks divided according to type of service and attribute thereof, the report blocks entry of report blocks set-inclusion, the measuring point that report blocks entry comprises, data store strategy and other attributes of measuring point.

5. method as claimed in claim 3, is characterized in that, the safety certification in described step (3) only has the data collection station through legal authentication could automatically be linked into system for guaranteeing; Safety certification node is verified that the authentication information of data collection station mate with the safety certificate in safety certificate memory node, if be verified return authentication, passes through result, otherwise authentification failure, return authentication does not pass through information.

6. method as claimed in claim 3, it is characterized in that, described step (5) particular content is: gateway routing node is resolved terminal data collection application list, according to the type of service of report blocks set in business report piece self-described information in this terminal data collection application list, form one or more service order form lists of classifying according to type of service, each service order form list comprises data collection station sign, data collection station transmission security key information, business gathers applying type, the data acquisition node order of supporting and the collection report blocks set of this business, wherein gather the one or more business report piece of report blocks set-inclusion entry, the measuring point that each entry comprises this entry, data store strategy and other descriptors, gateway routing node need to gather applying type according to business and service order form single transmit be arrived to the data capture management node of processing such business.

7. method as claimed in claim 3, it is characterized in that, preference policy in described step (6), the preference policy of selecting in the data acquisition node of data capture management node under a plurality of node administrations comprises following factor: the physical function parameter of the IO load of data acquisition node, cpu load, core original paper, data collection station are to the Qos of data acquisition node;

IO load judges with the par of Erlang process in data acquisition node 30 minutes;

Cpu load judges with the average occupancy of CPU in 30 minutes;

The physical function parameter of core original paper comprises cpu temperature, hard disk temperature, cpu fan rotating speed;

Specific data acquisition terminal to the Qos account form of specific data acquisition node is: the data collection station of selecting to be positioned at this data collection station identical and close communication topology location from all and specific data acquisition node are kept in communication the data collection station connecting, take these data collection stations to the mean value of the Qos of specific data acquisition node as this data acquisition data collection station to the Qos of this specified node, if there is not the data collection station of identical or close network topology position, allow not adopt this strategic factor;

Data acquisition node selects principle concrete in dominant strategy to be: IO load is low, cpu load is low, the physical function parameter of core original paper is normal, Qos is low.

8. method as claimed in claim 3, it is characterized in that, routing iinformation list in described step (6), be by data capture management node according to being that each business report entries match data acquisition node in service order form list is organized and formed according to preference policy, comprise data collection station sign and corresponding communications protocol type, acquisition node IP address, the acquisition node port numbers of business report piece;

Communication information list in described step (6), be that data capture management node generates according to the data acquisition node match information in routing iinformation list, comprise that data collection station sign, data collection station transmission security key information, business gather data acquisition node order and the corresponding business report piece items for information of applying type, support.

9. method as claimed in claim 3, it is characterized in that, the step of described step (9) is: data capture management node is a routing iinformation list of each service order form list generation, in service order form list, each report blocks entry is to should route information table Dan Zhongyi bar data acquisition node routing iinformation, and routing iinformation comprises communications protocol type, data acquisition node IP address and port numbers; Data collection station is initiated connection request to each data acquisition node, and data acquisition node be take the mode of Erlang process provides data acquisition channel as data collection station; Data collection station is being set up communication while being connected first with data acquisition node, and data acquisition node is described information in detail to data collection station request measuring point, and data collection station is uploaded the corresponding report blocks bar of this data acquisition node measuring point information now; After this key that data collection station and data acquisition node provide with data collection station carries out Data Encryption Transmission, and the analysis mode of stipulating in the measuring point self-described information that data acquisition node provides according to data collection station completes Data Analysis and data deposit.

10. method as claimed in claim 3, it is characterized in that, data storage in described step (10), data acquisition node transfers data to acquired data storage node, acquired data storage node completes data buffer storage according to predefined data store strategy as buffer memory deviation, the time interval, real time data section access function is provided and data persistence is arrived to database, database provides perdurable data access function; The data store strategy of different business datum report blocks also allows to customize description in report blocks.

Images