CN117937752A - All-in-one DCIM system for ultra-large data management - Google Patents

Abstract

The present invention discloses an integrated DCIM system for super-large data management, and belongs to the technical field of DCIM systems. In order to solve the problem that the existing distributed DCIM system architecture technology cannot manage future super-large data centers, there are system freezes, and the data refresh time cannot meet the requirements of end customers, the present invention uses a data access system, a data processing system, and a management fusion platform, and three management servers run at the same time. The load balancing of the server is achieved through the system operation control software, and the dynamic environment monitoring system, the power monitoring system, and the BA automatic control system are connected and integrated into a management platform, which not only improves the processing capacity of the server cluster, but also reduces the problem of cross-platform use for operation and maintenance personnel, and greatly improves the work efficiency of operation and maintenance.

Description

All-in-one DCIM system for ultra-large data management

Technical Field

The present invention relates to the technical field of DCIM systems, and in particular to an integrated DCIM system for ultra-large data management.

Background technique

The DCIM system can help administrators optimize the operating efficiency of data centers, reduce energy consumption, and provide predictive maintenance and capacity planning functions. It can also help administrators monitor the performance and status of equipment in real time, and provide alarm and notification functions to handle faults and problems in a timely manner. In general, the DCIM system can improve the reliability, availability and security of data centers, improve operation and maintenance efficiency, and reduce costs.

In recent years, with the breakthroughs in large model training, computing chips, GPU, NPU, 800G optical modules, high-density batteries and other technologies, and the rapid development of refrigeration technology, the power of a single cabinet has increased from 8kW to 60kW or higher, and mass production and project implementation have been achieved. In the future, the upstream and downstream industrial chains of liquid cooling technology will continue to improve, and the power of a single cabinet will continue to rise. More cabinets will be produced in the same building, which will bring great challenges and pressure to the building's DCIM system.

The existing distributed DCIM system architecture technology cannot manage future ultra-large data centers.

1. There will be system freezes and data refresh time cannot meet the requirements of end customers;

2. There are also problems of system fragmentation and difficulty in locating faults.

Summary of the invention

The purpose of the present invention is to provide an integrated DCIM system for ultra-large data management to solve the problem in the above-mentioned background technology that the existing distributed DCIM system architecture technology is unable to manage future ultra-large data centers.

To achieve the above object, the present invention provides the following technical solution: an integrated DCIM system for ultra-large data management, comprising:

Data access system for:

It is the data interface for each acquisition module of the entire DCIM system and also the source of feedback on equipment status. It receives equipment information and operating status from the dynamic environment monitoring system, power monitoring system, and BA system.

Data processing systems for:

The information collected at the terminal is calculated, analyzed and stored. It has the function of configurable linkage control to realize the automatic linkage control of related systems or equipment. It has the function of complex event analysis to automatically find the root cause alarm, which is convenient for quickly locating the root cause during operation and maintenance, quickly solving problems, and quickly recovering from faults. The data processing system includes 3 management servers and 3 storage servers. The management server analyzes and processes the collected data, reports and calculates the equipment faults, and the storage server stores the collected data in full. The information received by the data access system is uploaded to the data processing system through the switch;

Management converged platform for:

The dynamic environment monitoring system, power monitoring system and BA automatic control system are connected and integrated into a management platform, which improves the processing capacity of the server cluster.

Furthermore, the data access system includes: a data acquisition module, a data transmission module, a data cleaning and conversion module, a data protection module, a data flow control module and a data sending module, wherein

Data acquisition module for:

Collect data from various data sources, including substations and battery rooms, IT rooms, and refrigeration stations, as well as external data sources, including cloud services and external monitoring systems. Through data collection, the system can obtain real-time data information, providing a basis for subsequent data processing and analysis;

Data transmission module for:

Responsible for transmitting the collected data to ensure fast, stable and secure data transmission, using TCP/IP protocol for transmission;

Data cleaning and conversion module, used to:

During the data transmission process, the data can be cleaned and converted to ensure the quality and consistency of the data format. The data can be deduplicated, de-noised, and format converted to make the data more standardized and easier to manage.

Data Assurance Module for:

Responsible for ensuring the security of data, protecting data through encryption technology and identity authentication to prevent data from being tampered with, leaked or accessed without authorization during transmission;

Data flow control module, used for:

Control and manage data traffic to ensure balanced and stable data transmission. Prioritize and allocate bandwidth to data based on the needs and resources of the data center to ensure timely transmission of critical data.

Data sending module, used for:

The collected and organized data are sent to the data processing system.

Furthermore, the data cleaning and conversion module includes: a data deduplication module, a data denoising module and a data format conversion module;

Data deduplication module, used for:

The collected data is deduplicated using a hash algorithm.

Data denoising module, used to:

Through filtering technology, the data is denoised to improve the accuracy and reliability of the data;

Data format conversion module, used for:

According to the needs of the data center, the collected data is converted into a format that is consistent with the data format within the system, facilitating subsequent data processing and analysis.

Furthermore, the data cleaning conversion module also includes:

Data normalization module, used to:

Normalize the data and convert it into uniform units and magnitudes to facilitate data comparison and analysis;

Data verification module, used for:

The data is verified through cyclic redundancy check. If the data verification fails, the data is retransmitted or an alarm is issued.

Data completion module, used to:

Interpolate or complete historical data to fill in data gaps and ensure data integrity and continuity.

Furthermore, the data protection module includes:

Data encryption module for:

During data cleaning and conversion, symmetric encryption algorithms are used to encrypt sensitive data to ensure data security during transmission and storage;

Data desensitization module, used for:

For data involving personal privacy or sensitive information, data desensitization technology is used during cleaning and conversion to shield or replace sensitive information to protect user privacy and data security;

Data permission control module, used for:

During the data cleaning and conversion process, a strict data permission control mechanism is set up to limit users' access to and operation rights over data. Only authorized users can perform data cleaning and conversion operations to ensure data security and controllability.

Furthermore, the data protection module also includes:

Data auditing and monitoring module for:

Establish an audit and monitoring mechanism for data cleaning and conversion, record data cleaning and conversion operations for subsequent audit and tracing, monitor abnormal operations and data changes during data cleaning and conversion, and promptly identify and handle data security risks;

Data backup and recovery module for:

During the data cleaning and conversion process, data backup should be performed regularly to ensure data recoverability and durability. During the data cleaning and conversion process, data should be backed up in a timely manner to prevent data loss or damage and ensure data security and availability;

Data anomaly detection module, used to:

Establish an anomaly detection and alarm mechanism to monitor anomalies during data cleaning and conversion, and trigger the alarm mechanism in a timely manner once an anomaly is found.

Furthermore, the data flow control module includes:

Traffic monitoring and analysis module for:

By using the Snort traffic analysis tool to monitor and analyze data traffic in real time, abnormal traffic or attack behaviors can be discovered in time, thus protecting data security.

Traffic prioritization module for:

Data with large flow rates are sent first to ensure the timeliness of important data;

Traffic blocking module, used for:

Combined with the traffic monitoring and analysis results, set up abnormal traffic detection rules and configure the alarm mechanism. When abnormal traffic is detected, the alarm mechanism is triggered, and data cleaning conversion, data auditing and monitoring are stopped to prevent data leakage.

Furthermore, the data processing system comprises:

The data storage module stores the information sent by the sorting, including online storage and offline storage. The online storage is stored through the storage server;

The unit linkage module runs three management servers simultaneously and realizes server load balancing through system operation control software;

Positioning alarm module, used for:

Locate and alarm the monitored equipment, automatically find the root cause of the alarm, and facilitate quick root cause location, problem solving, and fault recovery during operation and maintenance;

Report generation module for:

Operation and maintenance personnel design reports that meet their personalized needs.

Furthermore, the positioning alarm module includes:

The identifier setting module is used to assign a unique identifier to each device, so as to ensure that each device has a clear identification and establish a coordinate system so that each device has its own coordinate point;

The fault direction determination module is used in the DCIM system. There are two types of faults: one is that the fault source is clear, and the other is that the fault source is unclear. Set up real-time monitoring of equipment status faults and use sensors and data collectors to record equipment status information. Both types of faults include power faults and environmental faults. Set power faults as D and environmental faults as H.

The precise fault marking module is used to mark a clear fault source as a precise fault (i.e., the precise representative symbol X), and to determine whether a specific fault mode has occurred by analyzing the monitoring data. The occurrence of a specific fault mode is a clear fault source;

The fault location module is used to identify the faulty device by analyzing the monitoring data after the fault source is clear, locate the coordinate point of the faulty device, and determine the specific device and direction where the problem occurs;

A faulty equipment display module is used to display the faulty equipment and direction on a plane diagram with coordinate axes in the user interface of the DCIM system, so that the user can accurately understand the location and status of the faulty equipment and direction;

The alarm notification module is used to determine the faulty equipment and direction. Once the DCIM system promptly issues an alarm and notifies the relevant personnel, they can take corrective measures quickly.

Furthermore, the report generation module includes:

The equipment status collection module is used to collect and record the equipment status data monitored in the DCIM system, including power and environmental data;

The data analysis and collation module is used to analyze and collate the collected equipment status data and extract data related to faults, anomalies or other important information;

The information fusion module is used to generate equipment status reports as needed, including a list of faulty equipment, equipment status change trends, and fault frequency statistics. The coordinate points of the faulty equipment are recorded at the same time. In the user interface of the DCIM system, the number of faults is displayed on a plane diagram containing coordinate axes, which is divided into three levels: red, yellow, and blue. Red means that the equipment has 5 or more faults, yellow means that the equipment has 2-4 faults, and blue means that the equipment has 0 or 1 fault.

Compared with the prior art, the present invention has the following beneficial effects:

1. Under the existing technology, the distributed DCIM system has stuck and the data refresh time cannot meet the requirements of end customers. The present invention transmits the collected data through the data transmission module to ensure the fast, stable and safe transmission of data. The TCP/IP protocol is selected for transmission. Compared with the RS485 interface, the transmission speed is faster and the efficiency is better, which helps to improve the response time of the whole system. The information sent is stored through the data storage module, including online storage and offline storage. The online storage is stored through the storage server. The three management servers are run at the same time through the unit linkage module. The load balancing of the server is achieved through the system operation control software. The positioning alarm module is used to locate and alarm the equipment to be monitored, and the root cause alarm is automatically found, which is convenient for quickly locating the root cause during operation and maintenance, quickly solving the problem, and quickly recovering from the fault. When the system is running, the three management servers are running at the same time. It runs at the same time, realizes the load balancing of the server through the system operation control software, connects the dynamic environment monitoring system, power monitoring system and BA automatic control system, and integrates them into a management platform. This not only improves the processing capacity of the server cluster, but also reduces the problem of cross-platform use for operation and maintenance personnel, greatly improving the work efficiency of operation and maintenance. For a project with 2 million measurement points, if 10 servers need to be configured for processing and calculation according to the traditional method, the integrated platform only needs to be configured with 3 management servers. The refresh time of the traditional system architecture interface is more than 10 seconds, while the integrated technology can achieve a refresh time of 6 seconds for the 2 million measurement point system. If the management system needs to be expanded, it only needs to add servers in the cluster and debug them, without adjusting the various subsystems. For the traditional system architecture, the integrated technology can reduce 15%-20% of the investment and has great advantages in system application.

2. Under the existing technology, the current distributed DCIM system is not secure enough for data processing. The present invention uses a data cleaning and conversion module, a data security module, and a data flow control module, and uses a data deduplication module to perform deduplication processing on the collected data through a hash algorithm deduplication algorithm to ensure that the obtained data is not repeated. The data denoising module uses a filtering technology to perform denoising processing on the data to improve the accuracy and reliability of the data. The data format conversion module converts the format of the collected data according to the needs of the data center to make it consistent with the data format inside the system, which is convenient for subsequent data processing and analysis. The data is then normalized by the data normalization module to convert it into a unified unit and magnitude to facilitate data comparison and analysis. The data verification module verifies the data using a cyclic redundancy check. If the data verification fails, the data is retransmitted or an alarm is processed. Finally, the data completion module interpolates or completes according to historical data to fill in the gaps in the data to ensure the integrity and continuity of the data, so that during the data transmission process, the data can be cleaned and converted to ensure the quality and format consistency of the data, and the data can be deduplicated, de-noised, and format converted to make the data more standardized and easy to manage.

Secondly, during the data cleaning and conversion process, a symmetric encryption algorithm is used to encrypt sensitive data to ensure the security of data during transmission and storage. For data involving personal privacy or sensitive information, data desensitization technology is used to shield or replace sensitive information during cleaning and conversion to protect user privacy and data security. During the data cleaning and conversion process, a strict data permission control mechanism is set up to limit user access and operation permissions to data. Only authorized users can perform data cleaning and conversion operations to ensure data security and controllability. At the same time, an audit and monitoring mechanism for data cleaning and conversion is established to record data cleaning and conversion operations for subsequent auditing and tracing, monitor abnormal operations and data changes during data cleaning and conversion, and promptly discover and deal with data security risks. Secondly, combined with traffic monitoring and analysis results, abnormal traffic detection rules are set, and an alarm mechanism is configured. When abnormal traffic is detected, the alarm mechanism is triggered, and data cleaning conversion, data auditing and monitoring are stopped to prevent data leakage, thereby improving the accuracy and security of the equipment information and operating status of the receiving dynamic environment monitoring system, power monitoring system, and BA system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a topological diagram of an integrated DCIM system for ultra-large data management according to the present invention;

FIG2 is an overall flow chart of the integrated DCIM system for ultra-large data management according to the present invention;

3 is a flowchart of a data cleaning and conversion module, a data security module, and a data flow control module in an integrated DCIM system for ultra-large data management according to the present invention;

FIG. 4 is a flowchart of a positioning alarm module and a report generation module in the integrated DCIM system for ultra-large data management of the present invention.

Detailed ways

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Please refer to Figures 1 to 3, the present invention provides the following technical solutions:

An all-in-one DCIM system for large-scale data management, including:

Data access system for:

It is the data interface for each acquisition module of the entire DCIM system and also the source of feedback on equipment status. It receives equipment information and operating status from the dynamic environment monitoring system, power monitoring system, and BA system.

Data processing systems for:

The information collected at the terminal is calculated, analyzed and stored. It has the function of configurable linkage control to realize the automatic linkage control of related systems or equipment. It has the function of complex event analysis to automatically find the root cause alarm, which is convenient for quickly locating the root cause during operation and maintenance, quickly solving problems, and quickly recovering from faults. The data processing system includes 3 management servers and 3 storage servers. The management server analyzes and processes the collected data, reports and calculates the equipment faults, and the storage server stores the collected data in full. The information received by the data access system is uploaded to the data processing system through the switch;

Management converged platform for:

The dynamic environment monitoring system, power monitoring system and BA automatic control system are connected and integrated into a management platform, which improves the processing capacity of the server cluster.

The data access system includes: data acquisition module, data transmission module, data cleaning and conversion module, data protection module, data flow control module and data sending module.

Data acquisition module for:

Collect data from various data sources, including substations and battery rooms, IT rooms, and refrigeration stations, as well as external data sources, including cloud services and external monitoring systems. Through data collection, the system can obtain real-time data information, providing a basis for subsequent data processing and analysis;

Data transmission module for:

Responsible for transmitting the collected data, ensuring fast, stable and secure data transmission, and using TCP/IP protocol for transmission;

Data cleaning and conversion module, used to:

During the data transmission process, the data can be cleaned and converted to ensure the quality and consistency of the data format. The data can be deduplicated, de-noised, and format converted to make the data more standardized and easier to manage.

Data Assurance Module for:

Responsible for ensuring the security of data, protecting data through encryption technology and identity authentication to prevent data from being tampered with, leaked or accessed without authorization during transmission;

Data flow control module, used for:

Control and manage data traffic to ensure balanced and stable data transmission. Prioritize and allocate bandwidth to data based on the needs and resources of the data center to ensure timely transmission of critical data.

Data sending module, used for:

The collected and organized data are sent to the data processing system.

The data cleaning and conversion module includes: data deduplication module, data denoising module and data format conversion module;

Data deduplication module, used for:

The collected data is deduplicated using a hash algorithm.

Data denoising module, used to:

Through filtering technology, the data is denoised to improve the accuracy and reliability of the data;

Data format conversion module, used for:

According to the needs of the data center, the collected data is converted into a format that is consistent with the data format within the system, facilitating subsequent data processing and analysis.

The data cleaning and conversion module also includes:

Data normalization module, used to:

Normalize the data and convert it into a unified unit and magnitude to facilitate data comparison and analysis;

Data verification module, used for:

Verify the data through cyclic redundancy check. If the data verification fails, retransmit the data or issue an alarm.

Data completion module, used to:

Interpolate or complete historical data to fill in data gaps and ensure data integrity and continuity.

The data protection module includes:

Data encryption module for:

During data cleaning and conversion, symmetric encryption algorithms are used to encrypt sensitive data to ensure data security during transmission and storage;

Data desensitization module, used for:

For data involving personal privacy or sensitive information, data desensitization technology is used during cleaning and conversion to shield or replace sensitive information to protect user privacy and data security;

Data permission control module, used for:

During the data cleaning and conversion process, a strict data permission control mechanism is set up to limit users' access to and operation rights over data. Only authorized users can perform data cleaning and conversion operations to ensure data security and controllability.

The data protection module also includes:

Data auditing and monitoring module for:

Establish an audit and monitoring mechanism for data cleaning and conversion, record data cleaning and conversion operations for subsequent audit and tracing, monitor abnormal operations and data changes during data cleaning and conversion, and promptly identify and handle data security risks;

Data backup and recovery module for:

During the data cleaning and conversion process, data backup should be performed regularly to ensure data recoverability and durability. During the data cleaning and conversion process, data should be backed up in a timely manner to prevent data loss or damage and ensure data security and availability;

Data anomaly detection module, used to:

Establish an anomaly detection and alarm mechanism to monitor anomalies during data cleaning and conversion, and trigger the alarm mechanism in a timely manner once an anomaly is found.

The data flow control module includes:

Traffic monitoring and analysis module for:

By using the Snort traffic analysis tool to monitor and analyze data traffic in real time, abnormal traffic or attack behaviors can be discovered in time, thus protecting data security.

Traffic prioritization module for:

Data with large flow rates are sent first to ensure the timeliness of important data;

Traffic blocking module, used for:

Combined with the traffic monitoring and analysis results, set up abnormal traffic detection rules and configure the alarm mechanism. When abnormal traffic is detected, the alarm mechanism is triggered, and data cleaning conversion, data auditing and monitoring are stopped to prevent data leakage.

Specifically, the data deduplication module uses the hash algorithm to perform deduplication processing on the collected data to ensure that the obtained data is not repeated. The data denoising module uses filtering technology to perform denoising processing on the data to improve the accuracy and reliability of the data. The data format conversion module converts the format of the collected data according to the needs of the data center to make it consistent with the data format inside the system, which is convenient for subsequent data processing and analysis. The data normalization module then performs normalization processing on the data and converts it into a unified unit and magnitude to facilitate data comparison and analysis. The data verification module then verifies the data using cyclic redundancy check. If the data verification fails, the data is retransmitted or an alarm is processed. Finally, the data completion module interpolates or completes the historical data to fill the data gaps and ensure the integrity and continuity of the data, so that the data can be cleaned and converted during the data transmission process to ensure the quality and consistency of the format of the data. The data can be deduplicated, denoised, and format converted to make the data more standardized and easy to manage.

Secondly, during the data cleaning and conversion process, a symmetric encryption algorithm is used to encrypt sensitive data to ensure the security of data during transmission and storage. For data involving personal privacy or sensitive information, data desensitization technology is used to shield or replace sensitive information during cleaning and conversion to protect user privacy and data security. During the data cleaning and conversion process, a strict data permission control mechanism is set up to limit user access and operation permissions to data. Only authorized users can perform data cleaning and conversion operations to ensure data security and controllability. At the same time, an audit and monitoring mechanism for data cleaning and conversion is established to record data cleaning and conversion operations for subsequent auditing and tracing, monitor abnormal operations and data changes during data cleaning and conversion, and promptly discover and deal with data security risks. Secondly, combined with traffic monitoring and analysis results, abnormal traffic detection rules are set, and an alarm mechanism is configured. When abnormal traffic is detected, the alarm mechanism is triggered, and data cleaning conversion, data auditing and monitoring are stopped to prevent data leakage, thereby improving the accuracy and security of the equipment information and operating status of the receiving dynamic environment monitoring system, power monitoring system, and BA system.

Please refer to Figures 1 and 4, the present invention provides the following technical solutions:

The data processing system includes:

The data storage module stores the information sent by the sorting, including online storage and offline storage. The online storage is stored through the storage server;

The unit linkage module runs three management servers simultaneously and realizes server load balancing through system operation control software;

Positioning alarm module, used for:

Locate and alarm the monitored equipment, automatically find the root cause of the alarm, and facilitate quick root cause location, problem solving, and fault recovery during operation and maintenance;

Report generation module for:

Operation and maintenance personnel design reports that meet their personalized needs.

The positioning alarm module includes:

The identifier setting module is used to assign a unique identifier to each device, so as to ensure that each device has a clear identification and establish a coordinate system so that each device has its own coordinate point;

The fault direction determination module is used in the DCIM system. There are two types of faults: one is that the fault source is clear, and the other is that the fault source is unclear. Set up real-time monitoring of equipment status faults and use sensors and data collectors to record equipment status information. Both types of faults include power faults and environmental faults. Set power faults as D and environmental faults as H.

The precise fault marking module is used to mark a clear fault source as a precise fault (i.e., the precise representative symbol X), and to determine whether a specific fault mode has occurred by analyzing the monitoring data. The occurrence of a specific fault mode is a clear fault source;

The fault location module is used to identify the faulty device by analyzing the monitoring data after the fault source is clear, locate the coordinate point of the faulty device, and determine the specific device and direction where the problem occurs;

A faulty equipment display module is used to display the faulty equipment and direction on a plane diagram with coordinate axes in the user interface of the DCIM system, so that the user can accurately understand the location and status of the faulty equipment and direction;

The alarm notification module is used to determine the faulty equipment and direction. Once the DCIM system promptly issues an alarm and notifies the relevant personnel, they can take corrective measures quickly.

The report generation module includes:

The equipment status collection module is used to collect and record the equipment status data monitored in the DCIM system, including power and environmental data;

The data analysis and collation module is used to analyze and collate the collected equipment status data and extract data related to faults, anomalies or other important information;

The information fusion module is used to generate equipment status reports as needed, including a list of faulty equipment, equipment status change trends, and fault frequency statistics. The coordinate points of the faulty equipment are recorded at the same time. In the user interface of the DCIM system, the number of faults is displayed on a plane diagram containing coordinate axes, which is divided into three levels: red, yellow, and blue. Red means that the equipment has 5 or more faults, yellow means that the equipment has 2-4 faults, and blue means that the equipment has 0 or 1 fault.

Specifically, the collected data is transmitted through the data transmission module to ensure fast, stable and secure data transmission. The TCP/IP protocol is used for transmission. Compared with the RS485 interface, the transmission speed is faster and the efficiency is better, which helps to improve the response time of the entire system. The information sent is stored through the data storage module, including online storage and offline storage. The online storage is stored through the storage server. The three management servers are run simultaneously through the unit linkage module, and the server load balancing is achieved through the system operation control software. Through the positioning alarm module, the monitored equipment is located and alarmed, and the root cause alarm is automatically found, which is convenient for quickly locating the root cause during operation and maintenance, and quickly solving the problem. Rapid failure recovery: When the system is running, three management servers are running at the same time, and the load balancing of the servers is achieved through the system operation control software. At this time, the load rate of each server is 33.33%; if one server fails and exits the cluster service, the other two servers will take over the operation task. At this time, the load rate of the two main servers is 66.67%. The operation of the two servers does not affect the calculation and response time of the integrated platform system, and can maintain high-speed operation; if two servers fail to exit the service at the same time, the main server will bear all the loads. At this time, the system can be used normally, and functions such as data collection, analysis and alarm are not affected, but the response time will increase accordingly. When one of the servers fails, priority is given to the main server. To monitor the fault of this server, the monitoring direction is divided into two parts, one is self-monitoring, and the other is monitoring of the instruments carried by the server. There are multiple instruments for instrument monitoring, which can be listed as W1, W2, W3, ..... Wn. The direction of instrument monitoring is dynamic environment monitoring D and power monitoring H, and the direction of single instrument monitoring is W1=D1+H1, where D1 is at least one type in D, and H1 is at least one type in H. Priority is given to monitoring the smaller number of D1+H1, and so on, until all monitoring is completed. When monitoring W1, priority is given to monitoring D1, and then monitoring H1, connecting the dynamic environment monitoring system, power monitoring system, and BA automatic control system, and integrating them into a management platform. This not only improves the processing capacity of the server cluster, but also reduces the problem of cross-platform use for operation and maintenance personnel, greatly improving the efficiency of operation and maintenance. For a project with 2 million measurement points, if 10 servers are configured for processing and calculation in the traditional way, the integrated platform only needs to be configured with 3 management servers. The refresh time of the traditional system architecture interface is more than 10 seconds, while the integrated technology can achieve a refresh time of 6 seconds for the 2 million measurement point system. If the management system needs to be expanded, it only needs to add servers to the cluster and debug them, without adjusting the various subsystems. For the traditional system architecture, the integrated technology can reduce 15%-20% of the investment and has great advantages in system application.

The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims (10)

1. An ultra-large data management integrated DCIM system, comprising:

A data access system for:

The data interfaces of all acquisition modules of the whole DCIM system are also sources for feeding back the state of equipment, and equipment information and running states of the dynamic ring monitoring system, the electric power monitoring system and the BA system are received;

A data processing system for:

The method comprises the steps of calculating, analyzing and storing information acquired by the tail end, having a configuration linkage control function, realizing automatic linkage control of related systems or equipment, having a complex event analysis function, automatically searching for root alarms, facilitating quick positioning of the root in operation and maintenance, quickly solving problems and quickly recovering faults, wherein a data processing system comprises 3 management servers and 3 storage servers, the management servers analyze and process acquired data, report and calculate faults of the equipment, the storage servers store the acquired data in full quantity, and the information received by a data access system is uploaded to the data processing system through a switch;

A management fusion platform for:

The dynamic ring monitoring system, the electric power monitoring system and the BA automatic control system are integrated into a management platform, so that the processing capacity of the server cluster is improved.

2. The ultra-large data management integrated DCIM system of claim 1, wherein: the data access system comprises: data acquisition module, data transmission module, data cleaning conversion module, data guarantee module, data flow control module and data transmission module, wherein

The data acquisition module is used for:

data are collected from all data sources, including a substation, a battery room area, an IT (information technology) inter-package area, a refrigerating station area and external data sources, wherein the data sources comprise cloud services and an external monitoring system, and the system can acquire real-time data information through data collection so as to provide a basis for subsequent data processing and analysis;

the data transmission module is used for:

The system is responsible for transmitting the acquired data, ensures the rapid, stable and safe transmission of the data, and adopts TCP/IP protocol for transmission;

the data cleaning conversion module is used for:

In the data transmission process, the data can be cleaned and converted, the quality and format consistency of the data are ensured, the data can be subjected to de-duplication, de-noising and format conversion, and the data are more standard and easier to manage;

The data guarantee module is used for:

The security of the data is ensured, the data is protected by means of encryption technology and identity verification, and the data is prevented from being tampered, leaked or unauthorized access in the transmission process;

a data flow control module for:

the data flow is controlled and managed, so that balance and stable transmission of the data are ensured, priority and bandwidth are allocated to the data according to the requirements and resource conditions of a data center, and timely transmission of key data is ensured;

A data sending module, configured to:

and sending the collected and tidied data to a data processing system.

3. The ultra-large data management integrated DCIM system of claim 2, wherein: the data cleaning and converting module comprises: the device comprises a data de-duplication module, a data de-noising module and a data format conversion module;

the data deduplication module is used for:

Performing deduplication processing on the acquired data through a hashing algorithm deduplication algorithm;

the data denoising module is used for:

denoising the data by a filtering technology, so that the accuracy and reliability of the data are improved;

a data format conversion module for:

According to the requirements of a data center, format conversion is carried out on the collected data, so that the collected data is consistent with the data format in the system, and subsequent data processing and analysis are convenient.

4. The ultra-large data management integrated DCIM system of claim 3, wherein: the data cleaning and converting module further comprises:

The data normalization module is used for:

the data is normalized and converted into a unified unit and magnitude, so that the data can be conveniently compared and analyzed;

The data verification module is used for:

checking the data through cyclic redundancy check, and if the data check fails, retransmitting the data or carrying out alarm processing;

The data complement module is used for:

interpolation or complementation is carried out according to the historical data, so that the gap of the data is filled, and the integrity and continuity of the data are ensured.

5. The ultra-large data management integrated DCIM system of claim 4, wherein: the data guarantee module comprises:

a data encryption module for:

In the process of data cleaning and conversion, a symmetric encryption algorithm is adopted to encrypt sensitive data, so that the safety of the data in the process of transmission and storage is ensured;

a data desensitization module for:

for data related to personal privacy or sensitive information, when cleaning and converting, adopting a data desensitization technology to shield or replace the sensitive information so as to protect user privacy and data safety;

The data authority control module is used for:

In the process of data cleaning and conversion, a strict data authority control mechanism is set to limit the access and operation authority of a user to data, and only authorized users can perform data cleaning and conversion operation, so that the safety and controllability of the data are ensured.

6. The ultra-large data management integrated DCIM system of claim 5, wherein: the data guarantee module further comprises:

and the data auditing and monitoring module is used for:

Establishing an audit and monitoring mechanism for data cleaning and conversion, recording the cleaning and conversion operation of the data so as to facilitate the subsequent audit and tracing, monitoring the abnormal operation and data change in the data cleaning and conversion process, and timely finding and processing the data security risk;

a data backup and recovery module for:

In the process of data cleaning and conversion, data backup is carried out regularly, so that the restorability and durability of the data are ensured, and in the process of data cleaning and conversion, the data are backed up timely, so that the data are prevented from being lost or damaged, and the safety and usability of the data are ensured;

the data anomaly detection module is used for:

An abnormality detection and alarm mechanism is established, abnormal conditions in the data cleaning and conversion process are monitored, and once the abnormal conditions are found, the alarm mechanism is triggered in time.

7. The ultra-large data management integrated DCIM system of claim 6, wherein: the data flow control module comprises:

The flow monitoring and analyzing module is used for:

the data flow is monitored and analyzed in real time through the Snort flow analysis tool, so that abnormal flow or attack behavior can be found in time, and the safety of the data is protected;

traffic priority module for:

For data with larger flow, the data is sent preferentially, so that timeliness of important data is ensured;

A flow blocking module for:

And setting an abnormal flow detection rule by combining flow monitoring and analysis results, configuring an alarm mechanism, triggering the alarm mechanism when abnormal flow is monitored, and simultaneously performing data cleaning conversion and data audit and monitoring to prevent data leakage.

8. The ultra-large data management integrated DCIM system of claim 7, wherein: the data processing system includes:

The data storage module is used for storing the information sent by the whole machine, wherein the data storage module comprises on-line storage and off-line storage, and the on-line storage is stored through the storage server;

the unit linkage module is used for simultaneously operating 3 management servers and realizing the load balance of the servers through system operation control software;

the positioning alarm module is used for:

The equipment to be monitored carries out positioning alarm, automatically searches for root cause alarm, facilitates quick positioning of the root cause in operation and maintenance, quickly solves the problem and quickly recovers faults;

report generation module, which is used for:

The operation and maintenance personnel design report reports meeting the personalized requirements of the operation and maintenance personnel.

9. The ultra-large data management integrated DCIM system of claim 8, wherein: the positioning alarm module comprises:

the identifier setting module is used for distributing a unique identifier to each device, so that each device can be ensured to have a definite identifier, a coordinate system is established, and each device has a coordinate point belonging to the device;

The fault direction determining module is used for determining two types of faults in the DCIM system, wherein one type of faults is a clear fault source, the other type of faults is an unclear fault source, real-time monitoring on equipment state faults is set, the state information of the equipment is recorded by using a sensor and a data acquisition device, the two types of faults comprise power faults and environment faults, the power faults are set to be D, and the environment faults are set to be H;

The accurate fault marking module is used for marking accurate faults (namely accurate representative symbol X) when a clear fault source is encountered, determining whether a specific fault mode occurs or not through analyzing monitoring data, wherein the occurrence of the specific fault mode is the clear fault source;

the fault positioning module is used for determining equipment with faults through analyzing the monitoring data after the clear fault source is eliminated, positioning coordinate points of the faulty equipment, and determining specific equipment and direction with problems;

The fault equipment display module is used for displaying equipment and directions with faults on a plane graph containing coordinate axes in a user interface of the DCIM system so that a user can accurately know the positions and states of the fault equipment and directions;

and the alarm notification module is used for timely giving an alarm and notifying corresponding personnel by the DCIM system once the equipment and the direction of the fault are determined so that the DCIM system can quickly take corrective action.

10. The ultra-large data management integrated DCIM system of claim 9, wherein: the report generation module comprises:

the device state collection module is used for collecting and recording the device state data monitored in the DCIM system, wherein the device state data comprise power and environment data;

The data analysis and arrangement module is used for analyzing and arranging the collected equipment state data and extracting data related to faults, anomalies or other important information;

The information fusion module is used for generating a device state report, wherein the report comprises a fault device list, a device state change trend and a fault frequency statistics report, and simultaneously records coordinate points of the device with faults.