CN115203213A - Wind tunnel real-time data efficient acquisition and storage system - Google Patents
Wind tunnel real-time data efficient acquisition and storage system Download PDFInfo
- Publication number
- CN115203213A CN115203213A CN202211118807.6A CN202211118807A CN115203213A CN 115203213 A CN115203213 A CN 115203213A CN 202211118807 A CN202211118807 A CN 202211118807A CN 115203213 A CN115203213 A CN 115203213A
- Authority
- CN
- China
- Prior art keywords
- data
- acquisition
- database
- wind tunnel
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a wind tunnel real-time data high-efficiency acquisition and storage system, which relates to the field of data processing and comprises the following components: the distributed acquisition layer comprises a plurality of node servers which are respectively accessed to a plurality of wind tunnel data sources; a multi-production multi-storage layer comprising a plurality of central servers; the server side of the acquisition and storage middleware is respectively deployed on a plurality of central servers; the clients of the acquisition and storage middleware are respectively deployed on a plurality of node servers; a distributed storage tier comprising a database farm connected to a plurality of central servers; the invention supports distributed acquisition and distributed storage of data in a wind tunnel multi-type multi-protocol mode, realizes efficient processing of mass data and efficient parallel storage of the data by the system, finally realizes full data link communication of wind tunnel real-time data from acquisition, transmission, storage, processing and application, and provides data support for fusion of a wind tunnel test management system and a wind tunnel field information system and full link communication of the wind tunnel.
Description
Technical Field
The invention relates to the field of data processing, in particular to a wind tunnel real-time data efficient acquisition and storage system.
Background
The real-time database carries out real-time monitoring, fault early warning and fault diagnosis on various devices in the wind tunnel, and in addition, the production efficiency and the operation precision of the machine are further improved through the information interaction of an interconnection network formed by the devices, products and production indexes, so that the whole production process always keeps the characteristics of high efficiency, energy conservation and durability. Because the data volume produced in the wind tunnel test process is large and high in value, the system has high workload in the data acquisition and data processing part, the acquisition period of the wind tunnel is generally millisecond level, and extremely strict timing limit is provided in the aspects of data reading, writing, processing and dumping.
At present, with the increasing of the types and the quantity of equipment in wind tunnels and the continuous application of various service-oriented information systems, the types and the quantity of data of the wind tunnels far exceed the category of traditional data acquisition, the traditional single-machine type acquisition is restricted by hardware performance, the single-machine access points and the expansion capability can not meet the requirements of field data acquisition gradually, although the existing industrial real-time data acquisition equipment can support various industrial protocols, the wind tunnel data is more professional, the single-machine acquisition is not distributed and acquired and stored more stably, the expansibility and the replanting cost are higher, and the requirements of high real-time performance, high storage and high concurrency mass real-time data acquisition can not be met, so the acquisition mode of centralized deployment and distributed acquisition gradually becomes the mainstream realization mode of the wind tunnel data acquisition.
Disclosure of Invention
An object of the present invention is to solve at least the above problems and/or disadvantages and to provide at least the advantages described hereinafter. To achieve these objects and other advantages in accordance with the purpose of the invention, there is provided a wind tunnel real-time data efficient acquisition and storage system, comprising:
the distributed acquisition layer comprises a plurality of node servers which are respectively accessed to a plurality of wind tunnel data sources;
the multi-mining multi-storage layer comprises a plurality of central servers;
the server side of the acquisition and storage middleware is respectively deployed on a plurality of central servers; the clients of the acquisition and storage middleware are respectively deployed on a plurality of node servers;
a distributed storage tier comprising a database farm connected to a plurality of central servers.
Preferably, the number of the wind tunnel data sources is three, and the three wind tunnel data sources are respectively a first data source, a second data source and a third data source; the first data source is control system data and is led into the node server in a PLC/PC-Base mapping mode; the second data source is measurement system data and is introduced into the node server in an NI variable engine MQTT transmission mode; the third data source is health management system data and is imported into the node server in an OPC DA transmission mode.
Preferably, the number of the node servers is five, and the node servers are respectively two first node servers, two second node servers and one third node server; the number of the central servers is two, and the central servers are respectively a central main server and a central slave server; the two central servers are respectively connected with five node servers, so that data double acquisition is formed; the first data source is respectively connected with the two first node servers; the second data source is respectively connected with the two second node servers; the third data source is connected with a third node server thereof; the node servers are interconnected by using a dedicated connection as a communication link, and are connected with a time synchronization server together.
Preferably, the storage middleware is developed by a multi-thread data processing program based on a C # framework, and the driver is distributed in a DLL form to realize modularization of the program.
Preferably, the mining and storing middleware comprises a data interface and an interface component; the data interface and interface component is integrated and configured with one or more of PC-base, wonderware database, NI shared variable, MQTT service, modbus TCP, OPC DA/UA, webAPI, socket industrial data driven protocol.
Preferably, wherein the mining middleware comprises a point configuration component; the point configuration component is used for creating acquisition points connected with a plurality of wind tunnel data sources and realizing data acquisition and hierarchical management through a built-in logic grouping structure.
Preferably, the acquisition and storage middleware comprises a data cache component; the data cache component comprises an InfluxDB cache database used for storing write-in failure data and a matched service program;
the concrete implementation flow of the service program is configured as follows: after the data is failed to be written into the database group by the acquisition and storage middleware, two kinds of information are generated, wherein one kind of information is a log and is directly stored in the local of the node server to form a log file; the other information is write failure data which is directly stored in an InfluxDB cache database; and reading the unprocessed log files at regular time, trying to rewrite the data into the database group when finding that the data fails to be written, deleting the log files or changing the state of the log files into processed state if the data is successfully written, and waiting for the next processing if the data is failed to be written.
Preferably, the database groups are deployed in a server and disk array manner; the database group comprises a master-slave time sequence database which is in butt joint with a plurality of central servers; the master-slave time sequence database comprises an InfluxDB master database and an InfluxDB slave database; the export data of the acquisition and storage middleware are simultaneously imported into an InfluxDB master database and an InfluxDB slave database; and the InfluxDB master database and the InfluxDB slave database are mutually connected to keep data synchronization so as to form data double writing.
Preferably, the database group further includes an SQL Server database; the SQL Server database is respectively connected with the InfluxDB master database and the InfluxDB slave database.
Preferably, a real-time data publishing interface is configured on the acquisition and storage middleware; the real-time data publishing interface can be configured to be one of MQTT and WebAPI.
The invention at least comprises the following beneficial effects:
(1) The system adopts a distributed acquisition architecture. Through a plurality of sets of acquisition and storage middleware client sides, functions of data caching, classified acquisition and the like are realized, and high-efficiency acquisition of real-time data is ensured; with the continuous increase of the types and the quantity of equipment in wind tunnels and the continuous application of various service-oriented information systems, the distributed acquisition and storage architecture adopted by the system has good expansibility, flexibility and proper redundancy. The system can realize the data acquisition and storage service of other similar projects only by additionally arranging hardware, and ensures the system re-portability;
(2) The acquisition and storage middleware comprises industrial protocols such as OPC DA/UA and Modbus, and standardized and customized software supporting protocols such as TCP/IP and Socket, and realizes concurrent processing of distributed acquired data and load balancing of acquired data; the acquisition and storage middleware uses a distributed server structure, can realize millisecond acquisition on different types of data of the disperse system, and ensures the unification, synchronization and accurate update of the data. Multithreading reading and writing is adopted for reading and writing of a single data acquisition and storage middleware node, 2000 data acquisition points can be read at most, and efficient data acquisition of equipment of different manufacturers, different models and different software support systems can be met;
(3) The high-efficiency parallel storage of data is realized by adopting a distributed storage technology and combining a time sequence database means; the storage expandability and the storage efficiency can be improved by the distributed storage technology; the problem of classified data storage and the whole data circulation are solved through a database cluster of a time sequence database and a relational database. Finally, effective management of wind tunnel data is realized, and a uniform data resource pool is formed;
(4) The acquisition and storage middleware of the system is based on a C # framework, a multi-thread data processing program is developed, the multi-thread data processing program is designed, the concurrent processing of distributed acquired data and the load balance of the acquired data can be met, and the system can efficiently process a large amount of data.
(5) The drive program of distributed collection is released in a DLL form, so that the modularization of the program is realized, the program operation and abnormal maintenance are facilitated, and the utilization rate of system resources is effectively improved.
(6) The system adopts a double-acquisition and double-storage acquisition and storage mode, so that the reliability of data is improved.
(7) The system provides various data service functions, and can store all the acquisition point data into an InfluxDB real-time database for storage; before the data is stored in the InfluxDB database, the data can be directly issued to the corresponding data service system by the acquisition and storage middleware through the data interface, so that the real-time performance of data transmission is ensured, and the large time delay caused by the storage and the delivery of the data in and out of the database is avoided.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention.
Drawings
FIG. 1 is a wind tunnel real-time data distributed acquisition architecture diagram of the present invention;
FIG. 2 is a functional block diagram of the acquisition and storage system of the present invention;
FIG. 3 is a diagram of the architecture of the database cluster and the mining and warehousing middleware of the present invention;
FIG. 4 is a diagram of a distributed storage architecture of the present invention;
the system comprises a 1-distributed acquisition layer, a 10-wind tunnel data source, a 101-control system data, a 102-measurement system data, a 103-health management system data, an 11-node Server, a 111-first node Server, a 112-second node Server, a 113-third node Server, a 12-time synchronization Server, a 2-multi-acquisition multi-storage layer, a 21-center Server, a 211-center main Server, a 212-center slave Server, a 3-acquisition and storage middleware, a 31-interface component, a 32-configuration component, a 33-data cache component, a 331-InfluxDB cache database, a 34-real-time data distribution interface, a 4-distributed storage layer, a 41-group, a 42-master-slave time sequence database, a 421-InfluxDB main database, a 422-InfluxDB slave database and a 43-SQL Server database.
Detailed Description
The present invention is further described in detail below with reference to the attached drawings so that those skilled in the art can implement the invention by referring to the description text.
Fig. 1-4 show an implementation form of the present invention, including:
the distributed acquisition layer 1 comprises a plurality of node servers 11 which are respectively accessed to a plurality of wind tunnel data sources 10;
a multi-mining multi-storage layer 2 including a plurality of center servers 21;
the acquisition and storage middleware 3, the service ends of which are respectively deployed on a plurality of central servers 21; the clients of the acquisition and storage middleware 3 are respectively deployed on a plurality of node servers 11;
a distributed storage tier 4 comprising a database farm 41 connected to a plurality of central servers 21.
The working principle is as follows: in the invention, a plurality of data sources 10 in the wind tunnel system can be respectively transmitted to a plurality of different node servers 11 of the distributed acquisition layer 1 according to respective industrial data driving protocols, thereby realizing the function of distributed data acquisition; the clients of the acquisition and storage middleware 3 are respectively deployed on the node servers 11, and the servers of the acquisition and storage middleware 3 are respectively deployed on the central servers 21; therefore, a plurality of node servers 11 and a plurality of central servers 21 can build a multi-to-multi data acquisition and storage network, so that data acquired by all the node servers 11 can be uploaded to each central server 21, and multi-acquisition of the data is realized; each central server 21 writes all the acquired data into the database group 41 together, thereby realizing the multi-write and distributed storage of the data;
in the design, the distributed acquisition layer 1 configured in the whole wind tunnel data acquisition and storage system can be used for classifying and acquiring a plurality of data sources 10 in the wind tunnel system, is convenient to configure, regulate and control and ensures the high-efficiency acquisition of different types of real-time data; in the multi-acquisition multi-storage layer 2, a plurality of central servers 21 are deployed and are respectively connected with the plurality of node servers 11, so that a many-to-many data acquisition network is formed, and after the rear database group 41 is connected in a butt joint manner, when each central server 21 acquires and writes data, the condition that one central server 21 is down due to a fault and the system operation is influenced can be effectively prevented, so that the safety and the applicability of the whole data acquisition and storage system are further improved; the acquisition and storage middleware 3 can be packaged with a plurality of industrial data driving protocols, so that the concurrent processing of distributed acquisition data and the load balance of the acquisition data are realized; after the database group 41 in the distributed storage layer 4 is adopted to be in butt joint with a plurality of central servers, efficient parallel storage of data can be achieved.
In the above technical solution, the wind tunnel data sources 10 are three, which are respectively a first data source, a second data source and a third data source; the first data source is control system data 101 which is led into the node server 11 in a PLC/PC-Base mapping mode; the second data source is measurement system data 102, which is introduced into the node server 11 through an NI variable engine MQTT transmission mode; the third data source is health management system data 103, which is imported into the node server 11 by means of OPC DA transmission.
The working principle is as follows: because there are many sensing devices in the wind tunnel system, the data volume to be collected is large, and the format and the use are different, the mass data is divided into three data sources for collection, for example, the control system data 101 specifically collects real-time data uploaded by all sensors used for process control in the wind tunnel system, the measurement system data 102 specifically collects real-time data uploaded by all sensors used for parameter measurement in the wind tunnel system, and the health management system data 103 specifically collects "health parameters" of all devices in the wind tunnel system (i.e., parameters such as the service life, the operation time, alarm information parameters, and the operation state of the devices); through the configuration, the wind tunnel data source 10 can be docked into one of the node servers 11 by different industrial data driven protocols, data with different attributes can be transmitted into the node server 11 in the most efficient and stable mode, and the adaptability of the system to each data source 10 in the distributed acquisition layer 1 is further improved.
In the above technical solution, the number of the node servers 11 is five, and the node servers are respectively two first node servers 111, two second node servers 112, and one third node server 113; the number of the central servers 21 is two, and the central servers are respectively a central master server 211 and a central slave server 212; the two central servers 21 are respectively connected with the five node servers 11, so that data double acquisition is formed; the first data source 101 is respectively connected with two first node servers 111; the second data source 102 is respectively connected with two second node servers 112; the third data source 103 is connected to its third node server 113; a plurality of the node servers 11 are interconnected by using a dedicated connection as a communication link, and are commonly connected with a time synchronization server 12.
The working principle is as follows: by configuring five node servers 11, at a downlink end, a 2-2-1 data transmission link architecture can be formed together with three different data sources 10 of a wind tunnel, so that the problem of data transmission interruption caused by a fault of one data transmission line can be effectively prevented, and the safety and the stability of the device are greatly expanded; if wind tunnel field acquisition points are increased subsequently, a corresponding number of node servers can be increased according to different point location conditions, so that the expansion and extension of a distributed acquisition architecture are realized, and the expandability of system data acquisition is improved; the data time identification collected and stored by each system cannot be unified due to different computer clock references of each system among the original wind tunnel systems, so that the data correlation analysis among the systems is greatly influenced; therefore, according to the scheme, a time synchronization server 12 is configured in a system network, acquires standard time signals from a GPS satellite, synchronizes the time of each node server 11 on the network through NTP network time protocol, and provides high-precision time correction, so that the time synchronization of the data acquisition and storage of the whole system is consistent.
In the above technical solution, the storage middleware 3 is developed by a multithread data processing program based on a C # framework, and a driver is released in a DLL format to realize modularization of the program.
The working principle is as follows: a large amount of read-write operations exist in the database group 41, the conventional single-thread data processing method cannot well meet the requirement of a system on high-efficiency processing of a large amount of data, and in order to realize concurrent processing of data and improve the hardware efficiency of a server, the storage middleware 3 is developed based on a C # (. NET) framework and a multi-thread data processing program is designed, so that the data processing capacity and efficiency of the system are improved; meanwhile, when a problem occurs in the program, only the code of the related DLL can be modified and put into use again; when the program runs, the needed DLL module is loaded only when needed, and can be unloaded from the memory after use, so that the utilization rate of system resources can be effectively improved.
In the above technical solution, the storage middleware 3 includes a data interface and interface component 31; the data interface and interface component 31 is integrally configured with one or more of PC-base, wonderware database, NI shared variables, MQTT service, modbus TCP, OPC DA/UA, webAPI, socket industrial data driven protocol.
The working principle is as follows: the acquisition and storage middleware 3 can be used for packaging and configuring various industrial data driving protocols and supporting standardization and customization software of various protocols, so that concurrent processing of distributed acquisition data and load balancing of acquisition data can be realized.
In the above technical solution, the mining and storing middleware 3 includes a point configuration component 32; the point configuration component 32 will create acquisition points that interface with multiple wind tunnel data sources 10 and implement data acquisition and hierarchical management via built-in logical grouping structures.
The working principle is as follows: through the logical grouping structure in the point configuration component 32, the creation, management and grouping of data acquisition points can be performed, and the functions of creation and management of data channels, creation and management of equipment, management of issued signals, acquisition frequency change in special time periods and the like are also provided.
In the above technical solution, the storage middleware 3 includes a data cache component 33; the data cache component 33 includes an infilxdb cache database 331 for storing write failure data and a matching service program;
the concrete implementation flow of the service program is configured as follows: after the data is failed to be written into the database group 41 by the acquisition and storage middleware 3, two kinds of information are generated, wherein one kind of information is a log and is directly stored in the local of the node server 11 to form a log file; the other information is write failure data, and is directly stored in the infiluxdb cache database 331; by reading the unprocessed log file at regular time, when the data which has failed to be written is found, the data is tried to be rewritten into the database group 41, if the writing is successful, the log file is deleted or the state of the log file is changed to processed, and if the writing is failed, no operation is performed, and the next processing is waited.
The working principle is as follows: by arranging the data cache component 33, high reliability of real-time data acquisition can be ensured, each node server 11 has a data cache function so as to deal with the conditions of network interruption, offline for upgrading or maintaining a central machine room server and the like, and the data cache time is usually set to be more than 12 hours (2,000 times of data storage for 10 times/s) so as to meet the acquisition requirements of various data sources of an actual wind tunnel system.
In the above technical solution, the database group 41 is deployed in a manner of server plus disk array; the database cluster 41 includes a master-slave timing database 42 interfaced with a plurality of central servers 21; the master-slave timing database 42 includes an infixdb master database 421 and an infixdb slave database 422; the export data of the acquisition and storage middleware 3 is imported into an InfluxDB master database 421 and an InfluxDB slave database 422 at the same time; the infixdb master database 421 and the infixdb slave database 422 are connected to each other to keep data synchronization, so as to form data double-write.
The working principle is as follows: in the design, the deployment mode of the server and the disk array can enable the database group 41 to expand by adding disks, so that the expansibility and the applicability of equipment are improved, and the device is more suitable for the storage of mass data of a wind tunnel system; as a write-in end directly butted with a plurality of central servers 21, the master-slave time sequence database 42 is formed by two interconnected time sequence databases infiluxdb, all data acquired by the acquisition and storage middleware 3 are respectively stored in the master-slave time sequence database 42 and the slave-slave time sequence database 42, and the data can be written into the master-slave two databases at the same time, so that the data safety and the high availability of the system are ensured; the InfluxDB master-slave services mutually synchronize data and keep the data consistency; when data is written in, the InfluxDB engine filters the data according to the timestamp, the data identification and the data value, and repeated data writing is avoided, so that local data can be written in a master-slave time sequence database when being written in the database. By using an asynchronous writing mode, the data double-writing backup can be realized on the premise of not increasing the writing time.
In the above technical solution, the database group 41 further includes an SQL Server database 43; the SQL Server database 43 is connected to the infixdb master database 421 and the infixdb slave database 422, respectively.
The working principle is as follows: in this design, some of the main collected data written into the master-slave timing database 42 is directly stored in the master-slave timing database 42, but the other part of the real-time data (device operating status, alarm information, etc.) related to the "device operating information" is converted and stored as historical data into the SQL Server database 43, so as to complete the functions of device status monitoring and early warning, fault diagnosis, life prediction, maintenance recommendation, etc.
In the above technical solution, the acquisition and storage middleware 3 is configured with a real-time data publishing interface 34; the real-time data publishing interface 34 may be configured as one of MQTT and WebAPI.
The working principle is as follows: the acquisition and storage middleware 3 provides a real-time data distribution function by configuring the real-time data distribution interface 34, can automatically select a required data point location according to the requirements of other service systems, and after the acquisition and storage middleware finishes data acquisition of the corresponding point location, the data is temporarily not stored in the InfluxDB database, but is directly distributed to the corresponding data service system through the acquisition and storage middleware 3 through the data interface, so that the real-time performance of data transmission is ensured, and the large time delay caused by data storage and data discharge is avoided. The method can provide real-time data such as key operation parameters, equipment states, alarm information and the like for other service systems.
The number of apparatuses and the scale of the process described herein are intended to simplify the description of the present invention. Applications, modifications and variations of the present invention will be apparent to those skilled in the art. While embodiments of the invention have been described above, it is not limited to the applications set forth in the description and the embodiments, which are fully applicable in various fields of endeavor to which the invention pertains, and further modifications may readily be made by those skilled in the art, it being understood that the invention is not limited to the details shown and described herein without departing from the general concept defined by the appended claims and their equivalents.
Claims (10)
1. A wind tunnel real-time data efficient acquisition and storage system is characterized by comprising:
the distributed acquisition layer comprises a plurality of node servers which are respectively accessed to a plurality of wind tunnel data sources;
a multi-production multi-storage layer comprising a plurality of central servers;
the server side of the acquisition and storage middleware is respectively deployed on a plurality of central servers; the clients of the acquisition and storage middleware are respectively deployed on a plurality of node servers;
a distributed storage tier comprising a database farm connected to a plurality of central servers.
2. The wind tunnel real-time data efficient acquisition and storage system according to claim 1, wherein the plurality of wind tunnel data sources are three, namely a first data source, a second data source and a third data source; the first data source is control system data and is led into the node server in a PLC/PC-Base mapping mode; the second data source is measurement system data and is introduced into the node server in an NI variable engine/MQTT transmission mode; the third data source is health management system data which is imported into the node server in an OPC/DA transmission mode.
3. The wind tunnel real-time data efficient acquisition and storage system according to claim 2, wherein the number of said node servers is five, and the node servers are respectively two first node servers, two second node servers and a third node server; the number of the central servers is two, and the central servers are respectively a central main server and a central slave server; the two central servers are respectively connected with the five node servers, so that double data acquisition is formed; the first data source is respectively connected with the two first node servers; the second data source is respectively connected with the two second node servers; the third data source is connected with a third node server thereof; the node servers are interconnected by using a dedicated connection as a communication link, and are connected with a time synchronization server together.
4. The wind tunnel real-time data efficient acquisition and storage system according to claim 1, wherein the acquisition and storage middleware is developed by a multi-thread data processing program based on a C # framework, and a driver is issued in a DLL (delay locked loop) form to realize modularization of the program.
5. The wind tunnel real-time data efficient acquisition and storage system according to claim 1, wherein said acquisition and storage middleware comprises a data interface and an interface component; the data interface and interface component is integrated and configured with one or more of PC-base, wonderware database, NI shared variable, MQTT service, modbus TCP, OPC DA/UA, webAPI, socket industrial data driven protocol.
6. The wind tunnel real-time data efficient acquisition and storage system according to claim 1, wherein said acquisition and storage middleware comprises a point configuration component; the point configuration component is used for creating acquisition points connected with a plurality of wind tunnel data sources and realizing data acquisition and hierarchical management through a built-in logic grouping structure.
7. The wind tunnel real-time data efficient acquisition and storage system according to claim 1, wherein said acquisition and storage middleware comprises a data cache component; the data cache component comprises an InfluxDB cache database used for storing write-in failure data and a matched service program;
the concrete implementation flow of the service program is configured as follows: after the data is failed to be written into the database group by the acquisition and storage middleware, two kinds of information are generated, wherein one kind of information is a log and is directly stored in the local of the node server to form a log file; the other information is write failure data which is directly stored in an InfluxDB cache database; and reading the unprocessed log files at regular time, trying to rewrite the data into the database group when finding that the data fails to be written, deleting the log files or changing the state of the log files into processed state if the data is successfully written, and waiting for the next processing if the data is failed to be written.
8. The wind tunnel real-time data efficient acquisition and storage system according to claim 1, wherein said database groups are deployed in a server plus disk array manner; the database group comprises a master-slave time sequence database which is in butt joint with a plurality of central servers; the master-slave time sequence database comprises an InfluxDB master database and an InfluxDB slave database; the export data of the acquisition and storage middleware is simultaneously imported into an InfluxDB master database and an InfluxDB slave database; and the InfluxDB master database and the InfluxDB slave database are mutually connected to keep data synchronization so as to form data double writing.
9. The wind tunnel real-time data efficient acquisition and storage system according to claim 8, wherein said database group further comprises a SQL Server database; the SQL Server database is respectively connected with the InfluxDB master database and the InfluxDB slave database.
10. The wind tunnel real-time data efficient acquisition and storage system according to claim 1, wherein said acquisition and storage middleware is configured with a real-time data publishing interface; the real-time data publishing interface can be configured as one of MQTT and WebAPI.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211118807.6A CN115203213A (en) | 2022-09-15 | 2022-09-15 | Wind tunnel real-time data efficient acquisition and storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211118807.6A CN115203213A (en) | 2022-09-15 | 2022-09-15 | Wind tunnel real-time data efficient acquisition and storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115203213A true CN115203213A (en) | 2022-10-18 |
Family
ID=83573315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211118807.6A Pending CN115203213A (en) | 2022-09-15 | 2022-09-15 | Wind tunnel real-time data efficient acquisition and storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115203213A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118857665A (en) * | 2024-09-23 | 2024-10-29 | 中国空气动力研究与发展中心高速空气动力研究所 | Model data management system for wind tunnel tasks |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109857965A (en) * | 2018-12-29 | 2019-06-07 | 成都信息工程大学 | SOA-based Meteorological Service Product Release Server Control System and Method |
| CN113849483A (en) * | 2021-09-29 | 2021-12-28 | 中国兵器装备集团自动化研究所有限公司 | A real-time database system architecture for smart factories |
| CN114827140A (en) * | 2022-04-02 | 2022-07-29 | 中国兵器装备集团自动化研究所有限公司 | Real-time data centralized management and control system for wind tunnel site |
-
2022
- 2022-09-15 CN CN202211118807.6A patent/CN115203213A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109857965A (en) * | 2018-12-29 | 2019-06-07 | 成都信息工程大学 | SOA-based Meteorological Service Product Release Server Control System and Method |
| CN113849483A (en) * | 2021-09-29 | 2021-12-28 | 中国兵器装备集团自动化研究所有限公司 | A real-time database system architecture for smart factories |
| CN114827140A (en) * | 2022-04-02 | 2022-07-29 | 中国兵器装备集团自动化研究所有限公司 | Real-time data centralized management and control system for wind tunnel site |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118857665A (en) * | 2024-09-23 | 2024-10-29 | 中国空气动力研究与发展中心高速空气动力研究所 | Model data management system for wind tunnel tasks |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11258839B2 (en) | Data storage management and resource scaling | |
| US5968115A (en) | Complementary concurrent cooperative multi-processing multi-tasking processing system (C3M2) | |
| US8504523B2 (en) | Database management system | |
| US7860965B1 (en) | System and method for network traffic and I/O transaction monitoring of a high speed communications network | |
| US8266273B2 (en) | Realtime process history server | |
| US12216522B2 (en) | Methods and systems for expanding GPU memory footprint based on hybrid-memory | |
| Liu et al. | {Fine-Grained} Replicated State Machines for a Cluster Storage System | |
| CN115905413A (en) | A data synchronization platform based on Python coroutines and DataX | |
| US11995099B2 (en) | System and method for switching from consistent database to an eventual consistent database replica in real time while preventing reads of past versions of the data | |
| CN118377768A (en) | Data ETL method, device, equipment and medium based on service flow | |
| Cejka et al. | Java embedded storage for time series and meta data in Smart Grids | |
| CN115203213A (en) | Wind tunnel real-time data efficient acquisition and storage system | |
| CN113742393A (en) | Time sequence data processing system suitable for internet application | |
| CN116150263B (en) | Distributed graph calculation engine | |
| CN115562676B (en) | Triggering method of graph calculation engine | |
| CN117762495A (en) | Kafka-based asynchronous command processing method for realizing high concurrency and high availability | |
| CN117930737A (en) | Visual monitoring method and device for early warning of industrial production equipment | |
| US20250095100A1 (en) | Methods and systems for expanding gpu memory footprint based on hybrid-memory | |
| CN118132011B (en) | RAID card configuration information management method, system, device, equipment and medium | |
| CN120371548B (en) | Intelligent load balancing scheduling system and method based on redundant micro-service | |
| Singer et al. | The application of a database management system in an energy management system | |
| Sanghavi et al. | Goku: A Schemaless Time Series Database for Large Scale Monitoring at Pinterest | |
| CN118445364A (en) | Distributed resource synchronization method, system and device | |
| Hemminki | Large scale data ingestion pipelines for energy industry using AWS | |
| Tang et al. | High-Concurrency Data Service Model Based on MySQL, Redis, and MyCat |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20221018 |