Disclosure of Invention
The technical problem to be solved by the present invention is to overcome the defects in the prior art, and provide a distributed wave recording method and device for a power distribution network dynamic simulation system, so as to implement extension of the number of wave recorders, implement verification of message transmission correctness, and reduce the pressure of network transmission and data processing.
Therefore, the invention adopts the following technical scheme: a distributed wave recording method of a power distribution network dynamic simulation system comprises the following steps:
1) the server reads the local configuration file, obtains the number of the wave recorders to be connected and the IP address of each wave recorder, and the port number of each wave recorder is fixed;
2) the server is actively connected with the wave recorder, sends an instruction for reading the cfg configuration file after the connection is successful, and caches the cfg configuration file locally after the server receives the configuration file information;
3) initializing a UDP socket, binding a random port, and sending a port number to a wave recorder through a TCP/IP, wherein the corresponding wave recorder sends data to the port;
4) after the continuous wave recording starts, the wave recorder sends wave recording data to a designated UDP port, the server analyzes the message, judges whether the message is normal or not, and if not, the reading and writing of the file are finished;
5) judging the format of the message, wherein the first four bytes of the message are a packet number, the second two bytes are data length, the content of a data area is arranged behind the packet number, and the last byte is an ending identifier; if the packet number continuously indicates that the data transmission is normal, and if the packet number is discontinuous, the packet loss phenomenon is indicated, the caching and writing operation of the wave recording data of the section is finished, the generated cfg file is deleted, and the error record is written into a local log file;
6) when the package number is zero, the data area contains the line frequency, the sampling rate, the first point time and the triggering time information, the corresponding position in the cfg file is replaced by the received data, and the cfg file is named according to the triggering time plus the id number of the wave recorder;
7) when the packet number is larger than zero, the data in the data area is sampling data, the data area is divided into analog quantity data and switching value data, and the analog quantity, the number of switching value channels, the id number of the wave recorder and the storage format of the wave recording file are all acquired through the cfg file;
combining the sampled data into a wave recording file format and writing the wave recording file format into a buffer area; and after the wave recording is finished, the data is quickly written into the dat file in a memory mapping mode.
The invention relates to a high-concurrency processing mechanism, a data cache, high-speed data reading and writing and other technologies, and aims to realize that when a plurality of wave recorders carry out continuous high-speed recording simultaneously, wave recording data are accurately transmitted to a server through a gigabit switch, the server calculates the data to obtain a specified data format and places the specified data format in a buffer area, and after the wave recording of the current segment is finished, the data in the buffer area are stored in a specified dat file. Based on a data transmission architecture of network communication, cfg file information and wave recording data are transmitted through TCP/IP and UDP, and the cfg file is transmitted only once after connection is successful.
The invention can realize the expansion of the number of the wave recorders, the data throughput can reach 30MB/s, and the file reading and writing speed can reach 50 MB/s; the method realizes the verification of the message transmission correctness and reduces the pressure of network transmission and data processing.
As a supplement to the above-mentioned recording method, in step 2), when the command is sent, the format of the message is 0x68+ length (double bytes) + command type + data + crc parity +0x 16.
As a supplement to the above-mentioned recording method, in step 2), when the configuration file information is cached locally, for example, the line frequency, the sampling rate, the first point time and the trigger time in the configuration file are missing, the information needs to be completed by the post-operation.
As a supplement to the above-mentioned recording method, in step 7), each recorder creates two memory spaces with the same size on the server, and in a certain recording time period, the server writes the data in the recording storage format into the buffer, when the transmission end flag bit is 1, the transmission in this time period ends, and directly maps the file into the buffer by using a memory mapping method, and after the recording in the next time period starts, the recording data is written into another buffer, and the two memory spaces perform alternate read-write operations, thereby separating the read-write tasks. And a double-memory cache mechanism is adopted to separate data cache from file write-in, so that the accuracy of data write-in is improved, the response time of a system is reduced, and the throughput of the system is improved.
As a supplement to the above-mentioned recording method, the server starts a new process, monitors the size of the hard disk memory in real time, and deletes a certain number of files according to the file establishment sequence when the remaining memory is lower than a certain set value.
As a supplement to the wave recording method, an FTP server is built on the server, the path where the wave recording file is located is set as a physical path, and in a local area network, application software can call the wave recording file through the FTP.
The other technical scheme adopted by the invention is as follows: a distributed wave recording device of a power distribution network dynamic simulation system comprises:
a recorder IP information acquisition unit: the server reads the local configuration file, obtains the number of the wave recorders to be connected and the IP address of each wave recorder, and the port number of each wave recorder is fixed;
wave recorder connecting element: the server is actively connected with the wave recorder;
a configuration file transmitting unit: after the connection is successful, sending an instruction for reading the cfg configuration file, and after the server receives the configuration file information, caching the configuration file information locally;
a UDP port sending unit: initializing a UDP socket, binding a random port, and sending a port number to a wave recorder through a TCP/IP, wherein the corresponding wave recorder sends data to the port;
a message analysis unit: after the continuous wave recording starts, the wave recorder sends wave recording data to a designated UDP port, the server analyzes the message, judges whether the message is normal or not, and if not, the reading and writing of the file are finished;
judging the message format and processing unit: the first four bytes of the message are packet numbers, the last two bytes are data length, the last byte is data area content, and the last byte is an ending identifier; if the packet number continuously indicates that the data transmission is normal, and if the packet number is discontinuous, the packet loss phenomenon is indicated, the caching and writing operation of the wave recording data of the section is finished, the generated cfg file is deleted, and the error record is written into a local log file;
the first data processing unit: when the package number is zero, the data area contains the line frequency, the sampling rate, the first point time and the triggering time information, the corresponding position in the cfg file is replaced by the received data, and the cfg file is named according to the triggering time plus the id number of the wave recorder;
a second data processing unit: when the packet number is larger than zero, the data in the data area is sampling data, the data area is divided into analog quantity data and switching value data, and the analog quantity, the number of switching value channels, the id number of the wave recorder and the storage format of the wave recording file are all acquired through the cfg file; combining the sampled data into a wave recording file format and writing the wave recording file format into a buffer area;
a data writing unit: and after the wave recording is finished, the data is quickly written into the dat file in a memory mapping mode.
As a supplement to the above-mentioned wave recording device, when the wave recorder connection unit sends the command, the message adopts a format of 0x68+ length (double bytes) + command type + data + crc check bit +0x 16.
As a supplement to the above-mentioned wave recording device, when the configuration file information is cached locally in the wave recorder connection unit, for example, the line frequency, the sampling rate, the first point time and the trigger time in the configuration file are missing, the information needs to be completed by the post-operation.
As a supplement to the above-mentioned wave recording device, in the data writing unit, each wave recorder opens up two memory spaces with the same size on the server, in a certain wave recording time period, the server writes the data in the wave recording storage format into the buffer, when the transmission end flag bit is 1, the transmission in this time period ends, the file is directly mapped to the buffer by using a memory mapping method, after the wave recording in the next time period starts, the wave recording data is written into another buffer, and the two memory spaces perform alternate read-write operations, thereby separating the read-write tasks.
The invention has the advantages that:
1. the invention can conveniently realize the expansion of the system through the configuration file, and only needs to add the port information of the wave recorder in the configuration file when in use.
2. The transmission paths of the cfg file and the dat file are separated, the configuration file information is transmitted through the TCP/IP, the wave recording data is transmitted through the UDP, the configuration file information is transmitted only once after the TCP is connected, the UDP data is transmitted only after continuous wave recording, and the pressure of network transmission and data processing is reduced.
3. The invention can realize the verification of the message transmission correctness by designing the message, judges whether the data is lost or not continuously by checking the packet number, abandons the wave recording data of the segment if the data is lost, and avoids the error of the application software when calling the file.
4. The server opens up two memory spaces, and the two memories alternately perform data caching and file writing, so that the data caching and the file writing can be separated, the accuracy of the data written in the file is ensured, the throughput of the system is improved, and the system does not need to copy the data for multiple times due to the adoption of the memory mapping file writing, so that the response time after the data is received is reduced.
Detailed Description
The invention is further described with reference to the drawings and the detailed description.
Example 1
The embodiment provides a distributed wave recording method for a power distribution network dynamic simulation system, as shown in fig. 1, the steps are as follows:
1. after the server is started, the local configuration file is read, the number of the wave recorders to be connected and the IP address of each wave recorder are obtained, the port number of each wave recorder is fixed, and the IP address and the port number of each wave recorder are configured by the maintenance software of the wave recorder.
2. The server is actively connected with the wave recorder, after the connection is successful, an instruction for reading the cfg configuration file is sent, the message adopts a format of 0x68+ length (double bytes) + instruction type + data + crc check bit +0x16, after the server receives the configuration file information, the cache is carried out locally, and if the line frequency, the sampling rate, the first point time and the trigger time in the configuration file are missing, the information needs to be completed through later operation.
3. Initializing the UDP socket, binding a random port, sending the port number to the wave recorder through the TCP/IP, and sending data to the port by the corresponding wave recorder.
4. After the continuous wave recording starts, the wave recorder sends the wave recording data to the designated UDP port, the server analyzes the message and judges whether the message is normal, otherwise, the reading and writing of the file are finished, and the specific analysis and processing process is shown in FIG. 2.
5. Judging the format of the message, wherein the first four bytes of the message are a packet number, the second two bytes are data length, the last byte is data area content, the last byte is an ending identifier, if the packet number continuously indicates that the data transmission is normal, and if the packet number is discontinuous, the packet loss phenomenon is indicated, ending the caching and writing operation of the wave recording data, deleting the generated cfg file, and writing the error record into a local log file, thereby facilitating the inspection of a tester.
6. When the package number is zero, the data area contains the line frequency, the sampling rate, the first point time and the triggering time information, the corresponding position in the cfg file is replaced by the received data, and the cfg file is named according to the mode of triggering time + the id number of the wave recorder.
7. When the packet number is larger than zero, the data in the data area is sampling data, the data area is divided into analog quantity data and switching value data, and the analog quantity, the number of switching value channels, the id number of the wave recorder, the storage format of the wave recorder file and other information can pass through the cfg file; combining the sampled data into a wave recording file format and writing the wave recording file format into a buffer area; and after the wave recording is finished, the data is quickly written into the dat file in a memory mapping mode.
Each wave recorder opens up two memory spaces with the same size on a server, the server writes data in a wave recording storage format into the buffer area in a certain wave recording time period, when the transmission ending flag bit is 1, the transmission in the time period is ended, the file is directly mapped to the buffer area by adopting a memory mapping method, the file writing efficiency of a system is improved, after the wave recording in the next time period is started, the wave recording data is written into the other buffer area, and the two memory spaces perform alternate read-write operation, so that the read-write tasks are separated.
Because the memory space of the system hard disk is limited, the wave recorder needs to record waves continuously for 24 hours, if the size of the hard disk space of the system is 7T, the number of the wave recorders is 8, the number of analog quantity channels is 8, the number of switching quantity channels is 8, and the maximum sampling rate of the wave recorder is 100K, the highest writing rate of the dat file can reach 20.8M/s. The hard disk is full after recording for 90 continuous hours, so a new process is started in the server, the size of the hard disk memory is monitored in real time, and a certain number of files are deleted according to the file establishment sequence when the residual memory is lower than a certain set value.
And (3) building an FTP server on the server, setting the path of the wave recording file as a physical path, and calling the wave recording file by application software through the FTP in the local area network.
Example 2
The embodiment provides a distributed wave recording device for a power distribution network moving die system, which includes a wave recorder IP information obtaining unit, a wave recorder connection unit, a configuration file sending unit, a UDP port sending unit, a message parsing unit, a judgment message format judging and processing unit, a first data processing unit, a second data processing unit, and a data writing unit, as shown in fig. 3.
A recorder IP information acquisition unit: and the server reads the local configuration file, acquires the number of the wave recorders to be connected and the IP address of each wave recorder, and the port number of the wave recorder is fixed. And the IP address and the port number of the wave recorder are configured by the maintenance software of the wave recorder.
Wave recorder connecting element: the server is actively connected with the wave recorder.
A configuration file transmitting unit: after connection is successful, an instruction for reading the cfg configuration file is sent, the message adopts a format of 0x68+ length (double bytes) + instruction type + data + crc check bit +0x16, after the server receives the configuration file information, the server caches the configuration file information locally, and if the line frequency, the sampling rate, the first point time and the trigger time in the configuration file are missing, the information needs to be completed through operation after the completion.
A UDP port sending unit: initializing the UDP socket, binding a random port, sending the port number to the wave recorder through the TCP/IP, and sending data to the port by the corresponding wave recorder.
A message analysis unit: and after the continuous wave recording starts, the wave recorder sends the wave recording data to a specified UDP port, the server analyzes the message, judges whether the message is normal or not, and if not, the reading and writing of the file are finished.
Judging the message format and processing unit: the first four bytes of the message are packet numbers, the last two bytes are data length, the last byte is data area content, and the last byte is an ending identifier; if the packet number continuously indicates that the data transmission is normal, and if the packet number is discontinuous, the packet loss phenomenon is indicated, the caching and writing operation of the wave recording data of the section is ended, the generated cfg file is deleted, and the error record is written into the local log file.
The first data processing unit: when the package number is zero, the data area contains the line frequency, the sampling rate, the first point time and the triggering time information, the corresponding position in the cfg file is replaced by the received data, and the cfg file is named according to the triggering time plus the id number of the wave recorder.
A second data processing unit: when the packet number is larger than zero, the data in the data area is sampling data, the data area is divided into analog quantity data and switching value data, and the analog quantity, the number of switching value channels, the id number of the wave recorder and the storage format of the wave recording file are all acquired through the cfg file; and combining the sampled data into a recording file format and writing the recording file format into a buffer area.
A data writing unit: after the wave recording of the segment is finished, quickly writing data into a dat file in a memory mapping mode; each wave recorder opens up two memory spaces with the same size on a server, the server writes data in a wave recording storage format into the buffer area in a certain wave recording time period, when the transmission ending flag bit is 1, the transmission in the time period is ended, a file is directly mapped to the buffer area by adopting a memory mapping method, after wave recording in the next time period is started, wave recording data are written into the other buffer area, and alternate read-write operation is carried out on the two memory spaces, so that read-write tasks are separated.
According to the invention, through UDP data transmission, the data transmission throughput is increased, the double-cache alternate read-write and file memory mapping technologies are adopted, the problem that the processing speed of a network and a hard disk is not timely is effectively solved, and the distributed wave recording method of the power distribution network dynamic system is realized.
The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.