CN114691688A - A kind of high-frequency market data inspection method and device - Google Patents

Abstract

The invention provides a high-frequency market data inspection method and device, comprising the following steps: the high-frequency market data inspection method comprises the following steps: step 102: acquiring high-frequency market data which comprises snapshot data, one-by-one consignation data, one-by-one transaction data and K-line market data; step 104: if the maximum service serial numbers under different channel codes in the entrusted data and the transaction data do not correspond to the total number of the entrusted data and the transaction data, judging that the high-frequency market data is wrong; step 106: and when the high-frequency quotation data are wrong, generating a wrong record and sending a mail to a corresponding receiver. The high-frequency market data inspection method and device provided by the invention can be used for inspecting whether the high-frequency market data acquired through different channels have the phenomena of missing, messy codes and the like, so that the data quality is improved, and the labor cost is greatly reduced.

Description

A kind of high-frequency market data inspection method and device

technical field

The invention relates to the technical field of data inspection, in particular to a method and device for inspecting high-frequency market data.

Background technique

In order to ensure the accuracy of the data during the transmission process, the data sender usually needs to perform a series of calculations on the data, obtain and introduce the corresponding check code, and append it to the end of the valid data. According to the agreement, the data receiver uses the algorithm function to recalculate the received data and generate the check code, and then compare it with the check code sent by the data sender. If the verification code comparison is correct, the data is submitted to the application for further processing; if the verification code comparison is incorrect, it is considered that there is a certain error in the data.

Currently in the prior art, the more widely used check codes mainly include OR check (Xor), cumulative sum check (Add), cyclic redundancy check (Cyclic Redundancy Check, CRC), cumulative sum and complement check (CheckSum) et al. The two inspection methods, Xor and Add, both use bytes as the unit to verify the data. The algorithm is relatively simple, but due to the single verification form, it is easy to Errors and omissions occur, and the accuracy of the data integrity test is not high. The cyclic redundancy check (Cyclic Redundancy Check, CRC) and the accumulation and complement check (CheckSum), which have a low error rate, have problems such as complicated algorithms and coding steps.

Therefore, it is necessary to provide a data verification method and device, which can solve the above problems.

SUMMARY OF THE INVENTION

Aiming at the problems and deficiencies existing in the prior art, the present invention provides a high-frequency market data inspection method and device, which can inspect whether the high-frequency market data obtained through different channels is missing, garbled and other phenomena, thereby improving data quality and greatly reducing manpower cost.

The present invention solves the above-mentioned technical problems through the following technical solutions:

The present invention provides a method for testing high-frequency market data, and the method for testing high-frequency market data includes:

Step 102: Acquire high-frequency market data, which includes snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data;

Step 104: If the maximum business serial numbers under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data do not correspond to the total number of the transaction-by-transaction data and the transaction-by-transaction data, determine the The above high-frequency market data is wrong;

Step 106: When the high-frequency market data is wrong, generate an error record and send an email to the corresponding recipient.

Preferably, before the step 102, the method further includes:

Step 101 : retrieve the daily code value table through an API interface, and retrieve the high-frequency market data according to the scene number corresponding to the daily code value table.

Preferably, in the step 101, retrieving the high-frequency market data according to the scene number corresponding to the daily code value table further includes obtaining the file size of the high-frequency market data every first preset time. The file size of the high-frequency market data obtained this time is the same as the file size of the high-frequency market data obtained last time, and the download of the high-frequency market data is completed. If the file size is different from the file size of the high-frequency market data obtained last time, the high-frequency market data will be obtained again at the next time interval until the second preset time.

Preferably, if the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, the previously downloaded high-frequency market data is automatically deleted, and the high-frequency market data is downloaded and repaired again. The latter data is obtained, and the high-frequency market data is divided into blocks.

Preferably, the obtaining of the high-frequency market data in the step 102 includes that if the high-frequency market data is successfully read, it means that the high-frequency market data is normal, and if the reading of the high-frequency market data fails, it means that the high-frequency market data is normal. The high-frequency market data is abnormal. When the high-frequency market data is abnormal, an error record is generated and an email is sent to the corresponding recipient.

Preferably, after the step 104, it further includes:

Step 105: Generate the current day K-line market data according to the transaction data, compare the current day’s K-line market data with the official K-line market data, if the current day’s K-line market data and the official K-line market data If there is no one-to-one correspondence, indicating that the transaction data is missing, an error record will be generated and an email will be sent to the corresponding recipient.

Preferably, the current day's K-line market data includes yesterday's closing price, opening price, latest price, highest price, lowest price, transaction volume, transaction volume and average price of the corresponding securities.

Preferably, the error records are stored in a high-frequency data daily inspection table, and the error records in the high-frequency data daily inspection table include ID, trading market, trading day, verification time, verification result, and error type. , Description of the error type, specific content of the error, release time, update time, and update ID.

Preferably, if the snapshot data and the K-line market data cannot correspond, indicating that the snapshot data cannot correspond to the daily market transaction amount, an error record is generated and an email is sent to the corresponding recipient.

The present invention also provides a high-frequency market data inspection device, and the high-frequency market data inspection device includes:

A data acquisition module, which is used to acquire high-frequency market data, including snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data;

A data judgment module, which is used for judging if the maximum business serial number under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data is different from the total number of the transaction-by-transaction data and the transaction-by-transaction data. Correspondingly, it is judged that the high-frequency market data is wrong;

A data error reporting module, which is used to generate an error record and send an email to a corresponding recipient when the high-frequency market data is incorrect.

Compared with the prior art, the technical solutions of the embodiments of the present invention have the following beneficial effects:

The method and device for verifying high-frequency market data provided by the embodiments of the present invention compare the maximum service serial numbers under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data with the transaction-by-transmission data and the transaction-by-transaction data. Whether the total number of transaction data corresponding to each transaction is to determine whether the high-frequency market data is accurate, and when the high-frequency market data is wrong, an error record is generated and an email is sent to the corresponding recipient, thereby reducing the technical difficulty. At the same time, it greatly improves the accuracy of high-frequency market data inspection;

Further, the K-line market data on the current day is generated according to the transaction data, and the K-line market data on the current day is compared with the official K-line market data. If the K-line market data on the current day and the official K-line market data are compared. If there is no one-to-one correspondence, indicating that the transaction data is missing, an error record is generated and an email is sent to the corresponding recipient, thereby further ensuring the correct rate of the high-frequency market data inspection;

Further, if the snapshot data and the K-line market data cannot correspond, indicating that the snapshot data cannot correspond to the daily market transaction amount, an error record will be generated and an email will be sent to the corresponding recipient, so that the relevant recipients can be notified in time. The recipient, further ensure the correct rate of the high-frequency market data inspection.

Description of drawings

1 is a schematic flowchart of a high-frequency market data inspection method provided by an embodiment of the present invention;

2 is a schematic flowchart of a method for checking high-frequency market data provided by another embodiment of the present invention;

FIG. 3 is a schematic structural diagram of an apparatus for checking high-frequency market data provided by an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention.

The technical solutions of the present invention will be described in detail below with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

Based on the problems existing in the prior art, as shown in FIG. 1 , the present invention provides a method for testing high-frequency market data. The method for testing high-frequency market data includes:

Step 102: Acquire high-frequency market data, which includes snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data;

Step 104: If the maximum business serial numbers under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data do not correspond to the total number of the transaction-by-transaction data and the transaction-by-transaction data, determine the The above high-frequency market data is wrong;

Step 106: When the high-frequency market data is wrong, generate an error record and send an email to the corresponding recipient.

In step 102, the snapshot data includes the accumulated transaction amount, transaction volume and the last latest transaction price of the exchange.

In step 104, the business serial number is a field set by the exchange to facilitate order chronological ordering. The transaction-by-transaction data and the transaction-by-transaction data are sequentially arranged in the order of channel codes, and the service serial number in the transaction-by-transaction data will not appear repeatedly in the service sequence number in the transaction-by-transaction data. . Therefore, the maximum business serial number under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data should correspond to the total number of the transaction-by-transaction data and the transaction-by-transaction data. There is no missing phenomenon in the high-frequency market data. If there is a mismatch, it is determined that the high-frequency market data is wrong, and an error record needs to be generated and an email sent to the corresponding recipient.

For example, if the high-frequency market data is missing on several different trading days, an error record is generated: the data type is the order-by-transaction data and the transaction-by-transaction data, and the verification result is 0, indicating that the high-frequency market data is wrong In the case of , the error type is described as: the data of each order of entrustment and the data of each transaction are missing. At the same time, send an email to the corresponding recipient, and perform corresponding processing on the high-frequency market data. And, the updated data is automatically downloaded, and the verification process of step 104 is repeated.

In a specific implementation, if the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, the previously downloaded high-frequency market data will be automatically deleted, and the new high-frequency market data will be automatically deleted. The repaired data is downloaded, and the high-frequency market data is divided into blocks. Segmenting the high-frequency market data includes dividing the high-frequency market data into bond data, fund data, and stock data according to the security type of each security, so as to facilitate investors to obtain desired data. In a specific implementation, the obtaining of the high-frequency market data in the step 102 includes that if the high-frequency market data is successfully read, it indicates that the high-frequency market data is normal, and if the reading fails, the high-frequency market data indicates that the high-frequency market data is normal. The high-frequency market data is abnormal. When the high-frequency market data is abnormal, an error record is generated and an email is sent to the corresponding recipient.

For example, if the high-frequency market data is wrong in several different trading days, an error record is generated: the data type is the order-by-transaction data, and the verification result is 0, indicating that the high-frequency market data is wrong, and the error type is reported. Description: The file contains garbled characters. At the same time, send an email to the corresponding recipient, and perform corresponding processing on the high-frequency market data.

Referring now to FIG. 2 , the present invention provides a method for checking high-frequency market data, and the method for checking high-frequency market data includes:

Step 101: retrieve the daily code value table through the API interface, and retrieve the high-frequency market data according to the scene number corresponding to the daily code value table;

Step 102: Acquire high-frequency market data, which includes snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data;

Step 104: If the maximum business serial numbers under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data do not correspond to the total number of the transaction-by-transaction data and the transaction-by-transaction data, determine the The above high-frequency market data is wrong;

Step 105: Generate the current day K-line market data according to the transaction data, compare the current day’s K-line market data with the official K-line market data, if the current day’s K-line market data and the official K-line market data If there is no one-to-one correspondence, indicating that the transaction data is missing, an error record will be generated and an email will be sent to the corresponding recipient;

Step 106: When the high-frequency market data is wrong, generate an error record and send an email to the corresponding recipient.

In a specific implementation, the daily code value table includes information such as the securities official code, securities abbreviation, securities type, and trading market of the exchange, and the code value specifically refers to the securities official code. The scene numbers include snapshot data, transaction-by-transaction data, transaction-by-transaction commission data, and K-line market data, which are different scene numbers. In the step 101, retrieving the high-frequency market data according to the scene number corresponding to the daily code value table further includes obtaining the file size of the high-frequency market data every first preset time. The file size of the high-frequency market data is the same as the file size of the high-frequency market data obtained last time, and the download of the high-frequency market data is completed. If the file size of the high-frequency market data obtained this time and If the file sizes of the high-frequency market data obtained last time are different, the high-frequency market data will be obtained again at the next time interval until the second preset time.

The first preset time is usually 1 hour, and the second preset time is usually 23 o'clock. If there is no high-frequency market data that meets the above 17:00 acquisition requirements until 23:00, the second preset time of the day, it is considered that there is no high-frequency market data generated on that day. If no high-frequency market data consistent with the previous size is obtained, it is considered that there is no accurate high-frequency market data generated on that day.

For example, the first high-frequency market data is obtained at 17:00 every day. After a batch of data is obtained, the first preset time is 1 hour later, and the high-frequency market data is obtained again according to the first preset time, that is, 18:00. The high-frequency market data is obtained again after an interval of the first preset time for 1 hour, according to the first preset time, that is, 19:00, and the high-frequency market data is obtained again, and so on, until the second preset time is 23:00. High-frequency market data.

In step 105, the current day's K-line market data includes yesterday's closing price, opening price, latest price, highest price, lowest price, transaction volume, transaction volume, and average price of the corresponding securities. If the official K-line market data is missing, the error record will be stored in the high-frequency data daily inspection table, and an email will be sent to the corresponding recipient.

For example, there are high-frequency market data in many trading markets, but the official K-line market data is missing, and an error record is generated: the data type is K-line market data, and the verification result is 0, indicating that the high-frequency market data is wrong. In the case of , the error type is described as: high-frequency market data exists, but the official K-line market data is missing. At the same time, send an email to the corresponding recipient, and perform corresponding processing on the high-frequency market data.

In a specific implementation, the error records are stored in the high-frequency data daily inspection table, and the error records in the high-frequency data daily inspection table include ID, trading market, trading day, verification time, verification result, Error type, description of error type, specific content of error, release time, update time, and update ID. The details are shown in the following table:

Monitoring information Notes Monitoring information Notes ID ID ErrorType Error type ExchangeCode market place ErrorDesc Error type description FileType file type Details Error specific content TradingDay trading day InsertTime release time CheckTime Check time UpdateTime update time Result Check result UPID update id

In a specific implementation, if the snapshot data cannot correspond to the K-line market data, indicating that the snapshot data cannot correspond to the daily market transaction amount, an error record is generated and an email is sent to the corresponding recipient.

For example, if the snapshot data and the K-line market data cannot be matched in multiple trading markets, an error record is generated: the data type is transaction-by-transaction data and transaction-by-transaction data, and a verification result of 0 indicates a high-frequency market. If the data is wrong, the error type is described as: the snapshot data cannot correspond to the K-line market data. At the same time, send an email to the corresponding recipient, and perform corresponding processing on the high-frequency market data.

Based on the problems existing in the prior art, as shown in FIG. 3 , the present invention also provides a high-frequency market data inspection device 30, and the high-frequency market data inspection device 30 includes:

A data acquisition module 301, which is used to acquire high-frequency market data, including snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data;

The data judgment module 302 is used for judging if the maximum business serial number under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data is equal to the total number of the transaction-by-transaction data and the transaction-by-transaction data. If not, it is judged that the high-frequency market data is wrong;

The data error reporting module 303 is used to generate an error record and send an email to the corresponding recipient when the high-frequency market data is wrong.

To sum up, the method and device for verifying high-frequency market data provided by the embodiments of the present invention compare the maximum service serial numbers under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data with the transaction-by-transaction data. Whether the entrustment data corresponds to the total number of transaction data one by one, to judge whether the high-frequency market data is accurate, when the high-frequency market data is wrong, generate an error record and send an email to the corresponding recipient , so as to greatly improve the accuracy of high-frequency market data inspection while reducing the technical difficulty;

Further, the K-line market data on the current day is generated according to the transaction data, and the K-line market data on the current day is compared with the official K-line market data. If the K-line market data on the current day and the official K-line market data are compared. If there is no one-to-one correspondence, indicating that the transaction data is missing, an error record is generated and an email is sent to the corresponding recipient, thereby further ensuring the correct rate of the high-frequency market data inspection;

Further, if the snapshot data and the K-line market data cannot correspond, indicating that the snapshot data cannot correspond to the daily market transaction amount, an error record will be generated and an email will be sent to the corresponding recipient, so that the relevant recipients can be notified in time. The recipient, further ensure the correct rate of the high-frequency market data inspection.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present invention. scope.

Claims (10)

1. a high-frequency market data inspection method, is characterized in that, described high-frequency market data inspection method comprises: Step 102: Acquire high-frequency market data, which includes snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data; Step 104: If the maximum business serial numbers under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data do not correspond to the total number of the transaction-by-transaction data and the transaction-by-transaction data, determine the The above high-frequency market data is wrong; Step 106: When the high-frequency market data is wrong, generate an error record and send an email to the corresponding recipient. 2. The high-frequency market data inspection method according to claim 1, characterized in that, before the step 102, further comprising: Step 101 : retrieve the daily code value table through an API interface, and retrieve the high-frequency market data according to the scene number corresponding to the daily code value table. 3. high-frequency market data inspection method according to claim 2, is characterized in that, in described step 101, according to described daily code value table corresponding scene number fetching described high-frequency market data also comprises every interval first. Obtain the file size of the high-frequency market data once at a preset time. If the file size of the high-frequency market data obtained this time is the same as the file size of the high-frequency market data obtained last time, the high-frequency market data The market data download is completed. If the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, wait for the next time interval to obtain the high-frequency market data again. data until the second preset time. 4. high-frequency market data inspection method according to claim 3, is characterized in that, if the file size of the described high-frequency market data obtained this time is different from the file size of the described high-frequency market data obtained last time , the previously downloaded high-frequency market data is automatically deleted, the restored data is re-downloaded, and the high-frequency market data is divided into blocks. 5. The high-frequency market data inspection method according to claim 1, wherein the obtaining of the high-frequency market data in the step 102 comprises that if the high-frequency market data is successfully read, it represents the high-frequency market data. The data is normal. If the high-frequency market data fails to be read, it means that the high-frequency market data is abnormal. When the high-frequency market data is abnormal, an error record is generated and an email is sent to the corresponding recipient. 6. The high-frequency market data inspection method according to claim 1, characterized in that, after the step 104, further comprising: Step 105: Generate the current day K-line market data according to the transaction data, compare the current day’s K-line market data with the official K-line market data, if the current day’s K-line market data and the official K-line market data If there is no one-to-one correspondence, indicating that the transaction data is missing, an error record will be generated and an email will be sent to the corresponding recipient. 7. The high-frequency market data inspection method according to claim 6, wherein the K-line market data of the day comprises yesterday's closing price, opening price, latest price, highest price, lowest price, turnover, Volume and average price. 8. high-frequency market data inspection method according to claim 1 or 5 or 6, is characterized in that, described error record is stored in high-frequency data daily inspection table, described in the high-frequency data daily inspection table The error record includes ID, trading market, trading day, verification time, verification result, error type, description of the error type, specific content of the error, release time, update time and update ID. 9. The high-frequency market data inspection method according to claim 1, wherein, if the snapshot data and the K-line market data cannot correspond, indicating that the snapshot data and the daily market transaction amount cannot correspond, then generate. Errors are logged and mail is sent to the appropriate recipients. 10. A high-frequency market data inspection device, characterized in that the high-frequency market data inspection device comprises: A data acquisition module, which is used to acquire high-frequency market data, including snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data; The data judgment module is used to judge if the maximum business serial number under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data is different from the total number of the transaction-by-transaction data and the transaction-by-transaction data. Correspondingly, it is judged that the high-frequency market data is wrong; A data error reporting module, which is used to generate an error record and send an email to the corresponding recipient when the high-frequency market data is incorrect.

Images