[go: up one dir, main page]

CN112749868A - Data abnormity monitoring method and device - Google Patents

Data abnormity monitoring method and device Download PDF

Info

Publication number
CN112749868A
CN112749868A CN201911063024.0A CN201911063024A CN112749868A CN 112749868 A CN112749868 A CN 112749868A CN 201911063024 A CN201911063024 A CN 201911063024A CN 112749868 A CN112749868 A CN 112749868A
Authority
CN
China
Prior art keywords
data
calculation
service
monitoring
dimensions
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911063024.0A
Other languages
Chinese (zh)
Inventor
张玉岩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN201911063024.0A priority Critical patent/CN112749868A/en
Publication of CN112749868A publication Critical patent/CN112749868A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0635Risk analysis of enterprise or organisation activities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0609Buyer or seller confidence or verification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/02Banking, e.g. interest calculation or account maintenance

Landscapes

  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Finance (AREA)
  • Human Resources & Organizations (AREA)
  • Accounting & Taxation (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Development Economics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Educational Administration (AREA)
  • Tourism & Hospitality (AREA)
  • Quality & Reliability (AREA)
  • Operations Research (AREA)
  • Game Theory and Decision Science (AREA)
  • Technology Law (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a method and a device for monitoring data abnormity, and relates to the technical field of computers. One embodiment of the method comprises: according to the data attribute of the service data, calculating direction distribution is carried out on the service data so as to obtain first calculating direction data and second calculating direction data; performing multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data by using a preset analysis model; and generating monitoring suggestion information of the service data according to the analysis operation results of the multiple dimensions, wherein the monitoring suggestion information indicates that corresponding operation is executed on the service data. According to the embodiment, the abnormal condition of data interaction between the enterprise platform and the business side can be automatically monitored in real time, the monitoring error rate is reduced, and the abnormal condition can be responded to in the first time, so that the abnormal condition can be timely processed, and the enterprise loss is reduced.

Description

Data abnormity monitoring method and device
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for monitoring data abnormity.
Background
Data interaction between an enterprise platform and a business party is usually bidirectional, for example, in the process of settlement between a merchant and a platform service provider on an e-commerce platform (platform for short), accounts receivable and payable in two directions exist in the angle of the platform service provider, and the accounts receivable include warranty paid by the merchant, platform use fee, commission and the like; the accounts payable, such as the goods payment of the order collected by the platform, needs to be settled to the merchant in real time through the third party payment platform. The platform service provider determines the receivable items to be paid to the merchant, and the payment result is transmitted back to the platform financial system; the platform service provider can also form a payment bill with the due payment, and the payment bill is returned to the platform financial system after being settled by the third-party payment system.
In the platform merchant, a malicious hacker merchant may exist, collects payable accounts of the platform service provider by simulating order placing requests of normal consumers in batches, and then simulates return requests of the normal consumers in batches, so that the platform service provider is collected to pay the return fee.
In the existing settlement system of the e-commerce platform, accounts receivable and accounts payable of the platform are often settled independently, malicious cash register behaviors cannot be monitored in real time under the condition of lacking the dimensionality of a merchant, manual monitoring is relied on, errors are prone to occurring, a refund bill receivable bill cannot be timely notified to a third party payment system so as to be brought up for cash and intercepted, information is delayed, and serious benefit loss is caused to a platform service provider.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
for abnormal conditions of data interaction between an enterprise platform and a business party, automatic real-time monitoring cannot be achieved, the monitoring error rate is high, information is delayed, abnormality cannot be timely processed, and serious loss is caused to enterprises.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for monitoring data exception, which can automatically monitor an exception condition of data interaction between an enterprise platform and a service party in real time, reduce a monitoring error rate, and respond to the exception in the first time, thereby ensuring that the exception condition is handled in time and reducing enterprise loss.
To achieve the above object, according to an aspect of an embodiment of the present invention, a method for monitoring data abnormality is provided.
A method for monitoring data abnormity comprises the following steps: according to the data attribute of the service data, calculating direction distribution is carried out on the service data so as to obtain first calculating direction data and second calculating direction data; performing multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data by using a preset analysis model; and generating monitoring suggestion information of the service data according to the analysis operation results of the multiple dimensions, wherein the monitoring suggestion information indicates that corresponding operation is executed on the service data.
Optionally, the step of performing calculation direction allocation on the service data according to a data attribute of the service data to obtain first calculation direction data and second calculation direction data includes: extracting incremental data of the business data from a plurality of data sources, and performing calculation direction distribution on the incremental data according to data attributes of the incremental data to obtain the first calculation direction data and the second calculation direction data.
Optionally, the step of performing a multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data by using a preset analysis model includes: extracting calculation factors from the first calculation direction data and the second calculation direction data according to the plurality of dimensions respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions; inputting the first direction calculation factor and the second direction calculation factor with the same dimension into the preset analysis model for analysis and calculation to respectively obtain an incremental calculation result under each dimension; and respectively adding the incremental operation result of each dimension with the last analysis operation result of the corresponding dimension to obtain the analysis operation results of the plurality of dimensions.
Optionally, the calculation direction allocation is performed according to the service code in the data attribute and the service direction corresponding to the service code.
Optionally, the step of performing a multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data by using a preset analysis model includes: extracting calculation factors from the first calculation direction data and the second calculation direction data according to the plurality of dimensions respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions; and inputting the first direction calculation factor and the second direction calculation factor with the same dimension into the preset analysis model for analysis and calculation to respectively obtain an analysis and calculation result under each dimension.
Optionally, the step of generating monitoring suggestion information for the service data according to the analysis operation result of the multiple dimensions includes: comparing the analysis operation results of the plurality of dimensions with the threshold values of the corresponding dimensions; if the analysis operation results of the multiple dimensions are all smaller than the threshold value of the corresponding dimension, generating suggestion information of data freezing; if one or N analysis operation results in the analysis operation results of the multiple dimensions are smaller than the threshold value of the corresponding dimension, generating suggestion information of data early warning, wherein N is smaller than the total number of the dimensions; and if the analysis operation results of the plurality of dimensions are all larger than or equal to the threshold value of the corresponding dimension, generating suggestion information with normal data.
Optionally, before performing calculation direction allocation on the service data according to the data attribute of the service data, the method further includes: and filtering out the service data of which at least one field value does not accord with a preset condition according to three field values of a data source, a service code and a service direction corresponding to the service code in the data attribute.
According to another aspect of the embodiments of the present invention, a device for monitoring data abnormality is provided.
A data anomaly monitoring device, comprising: the calculation direction distribution module is used for performing calculation direction distribution on the service data according to the data attribute of the service data to obtain first calculation direction data and second calculation direction data; the analysis operation module is used for carrying out multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data by using a preset analysis model; and the monitoring suggestion information generating module is used for generating monitoring suggestion information of the service data according to the analysis operation results of the multiple dimensions, and the monitoring suggestion information indicates to execute corresponding operation on the service data.
Optionally, the calculation direction assignment module is further configured to: extracting incremental data of the business data from a plurality of data sources, and performing calculation direction distribution on the incremental data according to data attributes of the incremental data to obtain the first calculation direction data and the second calculation direction data.
Optionally, the analysis operation module is further configured to: extracting calculation factors from the first calculation direction data and the second calculation direction data according to the plurality of dimensions respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions; inputting the first direction calculation factor and the second direction calculation factor with the same dimension into the preset analysis model for analysis and calculation to respectively obtain an incremental calculation result under each dimension; and respectively adding the incremental operation result of each dimension with the last analysis operation result of the corresponding dimension to obtain the analysis operation results of the plurality of dimensions.
Optionally, the calculation direction allocation module performs the calculation direction allocation according to the service code in the data attribute and the service direction corresponding to the service code.
Optionally, the analysis operation module is further configured to: extracting calculation factors from the first calculation direction data and the second calculation direction data according to the plurality of dimensions respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions; and inputting the first direction calculation factor and the second direction calculation factor with the same dimension into the preset analysis model for analysis and calculation to respectively obtain an analysis and calculation result under each dimension.
Optionally, the monitoring suggestion information generation module is further configured to: comparing the analysis operation results of the plurality of dimensions with the threshold values of the corresponding dimensions; if the analysis operation results of the multiple dimensions are all smaller than the threshold value of the corresponding dimension, generating suggestion information of data freezing; if one or N analysis operation results in the analysis operation results of the multiple dimensions are smaller than the threshold value of the corresponding dimension, generating suggestion information of data early warning, wherein N is smaller than the total number of the dimensions; and if the analysis operation results of the plurality of dimensions are all larger than or equal to the threshold value of the corresponding dimension, generating suggestion information with normal data.
Optionally, the system further comprises a data filtering module, configured to: and filtering out the service data of which at least one field value does not accord with a preset condition according to three field values of a data source, a service code and a service direction corresponding to the service code in the data attribute.
According to yet another aspect of an embodiment of the present invention, an electronic device is provided.
An electronic device, comprising: one or more processors; the memory is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors realize the monitoring method for the data exception provided by the invention.
According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.
A computer-readable medium, on which a computer program is stored, which, when executed by a processor, implements the method of monitoring data anomalies provided by the present invention.
One embodiment of the above invention has the following advantages or benefits: the method comprises the steps of distributing calculation directions to business data according to data attributes of the business data, carrying out multi-dimensional analysis operation on first calculation direction data and second calculation direction data obtained by distributing the calculation directions by using a preset analysis model, and generating monitoring suggestion information according to analysis operation results of multiple dimensions so as to instruct to execute corresponding operation on the business data. The abnormal condition of data interaction between the enterprise platform and the business side can be automatically monitored in real time, the monitoring error rate is reduced, and the abnormal condition can be responded in the first time, so that the abnormal condition can be timely processed, and the enterprise loss is reduced.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main steps of a data anomaly monitoring method according to a first embodiment of the present invention;
FIG. 2 is a diagram illustrating the main steps of a method for monitoring data anomalies according to a second embodiment of the present invention;
FIG. 3 is a schematic flow chart illustrating the monitoring of data anomalies according to a third embodiment of the present invention;
FIG. 4 is a schematic diagram of a monitoring framework for data anomalies in accordance with a fourth embodiment of the present invention;
FIG. 5 is a schematic diagram of the main blocks of a data anomaly monitoring device according to a fifth embodiment of the present invention;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 7 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
Fig. 1 is a schematic diagram of the main steps of a data anomaly monitoring method according to a first embodiment of the present invention.
As shown in fig. 1, the method for monitoring data abnormality according to an embodiment of the present invention mainly includes the following steps S101 to S103.
Step S101: and according to the data attribute of the service data, calculating direction distribution is carried out on the service data so as to obtain first calculating direction data and second calculating direction data.
The service data is acquired from each upstream service system, after the service data is acquired, data attributes of the service data may be configured, where the data attributes may include a data source (i.e., a source system), a service code, a service direction corresponding to the service code, and the like, and may further include a service order state (e.g., a completed or unfinished state), a calculation state of the service order, and the like according to a monitoring requirement.
The calculation direction distribution can be performed according to the service codes in the data attributes and the service directions corresponding to the service codes.
Taking an example of data interaction between an e-commerce platform service provider and a merchant on an e-commerce platform (platform for short), the service data may be charging data, the service code may be a charge code (such as a charge ID), the service direction may be a charge direction corresponding to the charge code, the charging data may specifically be a charging detail in a service ticket, and one service ticket may correspond to multiple charging details. The charge code of the charging detail can be obtained from the service list, and the charging data can be distributed into charging pool data (first calculation direction data) which is receivable and charging pool data (second calculation direction data) which is payable according to the charge direction (such as receivable type charge or payable type charge) corresponding to the charge code.
As an optional implementation manner, before the service data is subjected to the calculation direction distribution according to the data attribute of the service data, at least one service data whose field value does not meet the preset condition may be filtered according to three field values of the data source, the service code, and the service direction corresponding to the service code in the data attribute. The service data that does not meet the preset condition is, for example: service data in which a field does not exist or a field value is empty.
Step S102: and carrying out multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data by using a preset analysis model.
Step S102 may specifically include: extracting calculation factors from the first calculation direction data and the second calculation direction data according to a plurality of dimensions respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions; and inputting the first direction calculation factor and the second direction calculation factor with the same dimension into a preset analysis model for analysis and operation to respectively obtain analysis and operation results under each dimension. The first direction calculation factor corresponding to a certain dimension is a calculation factor extracted from the first calculation direction data according to the dimension; the second direction calculation factor corresponding to a certain dimension is a calculation factor extracted from the second calculation direction data according to the dimension. The analysis operation performed by the preset analysis model may be a subtraction operation performed on the numerical sum of the second direction calculation factors and the numerical sum of the first direction calculation factors, and for the preset analysis model and the analysis operation manner thereof, the following detailed description of the inverse analysis model may be referred to.
Taking data interaction between an e-commerce platform service provider and a merchant on an e-commerce platform (platform for short) as an example, monitoring whether charging data between the e-commerce platform and the merchant is abnormal, wherein multiple dimensions may include: company dimensions, store dimensions, etc. A company corresponding to an e-commerce platform merchant may open a plurality of stores on the platform. And extracting calculation factors according to the company dimension and the shop dimension respectively according to the corresponding charging pool data and the charging pool data to obtain a factor to be received (namely a first direction calculation factor) and a factor to be paid (namely a second direction calculation factor) under the company dimension and a factor to be received and a factor to be paid under the shop dimension. The receivable factor and the payable factor are charging details corresponding to each business order, the receivable factor under the company dimension can be non-order related receivable charging details, the payable factor under the company dimension can be non-order related payable charging details, the receivable factor under the store dimension can be order related receivable charging details, and the payable factor under the store dimension can be order related payable charging details.
In the aspect of the e-commerce platform, the charging detail which is the detail of the fee which the e-commerce platform needs to charge to the merchant is receivable, and the charging detail which is the detail of the fee which the e-commerce platform needs to pay to the merchant is payable.
And respectively inputting the company dimension, the receivable factor and the payment factor under the shop dimension into a preset analysis model for analysis and calculation, so as to respectively obtain analysis and calculation results under the company dimension and the shop dimension.
Step S103: and generating monitoring suggestion information of the service data according to the analysis operation results of the multiple dimensions, wherein the monitoring suggestion information indicates to execute corresponding operations on the service data.
Specifically, the analysis operation results of multiple dimensions may be compared with the threshold values of the corresponding dimensions; if the analysis operation results of the multiple dimensions are all smaller than the threshold value of the corresponding dimension, generating data freezing suggestion information; if one or N analysis operation results in the analysis operation results of the multiple dimensions are smaller than the threshold value of the corresponding dimension, generating suggestion information of data early warning, wherein N is smaller than the total number of the dimensions; and if the analysis operation results of the multiple dimensions are all larger than or equal to the threshold value of the corresponding dimension, generating suggestion information that the data are normal.
The recommendation information of data freezing indicates an operation of performing data freezing on the service data, the recommendation information of data early warning indicates an operation of performing data early warning on the service data, and the recommendation information of data normal indicates an operation of keeping the data normal.
The thresholds for each dimension may be the same or different. Taking abnormal monitoring of the charging data as an example, the threshold of each dimension may be set to zero.
Different types of monitoring advice information can be displayed on the interface by different types of prompting lamps, for example, a red lamp is displayed when data freezing advice information is generated, a yellow lamp is displayed when data early warning advice information is generated, and a green lamp is displayed when data normal advice information is generated.
The embodiment of the invention can also send the generated monitoring suggestion information to the interactive system, and the interactive system automatically processes abnormal conditions at the first time according to the monitoring suggestion information, such as executing data freezing to intercept the withdrawal, outputting alarm prompt of data early warning and the like.
Fig. 2 is a schematic diagram of the main steps of a data anomaly monitoring method according to a second embodiment of the present invention.
As shown in fig. 2, the method for monitoring data abnormality according to an embodiment of the present invention mainly includes the following steps S201 to S205.
Step S201: and extracting incremental data of the business data from a plurality of data sources, and distributing the incremental data in a calculation direction according to the data attribute of the incremental data to obtain first calculation direction data and second calculation direction data.
The incremental data of the service data is newly added service data since the last analysis operation is performed on the service data, that is, the incremental data is service data which is not subjected to the analysis operation.
Step S202: and extracting calculation factors according to a plurality of dimensions from the first calculation direction data and the second calculation direction data respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions.
Step S203: inputting the first direction calculation factor and the second direction calculation factor with the same dimension into a preset analysis model for analysis and operation to respectively obtain an incremental operation result under each dimension.
Step S204: and respectively adding the incremental operation result of each dimension with the last analysis operation result of the corresponding dimension to obtain the analysis operation results of a plurality of dimensions.
Step S205: and generating monitoring suggestion information of the service data according to the analysis operation results of the multiple dimensions, wherein the monitoring suggestion information indicates to execute corresponding operations on the service data.
In the following, the financial monitoring (specifically, monitoring whether charging data is abnormal) for preventing a malicious hacker from checking a list and cash-out by a merchant of an e-commerce platform enterprise is taken as an example, and the data abnormality monitoring method according to the embodiment of the present invention is described in detail. The form brushing and cashing of the embodiment of the invention means that hacker merchants simulate normal consumers in batches by technical means, generate large-scale purchase orders with different IP addresses, and then return goods in batches, thereby cashing the payment of platform merchants.
It should be noted that the method for monitoring data abnormality in the embodiment of the present invention is not limited to the financial monitoring scenario, and is also applicable to various scenarios of monitoring data interaction abnormality between an enterprise platform and a business party.
Fig. 3 is a schematic flow chart of monitoring data abnormality according to a third embodiment of the present invention.
As shown in fig. 3, the monitoring process of data exception according to an embodiment of the present invention includes:
step S301: and extracting incremental data of the charging data from each upstream service system.
The charging data of each upstream service system can be stored in real time, and the incremental data can be extracted from the charging data stored in real time. The incremental data of the charging data is newly added charging data (which is not analyzed and calculated) after the charging data is analyzed and calculated for the last time.
The charging data of the embodiment of the invention is specifically the charging details or the bills which are not completed (namely, not enter the payment process). The bill is a business bill, and can correspond to a plurality of pieces of charging data.
Each service system is a source system of charging data, namely a data source. The core fields of the charging data may include: source system, primary key ID, cost ID, status, amount, etc.
Step S302: and performing calculation direction distribution on the increment data of the charging data.
After the increment data of the charging data is extracted, the extracted increment data can be filtered according to a preset rule, mainly for filtering invalid charging data, and the filtering rule can be used for filtering the charging data with the source system, the charge ID, the charge direction and other fields not existing or empty and the charge amount of 0, so as to determine the monitoring data range.
And configuring data attributes of the incremental data after the invalid charging data is filtered, wherein the data attributes include but are not limited to a source system, a charge code (charge ID), a charge direction, a document state and the like, and the data attributes can be used as charging factors to determine the calculation direction of the incremental data, so that preparation is made for subsequent analysis and calculation.
The incremental data of the charging data is allocated in the calculation direction, specifically, the incremental data can be allocated to a receivable pool or a payable pool, the charging data allocated to the receivable pool can be called as receivable charging pool data, and the charging data allocated to the payable pool can be called as payable charging pool data.
Step S303: and respectively carrying out analysis operation on the shop dimension and the company dimension on the incremental data subjected to the distribution of the calculation direction by using an inverse hanging analysis model to obtain incremental operation results under the shop and the company dimensions.
The inverse hang analysis model may be in the form of:
Figure BDA0002255957510000111
wherein,
Figure BDA0002255957510000112
to cope with the accounting sum of the accounting pool data,
Figure BDA0002255957510000113
indicating the charging sum of the charging pool data to be received. Specifically, AiIndicating the factor due in the pool due, BiIndicating the receivable factor in the receivable pool. A factor to be received or due refers to charging data (or called charging details) corresponding to one service order, and one service order can correspond to a plurality of pieces of charging data. n is the number of factors due and m is the number of factors to be received.
Step S304: and respectively adding the increment operation results of the shop dimension and the company dimension with the latest analysis operation results of the shop and the company dimension to obtain the charging reversed amount of the shop and the company dimension.
Step S305: comparing the charging reverse hanging amount of the shop dimension and the company dimension with zero respectively, and if the charging reverse hanging amount is less than zero, executing a step S306; if only one is smaller than zero, go to step S307; if both are greater than or equal to zero, go to step S308.
With E1Representing the amount of the charge to be reversed in the company dimension, E2Representing the amount of the charge to be reversed in the store dimension. Zero is a threshold for the store dimension and the company dimension, and is typically set to zero, or other values as desired, and the thresholds for the two dimensions may be set to different values.
Step S306: and generating suggestion information of data freezing.
When E is1< 0 and E2And when the data is less than 0, generating a red light, namely freezing the data, and sending the result to the interactive system in a message form.
Step S307: and generating recommendation information of data early warning.
When E is1< 0 or E2And if the sum is less than 0, generating a yellow light, namely data early warning, and sending the result to the interactive system in a message form.
Step S308: and generating suggestion information that the data are normal.
When E is10 and E2And when the number is not less than 0, generating a green light, namely the data is normal, and sending the result to the interactive system in a message form.
After the various monitoring suggestion information is sent to the interactive system, the interactive system can automatically trigger a corresponding instruction according to the monitoring suggestion information, for example, automatically trigger an interception instruction according to suggestion information of data freezing, and intercept the charging data before entering a payment process.
According to the embodiment of the invention, the charging reverse hanging amount calculated by multiple dimensions is utilized, the corresponding financial advice is generated and the interception instruction is automatically triggered through the established relation between each dimension and the reverse hanging amount, the incremental operation can be carried out on the big data in real time, the error rate of manual monitoring of financial staff is reduced, the system pneumatic control mechanism of a platform enterprise is perfected, the complex work of manual intervention and money pursuit is avoided, the malicious cash registering behavior of a merchant is prevented, the operating risk of the platform enterprise is reduced, the monitoring advice is generated and provided, and the financial staff of the enterprise can conveniently master and trace the real payment condition of the merchant in time.
FIG. 4 is a block diagram of a data anomaly monitoring framework according to a fourth embodiment of the present invention.
As shown in fig. 4, the a service system data source, the B service system data source, … …, and the N service system data source are upstream service systems for providing service data. Taking the service data as the charging data as an example, the charging data obtained from the service system can be stored in a centralized manner, and the storage process can be a flow type, that is, the newly added service data is stored in real time. The embodiment of the invention can only analyze and operate the incremental data of the billing data by utilizing a preset analysis model, and add the analysis and operation result based on the incremental data and the analysis and operation result of the last time to obtain the latest analysis and operation result, thereby being capable of processing in batches in real time. And then updating the stored last charging monitoring result data according to the latest analysis operation result so as to facilitate the use of the next analysis operation. After the incremental data analysis calculation is completed, updating the calculation state of the incremental data, such as updating to be completed, indicating that the analysis operation is completed, may be further included.
Fig. 5 is a schematic diagram of main blocks of a data abnormality monitoring apparatus according to a fifth embodiment of the present invention.
As shown in fig. 5, the apparatus 500 for monitoring data abnormality according to an embodiment of the present invention mainly includes: a calculation direction distribution module 501, an analysis operation module 502 and a monitoring suggestion information generation module 503.
The calculation direction allocating module 501 is configured to perform calculation direction allocation on the service data according to the data attribute of the service data to obtain first calculation direction data and second calculation direction data.
The calculation direction assignment module 501 may perform calculation direction assignment according to the service code in the data attribute and the service direction corresponding to the service code.
The analysis operation module 502 is configured to perform analysis operation on the first calculation direction data and the second calculation direction data in multiple dimensions by using a preset analysis model.
The monitoring suggestion information generating module 503 is configured to generate monitoring suggestion information for the service data according to the analysis operation result of the multiple dimensions, where the monitoring suggestion information indicates to perform a corresponding operation on the service data.
In an embodiment, the calculation direction assignment module 501 may be specifically configured to: and extracting incremental data of the business data from a plurality of data sources, and distributing the incremental data in a calculation direction according to the data attribute of the incremental data to obtain first calculation direction data and second calculation direction data.
The analysis operation module 502 may specifically be configured to: extracting calculation factors from the first calculation direction data and the second calculation direction data according to a plurality of dimensions respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions; inputting the first direction calculation factor and the second direction calculation factor with the same dimension into a preset analysis model for analysis and operation to respectively obtain an incremental operation result under each dimension; and respectively adding the incremental operation result of each dimension with the last analysis operation result of the corresponding dimension to obtain the analysis operation results of a plurality of dimensions.
In another embodiment, the analysis operation module 502 may be specifically configured to: extracting calculation factors from the first calculation direction data and the second calculation direction data according to a plurality of dimensions respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions; and inputting the first direction calculation factor and the second direction calculation factor with the same dimension into a preset analysis model for analysis and operation to respectively obtain analysis and operation results under each dimension.
The monitoring suggestion information generation module 503 may specifically be configured to: comparing the analysis operation results of the multiple dimensions with the threshold values of the corresponding dimensions; if the analysis operation results of the multiple dimensions are all smaller than the threshold value of the corresponding dimension, generating data freezing suggestion information; if one or N analysis operation results in the analysis operation results of the multiple dimensions are smaller than the threshold value of the corresponding dimension, generating suggestion information of data early warning, wherein N is smaller than the total number of the dimensions; and if the analysis operation results of the multiple dimensions are all larger than or equal to the threshold value of the corresponding dimension, generating suggestion information that the data are normal.
The monitoring suggestion information generation module 503 may also be configured to output the generated monitoring suggestion information so as to handle data exception in time.
Optionally, the monitoring apparatus 500 for data exception according to the embodiment of the present invention may further include a data filtering module, configured to: and filtering out at least one service data of which the field value does not accord with a preset condition according to three field values of a data source, a service code and a service direction corresponding to the service code in the data attribute.
According to the monitoring process executed by the monitoring apparatus 500 for data abnormality of the embodiment of the present invention, the monitoring apparatus 500 for data abnormality may be further divided into a plurality of levels: the system comprises a basic data layer, a configuration layer, a calculation processing layer and a suggestion output layer. The business data acquired from the data source can be stored in a basic data layer, the data source can be an upstream business system, the basic data layer is used for storing the business data of each upstream business system in a centralized manner, and the business data can be directly extracted from the data source or extracted from the basic data layer when the incremental data of the business data are extracted. The configuration layer includes a calculation direction distribution module 501 and a data filtering module, and is used for filtering the service data and distributing the calculation direction. The calculation processing layer comprises an analysis operation module 502, the recommendation output layer comprises a monitoring recommendation information generation module 503, and the description of the calculation processing layer and the recommendation output layer refers to the analysis operation module 502 and the monitoring recommendation information generation module 503.
The data anomaly monitoring device 500 of the embodiment of the invention pulls (acquires) data of each node (namely service data) of each service system to a basic data layer, performs calculation preprocessing such as data attribute configuration and the like through rule definition of each source data in a configuration layer, allocates calculation directions, combines a big data calculation engine and an analysis model in the calculation processing layer, generates monitoring suggestion information in real time through a trigger form of data increment, and pushes the monitoring suggestion information to a suggestion output layer for service reference and provides the monitoring suggestion information to an associated interactive system for specific execution actions. The interactive system of the embodiment of the invention takes financial monitoring as an example, and can be a payment processing system for processing charging data abnormity, such as early warning or interception of the charging data.
In addition, the detailed implementation of the data anomaly monitoring device in the embodiment of the present invention has been described in detail in the above data anomaly monitoring method, and therefore, the repeated content is not described again.
Fig. 6 shows an exemplary system architecture 600 of a data anomaly monitoring method or a data anomaly monitoring device to which an embodiment of the present invention may be applied.
As shown in fig. 6, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. The network 604 serves to provide a medium for communication links between the terminal devices 601, 602, 603 and the server 605. Network 604 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 601, 602, 603 to interact with the server 605 via the network 604 to receive or send messages or the like. The terminal devices 601, 602, 603 may have installed thereon various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 601, 602, 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 605 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 601, 602, 603. The background management server may analyze and perform other processing on the received data such as the service data, and feed back a processing result (for example, monitoring recommendation information — only an example) to the terminal device.
It should be noted that the method for monitoring data abnormality provided by the embodiment of the present invention is generally executed by the server 605, and accordingly, the device for monitoring data abnormality is generally disposed in the server 605.
It should be understood that the number of terminal devices, networks, and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 7, shown is a block diagram of a computer system 700 suitable for use in implementing a terminal device or server of an embodiment of the present application. The terminal device or the server shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU)701, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. The computer program executes the above-described functions defined in the system of the present application when executed by the Central Processing Unit (CPU) 701.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor comprises a calculation direction distribution module, an analysis operation module and a monitoring suggestion information generation module. The names of these modules do not form a limitation on the modules themselves in some cases, for example, the calculation direction assignment module may also be described as "a module for performing calculation direction assignment on business data according to data attributes of the business data to obtain first calculation direction data and second calculation direction data".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: according to the data attribute of the service data, calculating direction distribution is carried out on the service data so as to obtain first calculating direction data and second calculating direction data; performing multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data by using a preset analysis model; and generating monitoring suggestion information of the service data according to the analysis operation results of the multiple dimensions, wherein the monitoring suggestion information indicates that corresponding operation is executed on the service data.
According to the technical scheme of the embodiment of the invention, the calculation direction distribution is carried out on the business data according to the data attribute of the business data, the preset analysis model is utilized to carry out multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data obtained by the calculation direction distribution, and the monitoring suggestion information is generated according to the analysis operation results of the multiple dimensions so as to instruct to execute corresponding operation on the business data. The abnormal condition of data interaction between the enterprise platform and the business side can be automatically monitored in real time, the monitoring error rate is reduced, and the abnormal condition can be responded in the first time, so that the abnormal condition can be timely processed, and the enterprise loss is reduced.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A method for monitoring data abnormity is characterized by comprising the following steps:
according to the data attribute of the service data, calculating direction distribution is carried out on the service data so as to obtain first calculating direction data and second calculating direction data;
performing multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data by using a preset analysis model;
and generating monitoring suggestion information of the service data according to the analysis operation results of the multiple dimensions, wherein the monitoring suggestion information indicates that corresponding operation is executed on the service data.
2. The method according to claim 1, wherein the step of performing calculation direction assignment on the service data according to the data attribute of the service data to obtain a first calculation direction data and a second calculation direction data comprises:
extracting incremental data of the business data from a plurality of data sources, and performing calculation direction distribution on the incremental data according to data attributes of the incremental data to obtain the first calculation direction data and the second calculation direction data.
3. The method of claim 2, wherein the step of performing a multi-dimensional analysis operation on the first and second calculated orientation data using a predetermined analysis model comprises:
extracting calculation factors from the first calculation direction data and the second calculation direction data according to the plurality of dimensions respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions;
inputting the first direction calculation factor and the second direction calculation factor with the same dimension into the preset analysis model for analysis and calculation to respectively obtain an incremental calculation result under each dimension;
and respectively adding the incremental operation result of each dimension with the last analysis operation result of the corresponding dimension to obtain the analysis operation results of the plurality of dimensions.
4. The method according to claim 1, wherein the calculation direction assignment is performed according to a service code in the data attribute and a service direction corresponding to the service code.
5. The method of claim 1, wherein the step of performing a multi-dimensional analysis operation on the first and second calculated direction data using a predetermined analysis model comprises:
extracting calculation factors from the first calculation direction data and the second calculation direction data according to the plurality of dimensions respectively to obtain first direction calculation factors and second direction calculation factors corresponding to the plurality of dimensions;
and inputting the first direction calculation factor and the second direction calculation factor with the same dimension into the preset analysis model for analysis and calculation to respectively obtain an analysis and calculation result under each dimension.
6. The method according to claim 1, wherein the step of generating the monitoring recommendation information for the business data according to the analysis operation results of the plurality of dimensions includes:
comparing the analysis operation results of the plurality of dimensions with the threshold values of the corresponding dimensions;
if the analysis operation results of the multiple dimensions are all smaller than the threshold value of the corresponding dimension, generating suggestion information of data freezing;
if one or N analysis operation results in the analysis operation results of the multiple dimensions are smaller than the threshold value of the corresponding dimension, generating suggestion information of data early warning, wherein N is smaller than the total number of the dimensions;
and if the analysis operation results of the plurality of dimensions are all larger than or equal to the threshold value of the corresponding dimension, generating suggestion information with normal data.
7. The method of claim 1, wherein before performing the calculation direction assignment on the service data according to the data attribute of the service data, further comprising:
and filtering out the service data of which at least one field value does not accord with a preset condition according to three field values of a data source, a service code and a service direction corresponding to the service code in the data attribute.
8. A device for monitoring data anomalies, comprising:
the calculation direction distribution module is used for performing calculation direction distribution on the service data according to the data attribute of the service data to obtain first calculation direction data and second calculation direction data;
the analysis operation module is used for carrying out multi-dimensional analysis operation on the first calculation direction data and the second calculation direction data by using a preset analysis model;
and the monitoring suggestion information generating module is used for generating monitoring suggestion information of the service data according to the analysis operation results of the multiple dimensions, and the monitoring suggestion information indicates to execute corresponding operation on the service data.
9. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method recited in any of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
CN201911063024.0A 2019-10-31 2019-10-31 Data abnormity monitoring method and device Pending CN112749868A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911063024.0A CN112749868A (en) 2019-10-31 2019-10-31 Data abnormity monitoring method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911063024.0A CN112749868A (en) 2019-10-31 2019-10-31 Data abnormity monitoring method and device

Publications (1)

Publication Number Publication Date
CN112749868A true CN112749868A (en) 2021-05-04

Family

ID=75644993

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911063024.0A Pending CN112749868A (en) 2019-10-31 2019-10-31 Data abnormity monitoring method and device

Country Status (1)

Country Link
CN (1) CN112749868A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118662916A (en) * 2024-05-06 2024-09-20 广州三七极梦网络技术有限公司 Refund processing method, device, equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017128359A1 (en) * 2016-01-30 2017-08-03 深圳市博信诺达经贸咨询有限公司 E-commerce platform analysis method and system based on big data
CN107305665A (en) * 2016-04-19 2017-10-31 李辉 It is a kind of to differentiate wash sale, prevent the single method and device of brush
CN109034821A (en) * 2018-06-29 2018-12-18 深圳春沐源控股有限公司 transaction monitoring method and monitoring device, storage medium and server
CN110276620A (en) * 2019-06-28 2019-09-24 深圳前海微众银行股份有限公司 A method and device for determining abnormal transactions

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017128359A1 (en) * 2016-01-30 2017-08-03 深圳市博信诺达经贸咨询有限公司 E-commerce platform analysis method and system based on big data
CN107305665A (en) * 2016-04-19 2017-10-31 李辉 It is a kind of to differentiate wash sale, prevent the single method and device of brush
CN109034821A (en) * 2018-06-29 2018-12-18 深圳春沐源控股有限公司 transaction monitoring method and monitoring device, storage medium and server
CN110276620A (en) * 2019-06-28 2019-09-24 深圳前海微众银行股份有限公司 A method and device for determining abnormal transactions

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118662916A (en) * 2024-05-06 2024-09-20 广州三七极梦网络技术有限公司 Refund processing method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN107392801B (en) Method and device for controlling order disorder, storage medium and electronic equipment
CN112184240A (en) Refund request processing method and device
CN110427304A (en) O&amp;M method, apparatus, electronic equipment and medium for banking system
CN110473036A (en) A kind of method and apparatus generating order number
CN110730097A (en) Internet technology architecture construction method and device, electronic equipment and medium
CN112990991B (en) Method and device for combined invoicing
CN117114787B (en) Supply chain management method, system and terminal based on electronic invoice
CN110704273B (en) Processing method and device of configuration information, electronic equipment and storage medium
CN112749868A (en) Data abnormity monitoring method and device
CN111144694B (en) Operation flow management method and device based on contact analysis
CN113034294A (en) Credit risk display method and device
CN110895777B (en) Method and device for processing overtime service list
CN111383098A (en) Resource distribution method and device
CN113763083B (en) Information processing method and device
CN111526184B (en) Business auditing method and device
KR20190101868A (en) System for managing expense and method using therewith
CN115731028A (en) Early warning method, early warning device, electronic equipment and computer readable medium
CN115760010A (en) Examination method of customs declaration form, customs declaration method, device, equipment and storage medium
CN114693358A (en) Data processing method, device, electronic device and storage medium
CN112215531B (en) Method and device for solving logistics distribution problem
CN112685437A (en) Configuration-based commission calculation method, device, equipment and storage medium
CN113496113A (en) Method and device for generating user value information
CN112183969A (en) Payment equipment operation control method and device for supply order and electronic equipment
CN116342122A (en) Account auditing method, account auditing device, electronic equipment and computer readable medium
CN118838899A (en) Resource processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination