Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.
Term interpretation:
Flink, an open source stream processing framework developed by the Apache software Foundation, has at its heart a distributed stream data stream engine written in Java and Scala. The Flink executes any stream data program in a data parallel and pipeline manner, and the pipeline runtime system of the Flink can execute batch processing and stream processing programs. Furthermore, the runtime itself of the flank also supports the execution of the iterative algorithm.
TiDB is an open source distributed relational database which is independently designed and researched and developed by PingCAP company, is a fusion type distributed database product which simultaneously supports online transaction processing, online analysis processing and mixed transaction and analysis processing, and has important characteristics of horizontal expansion or contraction, high-availability financial grade, real-time mixed transaction and analysis processing, cloud native distributed database and the like. The goal is to provide users with one-stop online transaction, online analysis, hybrid transaction, and analysis solutions. TiDB is suitable for various application scenes such as high availability, high strong consistency requirement, large data scale and the like.
Clickhouse A column-based storage database for online analytical processing query massive parallel processing architecture, which was the source of Yandex in 2016 in Russia, can use structured query language to query real-time generation of analytical data reports.
Kafka, an open source stream processing platform developed by the Apache software Foundation, is written by Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action flow data for consumers in a web site. Such actions (web browsing, searching and other user actions) are a key factor in many social functions on modern networks. These data are typically addressed by processing logs and log aggregations due to throughput requirements. This is a viable solution for log data and offline analysis systems like big data analysis, but with the limitation of requiring real-time processing. The purpose of Kafka is to unify on-line and off-line message processing through a parallel loading mechanism of big data analysis, as well as to provide real-time messages through clusters.
In the disclosure, a data system is constructed by utilizing a link component and TiDB, after real-time service data is acquired, the real-time service data can be analyzed to determine the characteristic value of each dimension, when the characteristic value of at least one dimension meets a trigger threshold, the history data associated with the trigger threshold is acquired, then the history data and the service data can be processed to obtain push data, and the push data is pushed to an associated service processing client. Therefore, not only is the hysteresis of service data processing avoided, but also the speed of service personnel for acquiring service data is improved, and conditions are provided for improving the service processing efficiency.
The service data processing method according to the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.
It should be noted that, the business data processing method implemented by the present disclosure is configured in an interactive interface, and the interactive interface may be applied to any electronic device, so that the electronic device may implement a data processing function.
The electronic device may be any device with computing capability, for example, may be a personal computer (Personal Computer, abbreviated as PC), a mobile terminal, and the mobile terminal may be a hardware device with various operating systems, touch screens, and/or display screens, for example, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and the like.
Fig. 1 is a flow chart of a service data processing method according to an embodiment of the present disclosure.
As shown in fig. 1, the method includes:
Step 101, current service data and associated service processing rules are acquired.
In some possible implementations, multiple business processing rules may be included in a business system, with different business processing rules associated with different types of business. Therefore, the service system can firstly acquire the currently generated service data, and then acquire the service processing rule associated with the service data of the type according to the service type of the current service data.
The business system is a system for providing business services, and the business data currently generated by the business system is business data associated with the business services currently provided by the business system.
For example, if a customer A transfers 200 ten thousand money to an account B at a counter, the business system can acquire business data of 'that the customer A transfers 200 ten thousand money to the account B at a branch C on X day Y'.
In addition, the service processing rule refers to a rule set by a service person and used for processing the service data. The service processing rules associated with the service data may be one or more, and this disclosure is not limited thereto.
Alternatively, the business processing rules associated with different types of business may be the same or different.
For example, the business processing rules associated with the transfer type business may include counting the business of entering and exiting the transfer account and the collection account within approximately 2 months when the transfer amount is greater than 100 ten thousand. Or the business processing rules associated with the deposit business may include counting account amount variation information of the deposit user for about half a year when the single deposit amount is more than 50 ten thousand, etc., which is not limited by the present disclosure.
It should be noted that the above-mentioned service processing rules associated with different types of services are only schematic descriptions, and should not be taken as a limiting description of the service data processing method provided in the present disclosure.
And 102, analyzing the service data to determine the characteristic values of the service data in each dimension.
Each service data may include multiple dimensions of information, for example, each service data may include "user" dimension of information, "amount" dimension of information, "time" dimension of information, and "service type" dimension of information, and the feature values of the dimensions corresponding to different service data may not be identical. Therefore, after the current service data is acquired, the data system can analyze the current service data to obtain the characteristic values of the current service data in each dimension.
For example, the service data of 200 ten thousand transfers of the client A to the account B in the branch C on the X day and Y point are analyzed, and the characteristic value of the service data in the time dimension is determined to be X day and Y point, the characteristic value in the service type dimension is transferred, and the characteristic value in the amount dimension is determined to be 200 ten thousand.
It should be noted that the above examples are only schematic illustrations, and are not intended as a limiting illustration of the abnormal data acquisition method provided by the present disclosure.
And step 103, acquiring historical data based on a historical data acquisition rule associated with the triggering threshold under the condition that the characteristic value of at least one dimension meets the triggering threshold indicated in the business processing rule.
In the disclosure, trigger thresholds corresponding to different dimensions may be indicated in the service processing rule, and then after determining the feature value of the service data in each dimension, the data system may compare the feature value of each dimension with the trigger threshold of the corresponding dimension to determine whether the feature value in the dimension meets the trigger threshold.
In addition, the historical data acquisition rules associated with different trigger threshold combinations may be different, or the historical data acquisition rules associated with different trigger threshold combinations may be different. For example, the history data obtaining rule associated with the trigger threshold corresponding to the "amount" dimension may be "obtaining all service data of the same user in a month, the history data obtaining rule associated with the trigger threshold corresponding to the" service type "dimension is" obtaining service data of the same type as the current service in a month of the same user ", and the disclosure is not limited thereto.
In some possible implementations, if the feature value of one dimension triggers a trigger threshold set in the service processing rule, the service system may acquire the history data according to the history data acquisition rule associated with the trigger threshold of the one dimension.
In some possible implementation forms, if the feature values of the multiple dimensions trigger the corresponding trigger thresholds set in the service processing rule, the service system may acquire the historical data according to the historical data acquisition rule associated with the combination of the trigger thresholds of the multiple dimensions.
The historical data may be the previously acquired service data stored in the TiDB database, or the processing result data associated with the previous service data stored in the TiDB database, which is not limited in this disclosure.
And 104, analyzing the service data and the historical data to generate push data.
The push data may be data including a part of service data and a part of history data, or the push data may be other data for explaining abnormality of the service data, which is not limited in the present disclosure.
After the service system acquires the service data and the historical data, the service data and the historical data can be analyzed based on the service processing rules corresponding to the service data so as to determine whether the data needs to be pushed to service personnel, and push data is generated under the condition that the data needs to be pushed.
For example, if the service system analyzes the service data and the history data and determines that the account associated with the service data is frequently turned over in the last month, the interactive account is not fixed, and the amount of money is large, it may be determined that the user to which the account belongs may participate in the services such as illegal lending, so that the push data may be generated.
In some possible implementation forms, the processing rules associated with the service data may further include desensitizing the push data, or converting the push data into a data format that is convenient for different devices to receive, etc.
For example, the processing rule associated with the service data indicates that some data (such as user names) related to the push data need to be subjected to masking processing, and then the service system can carry out masking processing on the user names in the push data after generating the push data. Or the processing rule associated with the service data indicates that some data (such as money) related to the push data needs to be encrypted and transmitted, and the service system can encrypt the push data after generating the push data. Or the processing rule associated with the service data indicates that when the service processing client for receiving the push data is located in the mobile device, the push data needs to be converted into a PDF format, and when the service processing client for receiving the push data is located in the PC device, the push data needs to be converted into a text format and the like.
It should be noted that the above-mentioned analysis processing of different types of service data is only schematically illustrated, and is not intended to be a limiting description of the service data processing method provided in the present disclosure.
In some possible implementations, the service system may also determine that pushing data to the service personnel is not needed after parsing the service data and the history data, so that pushing is not needed to be generated.
And step 105, the push data is sent to a business processing client associated with the business data.
In the present disclosure, the service processing client associated with the service data may be determined based on a processing rule associated with the service data, or the associated service processing client may be determined based on a type to which the service data belongs, which is not limited in the present disclosure.
In some possible implementation forms, the push modes corresponding to different types of service data may also be different, or the push modes indicated by different service processing rules may also be different. In the disclosure, after determining push data, a target push mode associated with service data and an identifier of a service processing client may be determined first, and then the push data is sent to the service processing client based on the target push mode and the identifier of the service processing client.
In some possible implementation forms, the target push mode and the identifier of the service processing client may be obtained from the service processing rule associated with the service data, or the target push mode and the identifier of the service processing client may be determined based on the type of the service data, which is not limited in this disclosure.
The target pushing mode can be pushing through a WeChat public number, pushing through a short message, pushing through an in-station message on a PC platform, and the like. The service processing client can be located at the mobile phone end or the PC end, etc.
In the embodiment of the disclosure, after current service data is acquired, firstly, an associated service processing rule is determined, then the service data is analyzed to determine characteristic values of the service data in each dimension, and when the characteristic value of at least one dimension meets a trigger threshold indicated in the service processing rule, the history data is acquired based on a history data acquisition rule associated with the trigger threshold, then the service data and the history data are analyzed to generate push data, and the push data is pushed to an associated service processing client. Therefore, the real-time processing of the service data is realized, the timeliness of the service personnel for acquiring the service data is ensured, and the condition is provided for improving the service processing efficiency.
According to the analysis, after the service system obtains the service data, the service system can firstly determine the associated service processing rule, then process the service data in real time based on the service processing rule, and send the push data to the service processing client when needed, so that service personnel can grasp the state of the service data in real time, and further process the service. In the disclosure, in order to facilitate the service personnel to flexibly perform the configuration of the service rules, the service system may further provide an interactive service data processing rule configuration interface, and the service personnel performs the configuration of the service processing rules according to the requirements. The following describes a procedure of a service processing rule provided by an embodiment of the present disclosure with reference to fig. 2.
Fig. 2 is a flowchart of a service processing rule generating method according to an embodiment of the present disclosure, where the flowchart is shown in fig. 2, and the method includes:
In step 201, a service data processing rule setting request is received, wherein the setting request includes a service type.
In the disclosure, the service system may provide a visual interactive interface for the service personnel, and then the service personnel may trigger related buttons or controls in the interactive interface to send a service data processing rule setting request to the service system.
Alternatively, the service system may set different processing rule setting request buttons for different types of service data in the interactive interface, which is not limited in this disclosure.
For example, when the service personnel A clicks a service data processing rule setting button associated with the transfer service in the interactive interface, the service system receives a service data processing rule setting request containing the transfer service.
Step 202, based on the service type, generating and displaying a processing rule setting interface.
In the disclosure, in order to simplify the difficulty of configuring the processing rules by the service personnel and improve the configuration efficiency, different interfaces can be set for different processing rules of different service types. Therefore, after the service system receives the setting request, the corresponding processing rule setting interface can be displayed according to the service type contained in the setting request.
And then, the business personnel can perform rule setting in the displayed processing rule setting interface. For example, the service personnel can set or select time information in a "time" control in the processing rule setting interface according to the service requirement.
In some possible implementation forms, the processing rule setting interface may further include a trigger threshold setting or selecting control, and the service personnel may set or select the trigger threshold according to the service type.
Step 203, under the condition that a service processing rule generation request is received, generating a service processing rule associated with a service type based on configuration information input in each page control of the processing rule setting interface.
The configuration information input in the page control refers to information input in the page control or information indicated by a target option selected by the page control in the process of configuring a processing rule by service personnel.
In the disclosure, after setting each settable page control in the processing rule setting interface is completed, a service person can trigger a "complete" or "submit" button in the processing rule setting interface to send a service processing rule generating request to a service system, and then the service system can generate an associated service processing rule based on configuration information input by the service person in each page control of the interface.
It will be appreciated that the configuration information of the processing rule setting interface is different, and the service processing rules generated by the service system are also different.
In addition, after the user completes the configuration of the setting interface, the system background can generate associated business processing rules according to the configuration information.
It can be understood that by the method for generating the business processing rules, which is provided by the present disclosure, business personnel can generate the business processing rules required by the business personnel only through simple page interaction without using program configuration in the background, thereby not only reducing the generation difficulty of the business processing rules, but also providing conditions for flexible updating of the business processing rules.
It should be noted that all of the above examples are illustrative only and are not intended to be limiting of the business data processing methods provided by the present disclosure.
In the embodiment of the disclosure, after receiving a service data processing rule setting request, a service system may generate and display a corresponding processing rule setting interface according to a service type included in the request, and then after receiving a service processing rule generating request, generate a service processing rule associated with the service type based on configuration information input in each page control of the processing rule setting interface. Therefore, the processing rules corresponding to different types of services can be generated through visual interface interaction, the generation difficulty of the service processing rules is reduced, and the efficiency and the flexibility of generating the service processing rules are improved.
In order to achieve the above embodiments, the embodiments of the present disclosure further provide a processing device for service data.
Fig. 3 is a schematic structural diagram of a service data processing device according to an embodiment of the present disclosure.
As shown in fig. 3, the service data processing apparatus 300 includes a first obtaining module 301, a first parsing module 302, a second obtaining module 303, a second parsing module 304, and a pushing module 305.
A first obtaining module 301, configured to obtain current service data and associated service processing rules;
The first parsing module 302 is configured to parse the service data to determine a feature value of the service data in each dimension;
A second obtaining module 303, configured to obtain, based on a history data obtaining rule associated with a trigger threshold, history data when a feature value of at least one dimension meets the trigger threshold indicated in the service processing rule;
the second parsing module 304 is configured to parse the service data and the history data to generate push data;
The pushing module 305 is configured to send the push data to a service processing client associated with the service data.
In one possible implementation manner of the embodiment of the disclosure, the display device further comprises a receiving module, a display module and a generating module.
The receiving module is used for receiving a service data processing rule setting request, wherein the setting request comprises a service type;
The display module is used for generating and displaying a processing rule setting interface based on the service type;
And the generation module is used for generating the business processing rule associated with the business type based on the configuration information input in each page control of the processing rule setting interface under the condition of receiving the business processing rule generation request.
In one possible implementation manner of the embodiment of the disclosure, the method further comprises a determining module and a storage module.
The determining module is used for determining a target database according to the service type of the service data;
And the storage module is used for storing the push data into the target database.
It should be noted that, the explanation of the foregoing embodiment of the service data processing method is also applicable to the device of this embodiment, so that the description is omitted here.
In the disclosure, after current service data is obtained, firstly, an associated service processing rule is determined, then the service data is analyzed to determine characteristic values of the service data in each dimension, and under the condition that the characteristic value of at least one dimension meets a trigger threshold indicated in the service processing rule, the history data is obtained based on a history data obtaining rule associated with the trigger threshold, then the service data and the history data are analyzed to generate push data, and the push data is pushed to an associated service processing client. Therefore, the real-time processing of the service data is realized, the timeliness of the service personnel for acquiring the service data is ensured, and the condition is provided for improving the service processing efficiency.
According to an embodiment of the disclosure, the disclosure further provides an electronic device.
Fig. 4 is a schematic structural diagram of an electronic device 400 according to an embodiment of the present application.
As shown in fig. 4, the electronic device may include a transceiver 401, a processor 402, and a memory 403.
Processor 402 executes the computer-executable instructions stored in memory, causing processor 402 to perform the aspects of the embodiments described above. The processor 402 may be a general purpose processor including a central processing unit (Central Processing Unit, CPU), a network processor (network processor, NP), etc., or may be a digital signal processor (DIGITAL SIGNAL processor, DSP), application Specific Integrated Circuit (ASIC), field programmable gate array (Field programmable GATE ARRAY, FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
The memory 403 is connected to the processor 402 via a system bus and communicates with each other, the memory 403 being arranged to store computer program instructions.
Transceiver 401 may be used to obtain tasks to be run and configuration information for the tasks to be run.
The system bus may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus, or an extended industry standard architecture (Extended industry standard architecture, EISA) bus, among others. The system bus may be classified into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus. The transceiver is used to enable communication between the database access device and other computers (e.g., clients, read-write libraries, and read-only libraries). The memory may include random access memory (Random access memory, RAM) and may also include Non-volatile memory (Non-volatile memory).
The electronic device provided by the embodiment of the application can be the terminal device of the embodiment.
The embodiment of the application also provides a chip for running the instruction, and the chip is used for executing the technical scheme of the task scheduling method in the embodiment.
The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores computer instructions, and when the computer instructions run on a computer, the computer is caused to execute the technical scheme of the task scheduling method in the embodiment.
The embodiment of the application also provides a computer program product, which comprises a computer program stored in a computer readable storage medium, wherein at least one processor can read the computer program from the computer readable storage medium, and the technical scheme of the task scheduling method in the embodiment can be realized when the at least one processor executes the computer program.
Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.