CN115004167B - Data processing methods and related products - Google Patents
Data processing methods and related products Download PDFInfo
- Publication number
- CN115004167B CN115004167B CN202080094736.3A CN202080094736A CN115004167B CN 115004167 B CN115004167 B CN 115004167B CN 202080094736 A CN202080094736 A CN 202080094736A CN 115004167 B CN115004167 B CN 115004167B
- Authority
- CN
- China
- Prior art keywords
- data
- target
- time stamp
- timestamp
- heart rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
一种数据处理方法及相关产品，应用于电子设备，该方法包括：获取待处理数据集，所述待处理数据集包括多个数据，每一数据对应一个时间戳(101)；在RocksDB存储引擎运行时，获取当前时间戳(102)；将所述当前时间戳与所述多个数据中每一数据对应的时间戳进行比对，得到至少一个目标时间戳(103)；删除所述至少一个目标时间戳对应的数据，并将所述待多个数据中的剩余数据进行合并，得到目标数据集(104)。该方法能够提升数据删除效率，有助于提升系统性能。
A data processing method and related products are applied to electronic devices, the method comprising: obtaining a data set to be processed, the data set to be processed comprising multiple data, each data corresponding to a timestamp (101); when the RocksDB storage engine is running, obtaining a current timestamp (102); comparing the current timestamp with the timestamp corresponding to each data in the multiple data to obtain at least one target timestamp (103); deleting the data corresponding to the at least one target timestamp, and merging the remaining data in the multiple data to obtain a target data set (104). The method can improve data deletion efficiency and help improve system performance.
Description
Technical Field
The application relates to the technical field of data processing, in particular to a data processing method and related products.
Background
Along with the popularization and application of electronic equipment (such as mobile phones, tablet computers and the like), the electronic equipment can support more and more applications, has more and more functions, and develops towards diversification and individuation, so that the electronic equipment becomes an indispensable electronic product in the life of users.
Currently, when the expired data needs to be deleted in time through service logic, and the data needs to be deleted in combination with service running states, such as during the low peak period of service, the influence on online service is avoided. The method has the main defects that concentrated deletion affects the reading and writing of the service during the deletion period, which causes the delay rise of the reading and writing, and the service low peak period is generally early morning, large-scale data deletion is carried out, unattended operation has a certain risk, so the problem of reasonably deleting the expired data to improve the on-line service efficiency is urgently solved.
Disclosure of Invention
The embodiment of the application provides a data processing method and related products, which can improve the intelligence of deleting out-of-date data and are beneficial to improving the on-line service efficiency.
In a first aspect, an embodiment of the present application provides a data processing method, applied to an electronic device, including:
acquiring a data set to be processed, wherein the data set to be processed comprises a plurality of data, and each data corresponds to a time stamp;
Acquiring a current time stamp when the RocksDB storage engine runs;
Comparing the current time stamp with a time stamp corresponding to each data in the plurality of data to obtain at least one target time stamp;
Deleting the data corresponding to the at least one target time stamp, and merging the rest data in the data to be processed to obtain a target data set.
In a second aspect, an embodiment of the present application provides a data processing apparatus, which is applied to an electronic device, where the apparatus includes a first acquisition unit, a second acquisition unit, an alignment unit, and a deletion unit,
The first acquisition unit is used for acquiring a data set to be processed, wherein the data set to be processed comprises a plurality of data, and each data corresponds to one time stamp;
the second obtaining unit is used for obtaining the current time stamp when the RocksDB storage engine runs;
The comparison unit is used for comparing the current time stamp with the time stamp corresponding to each data in the plurality of data to obtain at least one target time stamp;
the deleting unit is configured to delete data corresponding to the at least one target timestamp, and combine remaining data in the plurality of data to obtain a target data set.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps in the first aspect of the embodiment of the present application.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to perform part or all of the steps described in the first aspect of the embodiments of the present application.
In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product comprises a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.
Drawings
The drawings that accompany the embodiments or the prior art description can be briefly described as follows.
In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present application;
FIG. 1B is a flow chart of a data processing method according to an embodiment of the present application;
FIG. 1C is a schematic illustration of a combined implementation principle disclosed in an embodiment of the present application;
FIG. 2 is a flow chart of another data processing method disclosed in an embodiment of the present application;
FIG. 3 is a flow chart of another data processing method disclosed in an embodiment of the present application;
FIG. 4 is a schematic diagram of another electronic device according to an embodiment of the present application;
FIG. 5A is a schematic diagram of a data processing apparatus according to an embodiment of the present application;
FIG. 5B is a schematic diagram of another data processing apparatus according to an embodiment of the present application;
FIG. 5C is a schematic diagram of another data processing apparatus according to an embodiment of the present application;
fig. 5D is a schematic structural diagram of another data processing apparatus according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The electronic device according to the embodiment of the present application may include various handheld devices, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem, and various forms of User Equipment (UE), mobile Stations (MS), terminal devices (TERMINAL DEVICE), etc. For convenience of description, the above-mentioned devices are collectively referred to as electronic devices, and the above-mentioned electronic devices may also be servers, service platforms, and the like.
Embodiments of the present application are described in detail below.
Referring to fig. 1A, fig. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present application, and the electronic device 100 may include a control circuit, and the control circuit may include a storage and processing circuit 110. The storage and processing circuit 110 may be a memory such as a hard drive memory, a non-volatile memory (e.g., flash memory or other electronically programmable read only memory used to form a solid state drive, etc.), a volatile memory (e.g., static or dynamic random access memory, etc.), etc., as embodiments of the application are not limited. Processing circuitry in the storage and processing circuitry 110 may be used to control the operation of the electronic device 100. The processing circuitry may be implemented based on one or more microprocessors, microcontrollers, baseband processors, power management units, audio codec chips, application specific integrated circuits, display driver integrated circuits, and the like.
The storage and processing circuitry 110 may be used to run software in the electronic device 100, such as internet browsing applications, voice over internet protocol (voice over internet protocol, VOIP) telephone call applications, email applications, media playback applications, operating system functions, and the like. Such software may be used to perform some control operations, such as image acquisition based on a camera, ambient light measurement based on an ambient light sensor, proximity sensor measurement based on a proximity sensor, information display functions implemented based on status indicators such as status indicators of light emitting diodes, touch event detection based on a touch sensor, functions associated with displaying information on multiple (e.g., layered) displays, operations associated with performing wireless communication functions, operations associated with collecting and generating audio signals, control operations associated with collecting and processing button press event data, and other functions in electronic device 100, to name a few.
The electronic device 100 may also include an input-output circuit 150. The input-output circuit 150 is operable to cause the electronic device 100 to effect input and output of data, i.e., to allow the electronic device 100 to receive data from an external device and also to allow the electronic device 100 to output data from the electronic device 100 to an external device. The input-output circuit 150 may further include a sensor 170. The sensor 170 may include an ambient light sensor, a proximity sensor based on light and capacitance, a touch sensor (e.g., based on an optical touch sensor and/or a capacitive touch sensor, where the touch sensor may be part of a touch display screen or may be used independently as a touch sensor structure), an acceleration sensor, a gravity sensor, and other sensors, etc.
The input-output circuit 150 may also include one or more displays, such as display 130. Display 130 may include one or a combination of several of a liquid crystal display, an organic light emitting diode display, an electronic ink display, a plasma display, and a display using other display technologies. Display 130 may include an array of touch sensors (i.e., display 130 may be a touch-sensitive display screen). The touch sensor may be a capacitive touch sensor formed of an array of transparent touch sensor electrodes, such as Indium Tin Oxide (ITO) electrodes, or may be a touch sensor formed using other touch technologies, such as acoustic wave touch, pressure sensitive touch, resistive touch, optical touch, etc., as embodiments of the application are not limited.
The audio component 140 may be used to provide audio input and output functionality for the electronic device 100. The audio components 140 in the electronic device 100 may include speakers, microphones, buzzers, tone generators, and other components for generating and detecting sound.
The communication circuitry 120 may be used to provide the electronic device 100 with the ability to communicate with external devices. The communication circuit 120 may include analog and digital input-output interface circuits, and wireless communication circuits based on radio frequency signals and/or optical signals. The wireless communication circuitry in the communication circuitry 120 may include radio frequency transceiver circuitry, power amplifier circuitry, low noise amplifiers, switches, filters, and antennas. For example, wireless communication circuitry in communication circuitry 120 may include circuitry to support Near Field Communication (NFC) by transmitting and receiving near field coupled electromagnetic signals. For example, the communication circuit 120 may include a near field communication antenna and a near field communication transceiver. The communication circuit 120 may also include a cellular telephone transceiver and antenna, a wireless local area network transceiver circuit and antenna, and the like.
The electronic device 100 may further include a battery, power management circuitry, and other input-output units 160. The input-output unit 160 may include buttons, levers, click wheels, scroll wheels, touch pads, keypads, keyboards, cameras, light emitting diodes, and other status indicators, etc.
A user may control the operation of the electronic device 100 by inputting commands through the input-output circuit 150, and may use output data of the input-output circuit 150 to enable receiving status information and other outputs from the electronic device 100.
Referring to fig. 1B, fig. 1B is a flowchart of a data processing method according to an embodiment of the present application, where the data processing method described in the embodiment is applied to an electronic device as shown in fig. 1A, and the data processing method includes:
101. A data set to be processed is obtained, wherein the data set to be processed comprises a plurality of data, and each data corresponds to one time stamp.
In the embodiment of the present application, the data set to be processed may include a plurality of data, where the data may be at least one of text data, video data, audio data, cache data, a compressed package, a folder, and the like, which is not limited herein. Each data may correspond to a key-value pair, each data corresponding to a time stamp for recording the time of generation or expiration of the data. The timestamp field corresponding to the timestamp may be located at the end of the key-value pair.
102. At RocksDB when the storage engine is running, the current timestamp is obtained.
In the embodiment of the application, rocksDB storage engine is a persistent KV storage engine, which is a database with a log structure and is optimized for fast storage.
In particular implementations, the electronic device may obtain a current timestamp that records the current time when RocksDB storage engines are running.
103. And comparing the current time stamp with the time stamp corresponding to each data in the plurality of data to obtain at least one target time stamp.
The electronic device compares the current time stamp with the time stamp corresponding to each of the plurality of data to obtain at least one target time stamp, and the data corresponding to the target time stamp can be used as the data to be deleted.
In a possible example, the step 103 of comparing the current timestamp with a timestamp corresponding to each of the plurality of data to obtain at least one target timestamp may include the following steps:
and comparing the current time stamp with the time stamp corresponding to each data in the plurality of data through a hook function in the RocksDB storage engine to obtain at least one target time stamp.
The RocksDB storage engine of the electronic device may include a hook function, and the hook function in the RocksDB storage engine is used to compare the current timestamp with the timestamp corresponding to each of the plurality of data, so as to obtain at least one target timestamp, where the data corresponding to the timestamp may be used as the data to be deleted, and by this way, the future data may be selected.
In one possible example, the step 103 of comparing the current timestamp with the timestamp corresponding to each of the plurality of data to obtain at least one target timestamp may include the following steps:
A31, determining the time length between the current time stamp and the time stamp corresponding to each data in the plurality of data and the current time stamp to obtain a plurality of time lengths;
a32, selecting a time length larger than a preset threshold value from the time lengths to obtain at least one target time length;
A33, taking the time stamp of the data corresponding to the at least one target time length as the at least one target time stamp.
In the embodiment of the application, the preset threshold value can be set by a user or default by the system. The time stamp of each data in the plurality of data is used for recording the corresponding existing time of the data, and the existing time can be understood as the generation time or the storage time of the data.
In a specific implementation, the electronic device may determine a time length between a current time stamp and a time stamp corresponding to each of the plurality of data and the current time stamp, so as to obtain a plurality of time lengths, and further may select a time length greater than a preset threshold from the plurality of time lengths, so as to obtain at least one target time length, and may use the time stamp of the data corresponding to the at least one target time length as at least one target time stamp, so that some time stamps with longer existence time may be selected.
Further, in one possible example, before the step a31, the method may further include the following steps:
A34, acquiring the target CPU utilization rate of the electronic equipment;
a35, determining the preset threshold corresponding to the target CPU utilization according to the mapping relation between the preset CPU utilization and the threshold.
In a specific implementation, the electronic device may obtain a target CPU utilization of the electronic device, and determine a preset threshold corresponding to the target CPU utilization according to the mapping relationship between the preset CPU utilization and the threshold, so as to dynamically adjust the threshold, for example, when the CPU utilization is high, some data may be deleted more, and when the CPU utilization is low, some data may be deleted less.
In one possible example, the step 103 of comparing the current timestamp with the timestamp corresponding to each of the plurality of data to obtain at least one target timestamp may include the following steps:
b31, acquiring a time stamp corresponding to each data in the plurality of data to obtain a plurality of time stamps;
b32, determining that the plurality of time stamps are earlier than the time stamp of the current time stamp, and obtaining the at least one target time stamp.
In a specific implementation, the timestamp of each data in the plurality of data is used for recording the expiration time corresponding to the data, the electronic device may obtain the timestamp corresponding to each data in the plurality of data, obtain a plurality of timestamps, and determine that the plurality of timestamps are earlier than the timestamp of the current timestamp, so as to obtain at least one target timestamp.
104. Deleting the data corresponding to the at least one target time stamp, and merging the rest data in the data to be processed to obtain a target data set.
In the embodiment of the application, the merging can also be compression, and is used for sorting and merging the existing data, so that some data (such as outdated data) which are not valid any more are deleted, and the data size and the number of files are reduced, so that the reading speed is increased. The merging process may include Compaction (also referred to as major compatibility) process. The Compaciton process stores the file from a low Level to a high Level. As shown in fig. 1C, when the electronic device may implement merging of the data of Level1 (the first layer) and the data of Level2 (the second layer), a hook function satisfying the compaction filter mechanism is called, so that the expired data in Level1 is removed and the data is not merged into the file in Level 2.
In a specific implementation, the embodiment of the application combines the existing characteristics of RocksDB and utilizes the RocksDB compaction filter mechanism to automatically remove the expired data, and in this way, rocksdb can automatically delete the expired data without manually removing the data. Specifically, when one key-value pair is written, a 4-byte time stamp field is added at the tail of each value through program logic, the time stamp can represent the expiration time of the key, a hook function Filter (an interface function exposed by RocksDB) is exposed to the outside through RocksDB, when RocksDB is operated, the hook function Filter is automatically called each time data merging is carried out, whether the current time stamp is larger than the time stamp stored in the key-value is judged in the hook function, if the current time stamp is larger than the expiration time stored at the tail of the value, the key-value pair is expired, deletion is needed, and then the new file is not merged, so that the purpose of automatically clearing the expired data is achieved.
Based on the embodiment of the application, the electronic equipment can realize the outdated deletion hook function through the compaction filter mechanism of RocksDB, so that outdated data is automatically recovered, and the data is not required to be intensively deleted manually through business logic. The process of deleting data is evenly spread to each merging process of Rocksdb, and the data is deleted smoothly.
In a possible example, the step 104 of deleting the data corresponding to the at least one target timestamp may include the following steps:
41. determining the number of the target time stamps corresponding to the at least one target time stamp;
42. Determining the number of target threads corresponding to the at least one target timestamp according to a mapping relation between the number of preset timestamps and the number of threads;
43. And carrying out thread loading according to the number of the target threads, and deleting data corresponding to at least one target time stamp according to the loaded threads, wherein the data corresponding to each target time stamp corresponds to one thread.
In the embodiment of the application, the mapping relation between the preset time stamp number and the thread number can be stored in the electronic device in advance, and further, in a specific implementation, the electronic device can determine the target time stamp number corresponding to at least one target time stamp, determine the target thread number corresponding to at least one target time stamp according to the mapping relation between the preset time stamp number and the thread number, load threads according to the target thread number, delete data corresponding to at least one target time stamp according to the loaded threads, and each data corresponding to one thread, so that the threads of corresponding number can be configured to delete data according to the number of data to be deleted, thereby being beneficial to improving the data deleting efficiency and improving the systematic performance.
In one possible example, the method may further include the steps of:
C1, receiving a data clearing instruction;
and C2, responding to the data clearing instruction, and executing the step of acquiring the current time stamp.
In a specific implementation, the electronic device may receive a data clearing instruction input by a user, and may further respond to the data clearing instruction, and may further perform the step of obtaining the current timestamp.
In one possible example, the method may further include the steps of:
D1, acquiring operation environment parameters;
and D2, executing the step of acquiring the current time stamp when the running environment parameter meets a preset condition.
The preset conditions can be set by the user or default by the system. The operation environment parameter includes at least one of touch parameter, operation time, geographic position, external environment parameter, CPU temperature, CPU utilization rate, memory occupancy rate, etc., which are not limited herein. The touch parameter may be at least one of a touch force of the touch screen, a touch area of the touch screen, a touch duration of the touch screen, a number of times of touch of the touch screen, a touch temperature of the touch screen, and the like, which are not limited herein, and the environmental parameter may be at least one of an environmental light brightness, an environmental temperature, an environmental humidity, a magnetic field interference intensity, and the like, which are not limited herein. For example, the step of obtaining the current timestamp needs to be performed when the running time is a designated time, and for example, the step of obtaining the current timestamp needs to be performed when the geographic position is a designated position.
In one possible example, before the step D1, the following steps may be further included:
D3, determining the memory size of the data set to be processed;
and D4, executing the step of acquiring the operation environment parameters when the memory size is larger than the preset memory size.
In a specific implementation, the preset memory size can be set by the user or default by the system. The electronic device may determine a memory size of the data set to be processed, and execute the step of obtaining the operating environment parameter when the memory size is greater than a preset memory size, or else, if the memory size is less than or equal to the preset memory size, may not execute the subsequent step, so that the data cleaning may be performed only when a certain amount of memory exists in the data set.
In one possible example, before step 101, the following steps may be further included:
E21, acquiring target physiological state parameters of the user;
e22, determining a target emotion type corresponding to the target physiological state parameter;
and E23, executing the step of acquiring the data set to be processed when the target emotion type is a preset emotion type.
In the embodiment of the present application, the physiological status parameter may be various parameters for reflecting the physiological functions of the user, and the physiological status parameter may be at least one of heart rate, blood pressure, blood temperature, blood lipid content, blood glucose content, thyroxine content, epinephrine content, platelet content, blood oxygen content, and the like, which are not limited herein. The preset emotion type may be set by the user himself or by default. The predetermined emotion type may be at least one of clunking, crying, calm, agitation, depression, etc., without limitation.
In a specific implementation, the electronic device may acquire the target physiological state parameter of the user through a wearable device capable of being in communication connection with the electronic device, different physiological state parameters reflect the emotion type of the user, a mapping relationship between the physiological state parameter and the emotion type may be stored in the electronic device in advance, and further, the target emotion type corresponding to the target physiological state parameter may be determined according to the mapping relationship, further, the step 101 may be executed when the target emotion type is the preset emotion type, or else the step 101 may not be executed.
In one possible example, when the target physiological state parameter is a heart rate variation curve within a specified period of time, the determining the target emotion type corresponding to the target physiological state parameter in the step E22 may be implemented as follows:
e221, sampling the heart rate change curve to obtain a plurality of heart rate values;
E222, carrying out average value operation according to the plurality of heart rate values to obtain an average heart rate value;
e223, determining a target heart rate grade corresponding to the average heart rate value;
E224, determining a target first emotion value corresponding to the target heart rate level according to a mapping relation between the preset heart rate level and the first emotion value;
e225, carrying out mean square error operation according to the plurality of heart rate values to obtain a target mean square error;
e226, determining a target second emotion value corresponding to the target mean square error according to a mapping relation between the preset mean square error and the second emotion value;
E227, determining a target weight pair corresponding to the target heart rate level according to a mapping relation between the preset heart rate level and the weight pair, wherein the weight pair comprises a first weight and a second weight, the first weight is a weight corresponding to the first emotion value, and the second weight is a weight corresponding to the second emotion value;
e228, carrying out weighted operation according to the target first emotion value, the target second emotion value and the target weight value to obtain a final emotion value;
E229, determining the target emotion type corresponding to the target emotion value according to a mapping relation between a preset emotion value and the emotion type.
The appointed time period can be set by a user or defaults by a system, a mapping relation between a preset heart rate grade and a first emotion value, a mapping relation between a preset mean square error and a second emotion value, a mapping relation between a preset heart rate grade and a weight value pair, and a mapping relation between a preset emotion value and an emotion type can be stored in the electronic device in advance, the weight value pair can comprise a first weight and a second weight, the first weight is a weight corresponding to the first emotion value, the second weight is a weight corresponding to the second emotion value, the sum of the first weight and the second weight can be 1, and the value ranges of the first weight and the second weight are all 0-1. In the embodiment of the application, the emotion can be estimated through the heart rate variation curve.
In a specific implementation, the electronic device may sample the heart rate change curve, where a specific sampling manner may be that a plurality of heart rate values are obtained by uniformly sampling or randomly sampling, and average heart rate values may be obtained by performing mean operation according to the plurality of heart rate values, a mapping relationship between the heart rate values and heart rate levels may be stored in the electronic device in advance, further, a target heart rate level corresponding to the average heart rate value may be determined according to the mapping relationship, further, a target first emotion value corresponding to the target heart rate level may be determined according to the mapping relationship between the preset heart rate level and the first emotion value, further, mean square error operation may be performed according to the plurality of heart rate values, so as to obtain a target mean square error, and a target second emotion value corresponding to the target mean square error may be determined according to the mapping relationship between the preset mean square error and the second emotion value.
Further, the electronic device may further determine a target weight pair corresponding to the target heart rate level according to the mapping relationship between the preset heart rate level and the weight pair, where the target weight pair may include a target first weight and a target first weight, the target first weight is a weight corresponding to a target first emotion value, and the target second weight is a weight corresponding to a target second emotion value, and further, the electronic device may perform a weighted operation according to the target first emotion value, the target second emotion value, the target first weight and the target second weight, to obtain a final emotion value, where a specific calculation formula is as follows:
final emotion value = target first emotion value + target first weight + target second emotion value: -target second weight
Furthermore, according to the mapping relation between the preset emotion value and the emotion type, the target emotion type corresponding to the target emotion value can be determined. The average heart rate reflects the heart rate value of the user, the mean square error of the heart rate reflects the heart rate stability, and the emotion of the user is reflected through two dimensions of the average heart rate and the mean square error, so that the emotion type of the user can be accurately determined.
It can be seen that, the data processing method described in the above embodiment of the present application is applied to an electronic device, where a to-be-processed data set is obtained, where the to-be-processed data set includes a plurality of data, each data corresponds to a timestamp, when RocksDB storage engines run, a current timestamp is obtained, the current timestamp is compared with a timestamp corresponding to each of the plurality of data to obtain at least one target timestamp, the data corresponding to the at least one target timestamp is deleted, and the remaining data in the to-be-processed data are combined to obtain a target data set, and expired data can be automatically recovered through RocksDB without manual intervention or intensive deletion of data. And the deleting process is shared to each merging stage, so that the influence on the reading and writing of the online service is very small, the whole deleting process is very smooth, the system performance is improved, and the online service efficiency is improved.
In accordance with the foregoing, referring to fig. 2, fig. 2 is a flow chart of another data processing method according to an embodiment of the present application, where the data processing method described in the present embodiment is applied to the electronic device shown in fig. 1A, the method may include the following steps:
201. a data set to be processed is obtained, wherein the data set to be processed comprises a plurality of data, and each data corresponds to one time stamp.
202. At RocksDB the storage engine is running, a data clear instruction is received.
203. And responding to the data clearing instruction, and acquiring the current time stamp.
204. And comparing the current time stamp with the time stamp corresponding to each data in the plurality of data to obtain at least one target time stamp.
205. Deleting the data corresponding to the at least one target time stamp, and merging the rest data in the data to be processed to obtain a target data set.
The specific implementation process of the steps 201 to 205 may refer to the corresponding description in the method shown in fig. 1B, and will not be repeated herein.
It can be seen that, the data processing method described in the above embodiment of the present application is applied to an electronic device, and obtains a data set to be processed, where the data set to be processed includes a plurality of data, each data corresponds to a timestamp, and when RocksDB storage engines run, a data clearing instruction is received, and a current timestamp is obtained in response to the data clearing instruction, and the current timestamp is compared with a timestamp corresponding to each data in the plurality of data to obtain at least one target timestamp, and the data corresponding to the at least one target timestamp is deleted, and the remaining data in the plurality of data to be processed is combined to obtain a target data set, and the expired data can be automatically recovered through RocksDB, so that user instructions can be received, and data clearing is implemented without densely deleting the data. And the deleting process is shared to each merging stage, so that the influence on the reading and writing of the online service is very small, the whole deleting process is very smooth, the system performance is improved, and the online service efficiency is improved.
In accordance with the foregoing, referring to fig. 3, a flowchart of another embodiment of a data processing method according to the present application is shown, and the data processing method described in the present embodiment is applied to the electronic device shown in fig. 1A, and the method may include the following steps:
301. A data set to be processed is obtained, wherein the data set to be processed comprises a plurality of data, and each data corresponds to one time stamp.
302. At RocksDB, the storage engine runs, operating environment parameters are obtained.
303. And when the running environment parameters meet preset conditions, acquiring a current time stamp.
304. And comparing the current time stamp with the time stamp corresponding to each data in the plurality of data to obtain at least one target time stamp.
305. Deleting the data corresponding to the at least one target time stamp, and merging the rest data in the data to be processed to obtain a target data set.
The specific implementation process of the steps 301 to 305 may refer to the corresponding description in the method shown in fig. 1B, and will not be repeated herein.
It can be seen that, in the data processing method described in the embodiment of the present application, the data processing method is applied to an electronic device, a to-be-processed data set is obtained, the to-be-processed data set includes a plurality of data, each data corresponds to a timestamp, when RocksDB storage engines run, an operation environment parameter is obtained, and when the operation environment parameter meets a preset condition, a current timestamp is obtained, the current timestamp is compared with a timestamp corresponding to each data in the plurality of data to obtain at least one target timestamp, the data corresponding to the at least one target timestamp is deleted, and the remaining data in the plurality of data to be processed are combined to obtain a target data set, the expired data can be automatically recovered through RocksDB, no manual intervention is required, and when the operation environment parameter meets a certain condition, data deletion is realized, and intensive deletion of data is not required. And the deleting process is shared to each merging stage, so that the influence on the reading and writing of the online service is very small, the whole deleting process is very smooth, the system performance is improved, and the online service efficiency is improved.
In keeping with the above, FIG. 4 is a diagram of an electronic device provided by an embodiment of the present application, including a processor and a memory, and one or more programs stored in the memory and configured to be executed by the processor, the programs including instructions for:
acquiring a data set to be processed, wherein the data set to be processed comprises a plurality of data, and each data corresponds to a time stamp;
Acquiring a current time stamp when the RocksDB storage engine runs;
Comparing the current time stamp with a time stamp corresponding to each data in the plurality of data to obtain at least one target time stamp;
Deleting the data corresponding to the at least one target time stamp, and merging the rest data in the data to be processed to obtain a target data set.
It can be seen that, in the electronic device described in the above embodiment of the present application, a to-be-processed data set is obtained, where the to-be-processed data set includes a plurality of data, each data corresponds to a timestamp, when RocksDB storage engines run, a current timestamp is obtained, the current timestamp is compared with a timestamp corresponding to each of the plurality of data to obtain at least one target timestamp, the data corresponding to the at least one target timestamp is deleted, and the remaining data in the to-be-processed data are combined to obtain a target data set, and expired data can be automatically recovered through RocksDB without manual intervention or intensive deletion of data. And the deleting process is shared to each merging stage, so that the influence on the reading and writing of the online service is very small, the whole deleting process is very smooth, the system performance is improved, and the online service efficiency is improved.
In one possible example, the timestamp of each of the plurality of data is used to record the existence time corresponding to the data, and in the aspect of comparing the current timestamp with the timestamp corresponding to each of the plurality of data to obtain at least one target timestamp, the program includes instructions for executing the following steps:
Determining the time length between the current time stamp and the time stamp corresponding to each data in the plurality of data and the current time stamp to obtain a plurality of time lengths;
selecting a time length greater than a preset threshold value from the time lengths to obtain at least one target time length;
and taking the time stamp of the data corresponding to the at least one target time length as the at least one target time stamp.
In one possible example, the program further comprises instructions for performing the steps of:
acquiring a target CPU utilization rate of the electronic equipment;
and determining the preset threshold corresponding to the target CPU utilization according to the mapping relation between the preset CPU utilization and the threshold.
In one possible example, the timestamp of each of the plurality of data is used to record an expiration time corresponding to the data, and in the comparing the current timestamp with the timestamp corresponding to each of the plurality of data to obtain at least one target timestamp, the program includes instructions for performing the following steps:
acquiring a time stamp corresponding to each data in the plurality of data to obtain a plurality of time stamps;
determining that the plurality of time stamps are earlier than the time stamp of the current time stamp, and obtaining the at least one target time stamp.
In one possible example, in said comparing the current timestamp with a timestamp corresponding to each of the plurality of data to obtain at least one target timestamp, the program includes instructions for:
and comparing the current time stamp with the time stamp corresponding to each data in the plurality of data through a hook function in the RocksDB storage engine to obtain at least one target time stamp.
In one possible example, in the deleting the data corresponding to the at least one target timestamp, the program includes instructions for:
Determining the number of the target time stamps corresponding to the at least one target time stamp;
Determining the number of target threads corresponding to the at least one target timestamp according to a mapping relation between the number of preset timestamps and the number of threads;
and carrying out thread loading according to the number of the target threads, and deleting data corresponding to at least one target time stamp according to the loaded threads, wherein the data corresponding to each target time stamp corresponds to one thread.
In one possible example, the program further comprises instructions for performing the steps of:
receiving a data clearing instruction;
And responding to the data clearing instruction, and executing the step of acquiring the current time stamp.
In one possible example, the program further comprises instructions for performing the steps of:
Acquiring an operation environment parameter;
And executing the step of acquiring the current time stamp when the running environment parameter meets a preset condition.
In one possible example, the program further comprises instructions for performing the steps of:
determining the memory size of the data set to be processed;
And executing the step of acquiring the operation environment parameters when the memory size is larger than the preset memory size.
The foregoing description of the embodiments of the present application has been presented primarily in terms of a method-side implementation. It will be appreciated that the electronic device, in order to achieve the above-described functions, includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The embodiment of the application can divide the functional units of the electronic device according to the method example, for example, each functional unit can be divided corresponding to each function, and two or more functions can be integrated in one processing unit. The integrated units may be implemented in hardware or in software functional units. It should be noted that, in the embodiment of the present application, the division of the units is schematic, which is merely a logic function division, and other division manners may be implemented in actual practice.
Referring to fig. 5A, fig. 5A is a schematic structural diagram of a data processing apparatus according to the present embodiment. The data processing apparatus is applied to an electronic device as shown in fig. 1A, and includes a first acquisition unit 501, a second acquisition unit 502, an alignment unit 503, and a deletion unit 504, wherein,
The first obtaining unit 501 is configured to obtain a data set to be processed, where the data set to be processed includes a plurality of data, and each data corresponds to a time stamp;
the second obtaining unit 502 is configured to obtain a current timestamp when the RocksDB storage engine runs;
the comparing unit 503 is configured to compare the current timestamp with a timestamp corresponding to each of the plurality of data, to obtain at least one target timestamp;
The deleting unit 504 is configured to delete data corresponding to the at least one target timestamp, and combine remaining data in the plurality of data to obtain a target data set.
It can be seen that, the data processing apparatus described in the above embodiment of the present application is applied to an electronic device, and obtains a data set to be processed, where the data set to be processed includes a plurality of data, each data corresponds to a timestamp, when RocksDB storage engines run, the current timestamp is obtained, the current timestamp is compared with the timestamp corresponding to each data in the plurality of data to obtain at least one target timestamp, the data corresponding to the at least one target timestamp is deleted, and the remaining data in the plurality of data to be processed are combined to obtain a target data set, and expired data can be automatically recovered through RocksDB without manual intervention or intensive deletion of data. And the deleting process is shared to each merging stage, so that the influence on the reading and writing of the online service is very small, the whole deleting process is very smooth, the system performance is improved, and the online service efficiency is improved.
In one possible example, the timestamp of each data in the plurality of data is used to record the existence time corresponding to the data, and in the aspect of comparing the current timestamp with the timestamp corresponding to each data in the plurality of data, the comparing unit 503 is specifically configured to:
Determining the time length between the current time stamp and the time stamp corresponding to each data in the plurality of data and the current time stamp to obtain a plurality of time lengths;
selecting a time length greater than a preset threshold value from the time lengths to obtain at least one target time length;
and taking the time stamp of the data corresponding to the at least one target time length as the at least one target time stamp.
In one possible example, the comparison unit 503 is further specifically configured to:
acquiring a target CPU utilization rate of the electronic equipment;
and determining the preset threshold corresponding to the target CPU utilization according to the mapping relation between the preset CPU utilization and the threshold.
In one possible example, the timestamp of each data in the plurality of data is used to record an expiration time corresponding to the data, and in the aspect of comparing the current timestamp with the timestamp corresponding to each data in the plurality of data to obtain at least one target timestamp, the comparing unit 503 is specifically configured to:
acquiring a time stamp corresponding to each data in the plurality of data to obtain a plurality of time stamps;
determining that the plurality of time stamps are earlier than the time stamp of the current time stamp, and obtaining the at least one target time stamp.
In one possible example, in the aspect of comparing the current timestamp with the timestamp corresponding to each of the plurality of data to obtain at least one target timestamp, the comparing unit 503 is specifically configured to:
and comparing the current time stamp with the time stamp corresponding to each data in the plurality of data through a hook function in the RocksDB storage engine to obtain at least one target time stamp.
In one possible example, in the aspect of deleting the data corresponding to the at least one target timestamp, the deleting unit 504 is specifically configured to:
Determining the number of the target time stamps corresponding to the at least one target time stamp;
Determining the number of target threads corresponding to the at least one target timestamp according to a mapping relation between the number of preset timestamps and the number of threads;
and carrying out thread loading according to the number of the target threads, and deleting data corresponding to at least one target time stamp according to the loaded threads, wherein the data corresponding to each target time stamp corresponds to one thread.
In a possible example, as shown in fig. 5B, fig. 5B is a further modified structure of the data processing apparatus shown in fig. 5A, which may further include a receiving unit 505, in which,
The receiving unit 505 is configured to receive a data clear instruction;
The step of acquiring the current timestamp is performed by the second acquisition unit 502 in response to the data clear instruction.
In a possible example, as shown in fig. 5C, fig. 5C is a further variant of the data processing apparatus shown in fig. 5A, which may further comprise, in comparison with fig. 5A, a third acquisition unit 506, wherein,
The third obtaining unit 506 is configured to obtain an operating environment parameter;
and when the running environment parameter meets a preset condition, the second obtaining unit 502 executes the step of obtaining the current time stamp.
In a possible example, as shown in fig. 5D, fig. 5D is a further modified structure of the data processing apparatus shown in fig. 5C, which may further include a determination unit 507, in which,
The determining unit 507 is configured to determine a memory size of the data set to be processed;
And when the memory size is larger than a preset memory size, the third obtaining unit 506 performs the step of obtaining the operating environment parameter.
It may be understood that the functions of each program module of the data processing apparatus of the present embodiment may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the relevant description of the foregoing method embodiment, which is not repeated herein.
The embodiment of the present application also provides a computer storage medium storing a computer program for electronic data exchange, the computer program causing a computer to execute part or all of the steps of any one of the data processing methods described in the above method embodiments.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform part or all of the steps of any one of the data processing methods described in the method embodiments above.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, such as the division of the units, merely a logical function division, and there may be additional manners of dividing the actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units described above may be implemented either in hardware or in software program modules.
The integrated units, if implemented in the form of software program modules, may be stored in a computer-readable memory for sale or use as a stand-alone product. Based on this understanding, the technical solution of the present application may be embodied essentially or partly in the form of a software product, or all or part of the technical solution, which is stored in a memory, and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or part of the steps of the method according to the embodiments of the present application. The memory includes a U disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, etc. which can store program codes.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and the program may be stored in a computer readable memory, which may include a flash disk, a ROM, a RAM, a magnetic disk, an optical disk, etc.
The foregoing has outlined rather broadly the more detailed description of embodiments of the application, wherein the principles and embodiments of the application are explained in detail using specific examples, the above examples being provided solely to facilitate the understanding of the method and core concepts of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (18)
1. A data processing method, applied to an electronic device, comprising:
acquiring a target physiological state parameter of a user;
determining a target emotion type corresponding to the target physiological state parameter;
When the target emotion type is a preset emotion type, acquiring a data set to be processed, wherein the data set to be processed comprises a plurality of data, and each data corresponds to a time stamp;
Acquiring operation environment parameters when the RocksDB storage engine operates;
when the running environment parameters meet preset conditions, acquiring a current time stamp;
Comparing the current time stamp with a time stamp corresponding to each data in the plurality of data to obtain at least one target time stamp;
Deleting data corresponding to the at least one target timestamp, and combining the rest data in the plurality of data to obtain a target data set, wherein the combination comprises compression and is used for finishing and combining the existing data, so that some data which are not valid any more are deleted, the data size and the number of files are reduced, and the reading speed is increased;
when the target physiological state parameter is a heart rate variation curve within a specified time period, the determining the target emotion type corresponding to the target physiological state parameter includes:
Sampling the heart rate change curve to obtain a plurality of heart rate values;
carrying out average value operation according to the plurality of heart rate values to obtain an average heart rate value;
determining a target heart rate level corresponding to the average heart rate value;
Determining a target first emotion value corresponding to the target heart rate level according to a mapping relation between the preset heart rate level and the first emotion value;
Performing mean square error operation according to the plurality of heart rate values to obtain a target mean square error;
Determining a target second emotion value corresponding to the target mean square error according to a mapping relation between the preset mean square error and the second emotion value;
determining a target weight pair corresponding to the target heart rate level according to a mapping relation between the preset heart rate level and the weight pair, wherein the weight pair comprises a first weight and a second weight, the first weight is a weight corresponding to the first emotion value, and the second weight is a weight corresponding to the second emotion value;
weighting operation is carried out according to the target first emotion value, the target second emotion value and the target weight value, so that a final emotion value is obtained;
And determining the target emotion type corresponding to the target emotion value according to a mapping relation between the preset emotion value and the emotion type.
2. The method of claim 1, wherein the timestamp of each of the plurality of data is used for recording a presence time corresponding to the data, and the comparing the current timestamp with the timestamp corresponding to each of the plurality of data to obtain at least one target timestamp includes:
Determining the time length between the current time stamp and the time stamp corresponding to each data in the plurality of data and the current time stamp to obtain a plurality of time lengths;
selecting a time length greater than a preset threshold value from the time lengths to obtain at least one target time length;
and taking the time stamp of the data corresponding to the at least one target time length as the at least one target time stamp.
3. The method according to claim 2, wherein the method further comprises:
acquiring a target CPU utilization rate of the electronic equipment;
and determining the preset threshold corresponding to the target CPU utilization according to the mapping relation between the preset CPU utilization and the threshold.
4. The method of claim 1, wherein the timestamp of each of the plurality of data is used for recording an expiration time corresponding to the data, and the comparing the current timestamp with the timestamp corresponding to each of the plurality of data to obtain at least one target timestamp comprises:
acquiring a time stamp corresponding to each data in the plurality of data to obtain a plurality of time stamps;
determining that the plurality of time stamps are earlier than the time stamp of the current time stamp, and obtaining the at least one target time stamp.
5. The method according to any one of claims 1-4, wherein comparing the current timestamp with a timestamp corresponding to each of the plurality of data to obtain at least one target timestamp includes:
and comparing the current time stamp with the time stamp corresponding to each data in the plurality of data through a hook function in the RocksDB storage engine to obtain at least one target time stamp.
6. The method according to any one of claims 1-4, wherein deleting the data corresponding to the at least one target timestamp comprises:
Determining the number of the target time stamps corresponding to the at least one target time stamp;
Determining the number of target threads corresponding to the at least one target timestamp according to a mapping relation between the number of preset timestamps and the number of threads;
and carrying out thread loading according to the number of the target threads, and deleting data corresponding to at least one target time stamp according to the loaded threads, wherein the data corresponding to each target time stamp corresponds to one thread.
7. The method according to any one of claims 1-4, further comprising:
receiving a data clearing instruction;
And responding to the data clearing instruction, and executing the step of acquiring the current time stamp.
8. The method according to claim 1, wherein the method further comprises:
determining the memory size of the data set to be processed;
And executing the step of acquiring the operation environment parameters when the memory size is larger than the preset memory size.
9. A data processing device is characterized by being applied to electronic equipment and comprising a first acquisition unit, a second acquisition unit, an alignment unit and a deletion unit,
The first acquisition unit is used for acquiring a target physiological state parameter of a user, determining a target emotion type corresponding to the target physiological state parameter, and acquiring a data set to be processed when the target emotion type is a preset emotion type, wherein the data set to be processed comprises a plurality of data, and each data corresponds to one time stamp;
the second acquisition unit is used for acquiring the running environment parameters when the RocksDB storage engine runs, and acquiring the current time stamp when the running environment parameters meet the preset conditions;
The comparison unit is used for comparing the current time stamp with the time stamp corresponding to each data in the plurality of data to obtain at least one target time stamp;
the deleting unit is used for deleting the data corresponding to the at least one target timestamp, combining the rest data in the plurality of data to obtain a target data set, wherein the combination comprises compression and is used for finishing and combining the existing data, so that some data which are not valid any more are deleted, the data scale and the number of files are reduced, and the reading speed is increased;
when the target physiological state parameter is a heart rate variation curve within a specified time period, the determining the target emotion type corresponding to the target physiological state parameter includes:
Sampling the heart rate change curve to obtain a plurality of heart rate values;
carrying out average value operation according to the plurality of heart rate values to obtain an average heart rate value;
determining a target heart rate level corresponding to the average heart rate value;
Determining a target first emotion value corresponding to the target heart rate level according to a mapping relation between the preset heart rate level and the first emotion value;
Performing mean square error operation according to the plurality of heart rate values to obtain a target mean square error;
Determining a target second emotion value corresponding to the target mean square error according to a mapping relation between the preset mean square error and the second emotion value;
determining a target weight pair corresponding to the target heart rate level according to a mapping relation between the preset heart rate level and the weight pair, wherein the weight pair comprises a first weight and a second weight, the first weight is a weight corresponding to the first emotion value, and the second weight is a weight corresponding to the second emotion value;
weighting operation is carried out according to the target first emotion value, the target second emotion value and the target weight value, so that a final emotion value is obtained;
And determining the target emotion type corresponding to the target emotion value according to a mapping relation between the preset emotion value and the emotion type.
10. The apparatus of claim 9, wherein the timestamp of each of the plurality of data is used to record a corresponding time of presence of the data, in the aspect of comparing the current timestamp with the timestamp corresponding to each data in the plurality of data to obtain at least one target timestamp, the comparison unit is specifically configured to:
Determining the time length between the current time stamp and the time stamp corresponding to each data in the plurality of data and the current time stamp to obtain a plurality of time lengths;
selecting a time length greater than a preset threshold value from the time lengths to obtain at least one target time length;
and taking the time stamp of the data corresponding to the at least one target time length as the at least one target time stamp.
11. The device according to claim 10, wherein the comparison unit is further specifically configured to:
acquiring a target CPU utilization rate of the electronic equipment;
and determining the preset threshold corresponding to the target CPU utilization according to the mapping relation between the preset CPU utilization and the threshold.
12. The apparatus of claim 9, wherein the timestamp of each of the plurality of data is used to record an expiration time for the corresponding data, in the aspect of comparing the current timestamp with the timestamp corresponding to each data in the plurality of data to obtain at least one target timestamp, the comparison unit is specifically configured to:
acquiring a time stamp corresponding to each data in the plurality of data to obtain a plurality of time stamps;
determining that the plurality of time stamps are earlier than the time stamp of the current time stamp, and obtaining the at least one target time stamp.
13. The apparatus according to any one of claims 9-12, wherein, in the aspect of comparing the current timestamp with a timestamp corresponding to each of the plurality of data to obtain at least one target timestamp, the comparing unit is specifically configured to:
and comparing the current time stamp with the time stamp corresponding to each data in the plurality of data through a hook function in the RocksDB storage engine to obtain at least one target time stamp.
14. The apparatus according to any one of claims 9-12, wherein, in the deleting the data corresponding to the at least one target timestamp, the deleting unit is specifically configured to:
Determining the number of the target time stamps corresponding to the at least one target time stamp;
Determining the number of target threads corresponding to the at least one target timestamp according to a mapping relation between the number of preset timestamps and the number of threads;
and carrying out thread loading according to the number of the target threads, and deleting data corresponding to at least one target time stamp according to the loaded threads, wherein the data corresponding to each target time stamp corresponds to one thread.
15. The apparatus according to any one of claims 9-12, further comprising a receiving unit, wherein,
The receiving unit is used for receiving a data clearing instruction;
And responding to the data clearing instruction by the second acquisition unit, and executing the step of acquiring the current time stamp.
16. An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-8.
17. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any one of claims 1-8.
18. A computer program product, characterized in that the computer program product comprises a non-transitory computer readable storage medium storing a computer program, the computer program being operable to cause a computer to perform the method of any one of claims 1-8.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/078426 WO2021179122A1 (en) | 2020-03-09 | 2020-03-09 | Data processing method and related products |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115004167A CN115004167A (en) | 2022-09-02 |
| CN115004167B true CN115004167B (en) | 2025-03-14 |
Family
ID=77671157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202080094736.3A Active CN115004167B (en) | 2020-03-09 | 2020-03-09 | Data processing methods and related products |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115004167B (en) |
| WO (1) | WO2021179122A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114358715B (en) * | 2021-12-22 | 2025-03-07 | 北京纪新泰富机电技术股份有限公司 | Data processing method, system, device and storage medium |
| CN116028515A (en) * | 2022-12-26 | 2023-04-28 | 杭州海康威视数字技术股份有限公司 | Data processing method, device and electronic equipment based on Flink framework |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8856083B2 (en) * | 2006-06-09 | 2014-10-07 | Oracle International Corporation | Framework to optimize delete all row operations on database objects to improve throughput, query ability and flashback |
| US9229960B2 (en) * | 2013-02-11 | 2016-01-05 | International Business Machines Corporation | Database management delete efficiency |
| US11461010B2 (en) * | 2015-07-13 | 2022-10-04 | Samsung Electronics Co., Ltd. | Data property-based data placement in a nonvolatile memory device |
| CN107193839A (en) * | 2016-03-15 | 2017-09-22 | 阿里巴巴集团控股有限公司 | Data aggregation method and device |
| CN107229649B (en) * | 2016-03-25 | 2019-07-12 | 腾讯科技（深圳）有限公司 | Data update system and method |
| CN107798009A (en) * | 2016-08-31 | 2018-03-13 | 阿里巴巴集团控股有限公司 | Data aggregation method, apparatus and system |
| GB201620476D0 (en) * | 2016-12-02 | 2017-01-18 | Omarco Network Solutions Ltd | Computer-implemented method of predicting performance data |
| EP3706398A4 (en) * | 2017-11-29 | 2021-01-06 | Huawei Technologies Co., Ltd. | Pressure measurement method and terminal |
| CN110135226B (en) * | 2018-02-09 | 2023-04-07 | 腾讯科技（深圳）有限公司 | Expression animation data processing method and device, computer equipment and storage medium |
| CN110134697B (en) * | 2019-05-22 | 2024-01-16 | 南京大学 | An automatic parameter tuning method, device and system for key-value storage engines |
| CN110353704B (en) * | 2019-07-12 | 2022-02-01 | 东南大学 | Emotion evaluation method and device based on wearable electrocardiogram monitoring |
| CN110811646B (en) * | 2019-10-23 | 2022-05-17 | 北京点滴灵犀科技有限公司 | Emotional stress comprehensive detection and analysis method and device |
-
2020
- 2020-03-09 CN CN202080094736.3A patent/CN115004167B/en active Active
- 2020-03-09 WO PCT/CN2020/078426 patent/WO2021179122A1/en not_active Ceased
Non-Patent Citations (1)
| Title |
|---|
| 基于RocksDB实现精准的TTL过期淘汰机制;OPPO数智技术;《https://segmentfault.com/a/1190000021185954》;20191203;1-10 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021179122A1 (en) | 2021-09-16 |
| CN115004167A (en) | 2022-09-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3502881B1 (en) | Method for preloading application, storage medium, and terminal device | |
| US20190188007A1 (en) | Method for Preloading Application, Storage Medium, and Terminal Device | |
| CN107092520B (en) | Recommended method, device, storage medium and mobile terminal for application program | |
| CN111338725B (en) | Interface layout method and related product | |
| CN107368400B (en) | CPU monitoring method and device, computer readable storage medium and mobile terminal | |
| WO2021232289A1 (en) | Image pulling method and related product | |
| CN106919313A (en) | The startup method and mobile terminal of a kind of application program | |
| CN111353608B (en) | Model transplantation method and related equipment | |
| CN107809542A (en) | Application program control method and device, storage medium and electronic equipment | |
| US20210278937A1 (en) | Method for Browsing Application Folder and Electronic Device | |
| WO2015027856A1 (en) | Information feedback method, apparatus, and terminal | |
| CN108874466A (en) | Control call method, electronic device and computer readable storage medium | |
| CN107544842A (en) | Application program processing method and device, computer equipment, storage medium | |
| CN106528335A (en) | A data backup method, device and terminal | |
| CN107562539A (en) | Applied program processing method and device, computer equipment, storage medium | |
| WO2019101043A1 (en) | Image processing method, terminal, computer storage medium and computer program | |
| JP6915074B2 (en) | Message notification method and terminal | |
| CN115004167B (en) | Data processing methods and related products | |
| CN113366523B (en) | Resource push methods and related products | |
| CN106502833B (en) | Data backup method and device | |
| CN106230919B (en) | File uploading method and device | |
| CN107832131A (en) | Closing application program method, apparatus, storage medium and electronic equipment | |
| CN113419795B (en) | Call relation display method and device, computer equipment and storage medium | |
| CN110058980A (en) | Terminal starts time method for early warning, electronic device and computer readable storage medium | |
| CN107391196B (en) | Application program updating method and mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |