CN115292268B

CN115292268B - Log uploading method and device, electronic equipment and storage medium

Info

Publication number: CN115292268B
Application number: CN202210897082.9A
Authority: CN
Inventors: 尹逊宫
Original assignee: Beijing QIYI Century Science and Technology Co Ltd
Current assignee: Beijing QIYI Century Science and Technology Co Ltd
Priority date: 2022-07-28
Filing date: 2022-07-28
Publication date: 2026-01-20
Anticipated expiration: 2042-07-28
Also published as: CN115292268A

Abstract

The embodiment of the invention provides a log uploading method, a device, electronic equipment and a storage medium, wherein the method comprises the steps of obtaining a log to be uploaded, wherein the log to be uploaded carries a server type to be uploaded; the method comprises the steps of determining a target storage space corresponding to a server type according to the server type and a preset corresponding relation between the server type and the storage space, storing a log to be uploaded into the target storage space, reading the log to be uploaded from the target storage space when a log uploading condition is met, and uploading the log to be uploaded to a target server according to the server type corresponding to the log to be uploaded. Because the electronic equipment can finish the storage and uploading of the logs according to the corresponding relation between the server type and the storage space for different types of logs, only one set of log uploading system is needed, the code quantity is reduced, and when the logs are uploaded, only the corresponding relation between the corresponding server type and the storage space is needed to be stored, so that a new system is not needed to be developed.

Description

Log uploading method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of data transmission technologies, and in particular, to a method and apparatus for uploading logs, an electronic device, and a storage medium.

Background

In order to better analyze user behaviors, more accurate recommended products are given to each user, and meanwhile, in order to analyze whether the design of the products is reasonable or not, whether the requirements of the current user can be met or not, various logs of the user uploaded by APP (Application) clients of the user need to be analyzed through a server. For example, service logs recorded with user behaviors, such as a page stay log of a user and a browse record log of the user, are uploaded to a service server for analysis, monitoring logs representing the health condition of the APP, such as a loading time log, a crash log and an interface time-consuming log of the page, are uploaded to the monitoring server for analysis, and some APP can be used as a traceless buried point for later deep digging of user attributes or habit.

Because the formats, the processing modes and the butt-joint server systems of the logs are different, a set of log uploading system is required to be specially arranged for each log to store and upload the log, so that the APP side has a plurality of redundant codes, and if uploading of one type of log is required to be newly added, a set of codes is required to be newly developed, thus the development workload is large, and the later expansion and maintenance are not facilitated.

Disclosure of Invention

The embodiment of the invention aims to provide a log uploading method, a log uploading device, electronic equipment and a storage medium, so as to reduce the code quantity of a log uploading system and facilitate the expansion of subsequent log uploading. The specific technical scheme is as follows:

In a first aspect, an embodiment of the present invention provides a log uploading method, where the method includes:

Acquiring a log to be uploaded, wherein the log to be uploaded carries the type of a server to be uploaded;

Determining a target storage space corresponding to the server type according to the server type and the corresponding relation between the preset server type and the storage space;

storing the log to be uploaded to the target storage space;

When the log uploading condition is met, the log to be uploaded is read from the target storage space, and the log to be uploaded is uploaded to a target server according to the server type corresponding to the log to be uploaded.

Optionally, the step of determining the target storage space corresponding to the server type according to the server type and a preset correspondence between the server type and the storage space includes:

and sending the log to be uploaded to a manager corresponding to each storage space, so that the manager corresponding to each storage space determines whether the corresponding storage space is used for storing the log to be uploaded according to the type of the server carried by the log to be uploaded and the corresponding relation between the preset type of the server and the storage space, and if so, determining the corresponding storage space as a target storage space.

Optionally, the method further comprises:

Judging whether the data volume in the storage space is larger than a first preset threshold value according to a first preset time interval;

if yes, deleting the data in the storage space by adopting a preset deleting mode corresponding to the storage space;

judging whether the data amount in the deleted storage space is larger than the first preset threshold value or not;

if so, deleting the data in the storage space according to the storage time so that the data volume in the storage space is not greater than the first preset threshold value.

Optionally, before the step of reading the log to be uploaded from the target storage space and according to the server type corresponding to the log to be uploaded, the method further includes:

Judging whether the data volume in the target storage space reaches a second preset threshold value or not according to a second preset time interval;

If so, it is determined that the log upload condition is satisfied.

Optionally, after the step of uploading the log to be uploaded to the target server, the method further includes:

judging whether the log to be uploaded is successfully uploaded;

If the uploading is successful, deleting the log to be uploaded from the target storage space;

If the uploading fails, determining a target log processing mode corresponding to the server type according to the server type and the corresponding relation between the preset server type and the log processing mode, wherein the log processing mode comprises deleting and retransmitting logs;

and processing the log to be uploaded according to the target log processing mode.

Optionally, the step of obtaining the log to be uploaded includes:

acquiring original logs generated by each preset buried point;

And converting the original log into a log to be uploaded in a target format according to a preset data processing mode.

Optionally, before the step of uploading the log to be uploaded to the target server according to the server type corresponding to the log to be uploaded, the method further includes:

Determining a target callback interface corresponding to the server type according to the server type and the corresponding relation between the preset server type and the callback interface;

And calling the callback interface to convert the log to be uploaded from a target format into a log format corresponding to the server type.

Optionally, the storage space at least comprises a hard disk, a memory and a database.

In a second aspect, an embodiment of the present invention provides a log uploading device, where the device includes:

the system comprises a log acquisition module, a server and a server, wherein the log acquisition module is used for acquiring a log to be uploaded, and the log to be uploaded carries the type of the server to be uploaded;

the storage space determining module is used for determining a target storage space corresponding to the server type according to the server type and the corresponding relation between the preset server type and the storage space;

The log storage module is used for storing the log to be uploaded to the target storage space;

And the log uploading module is used for reading the log to be uploaded from the target storage space when the log uploading condition is met, and uploading the log to be uploaded to a target server according to the server type corresponding to the log to be uploaded.

Optionally, the storage space determining module is specifically configured to:

Optionally, the apparatus further includes:

the first storage space judging module is used for judging whether the data volume in the storage space is larger than a first preset threshold value according to a first preset time interval;

the first data deleting module is used for deleting the data in the storage space by adopting a preset deleting mode corresponding to the storage space when the judging result of the first storage space judging module is yes;

The second storage space judging module is used for judging whether the data amount in the deleted storage space is larger than the first preset threshold value or not;

And the second data deleting module is used for deleting the data in the storage space according to the storage time when the judging result of the second storage space judging module is yes, so that the data volume in the storage space is not larger than the first preset threshold value.

Optionally, the apparatus further includes:

the third storage space judging module is used for judging whether the data amount in the target storage space reaches a second preset threshold value or not according to a second preset time interval;

and the uploading triggering module is used for determining that the log uploading condition is met when the judgment result is yes.

Optionally, the apparatus further includes:

the uploading judging module is used for judging whether the log to be uploaded is successfully uploaded;

the log deleting module is used for deleting the log to be uploaded from the target storage space when the judgment result is yes;

The log processing mode determining module is used for determining a target log processing mode corresponding to the server type according to the server type and the corresponding relation between the preset server type and the log processing mode when the judging result is negative, wherein the log processing mode comprises deleting and retransmitting logs;

And the log processing module is used for processing the log to be uploaded according to the target log processing mode.

Optionally, the log obtaining module includes:

the original log acquisition unit is used for acquiring original logs generated by each preset buried point;

the log format conversion unit is used for converting the original log into a log to be uploaded in a target format according to a preset data processing mode.

Optionally, the apparatus further includes:

the callback interface determining module is used for determining a target callback interface corresponding to the server type according to the server type and the corresponding relation between the preset server type and the callback interface;

and the log format conversion module is used for calling the callback interface to convert the log to be uploaded from a target format into a log format corresponding to the server type.

In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

A processor for implementing the method steps of any of the above first aspects when executing a program stored on a memory.

In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium having a computer program stored therein, which when executed by a processor, implements the method steps of any of the first aspects described above.

According to the scheme provided by the embodiment of the invention, the electronic equipment can acquire the log to be uploaded, wherein the log to be uploaded carries the server type to be uploaded, a target storage space corresponding to the server type is determined according to the server type and the corresponding relation between the preset server type and the storage space, the log to be uploaded is stored in the target storage space, when the log uploading condition is met, the log to be uploaded is read from the target storage space, and the log to be uploaded is uploaded to the target server according to the server type corresponding to the log to be uploaded. The electronic equipment can determine the target storage space for storing the logs to be uploaded and upload the logs to be uploaded to the corresponding target servers according to the types of the servers contained in the logs to be uploaded and the corresponding relation between the preset server types and the storage space, so that the logs to be uploaded of different types can be uploaded only by one set of log uploading system, one set of log uploading system does not need to be specially written for each type of logs to be uploaded, and the code quantity of the log uploading system can be reduced. And when other types of logs to be uploaded are expanded subsequently, the storage and uploading of the logs to be uploaded can be realized only by adding the corresponding relation between the type of the server to be uploaded and the storage space in the corresponding relation, a new log uploading system is not required to be developed, and the storage and uploading of the logs to be uploaded of other types are convenient to expand.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.

FIG. 1 is a flowchart of a log uploading method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a server configuration information storage manner according to the embodiment shown in FIG. 1;

FIG. 3 is a schematic diagram of a log distribution storage scheme based on the embodiment shown in FIG. 1;

FIG. 4 is another schematic diagram of a log distribution storage scheme based on the embodiment shown in FIG. 1;

FIG. 5 is a flow chart of a data deletion scheme based on the embodiment shown in FIG. 1;

FIG. 6 is a flow chart of a log upload condition determination method based on the embodiment shown in FIG. 1;

FIG. 7 is a flow chart of a log uploading method based on the embodiment shown in FIG. 1;

FIG. 8 is a flow chart of an uploaded log processing method based on the embodiment shown in FIG. 1;

FIG. 9 is a flow chart of a method for obtaining logs to be uploaded based on the embodiment shown in FIG. 1;

FIG. 10 is a flow chart of a log format conversion method based on the embodiment shown in FIG. 1;

FIG. 11 is a schematic diagram of a topology structure on which a log uploading method according to an embodiment of the present invention is based;

fig. 12 is a schematic structural diagram of a log uploading device according to an embodiment of the present invention;

fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

In order to reduce the code amount of a log uploading system and facilitate the expansion of subsequent log uploading, embodiments of the present invention provide a log uploading method, apparatus, electronic device, computer readable storage medium and computer program product. The following first describes a log uploading method provided by the embodiment of the present invention.

The log uploading method provided by the embodiment of the invention can be applied to any electronic equipment needing log uploading, for example, a terminal, a server and the like, and is not particularly limited. For clarity of description, hereinafter, referred to as an electronic device.

As shown in fig. 1, a log uploading method includes:

s101, acquiring a log to be uploaded;

and the log to be uploaded carries the server type to be uploaded.

S102, determining a target storage space corresponding to the server type according to the server type and the corresponding relation between the preset server type and the storage space;

s103, storing the log to be uploaded into the target storage space;

And S104, when the log uploading condition is met, reading the log to be uploaded from the target storage space, and uploading the log to be uploaded to a target server according to the server type corresponding to the log to be uploaded.

It can be seen that, in the scheme provided by the embodiment of the invention, the electronic device can acquire the log to be uploaded, wherein the log to be uploaded carries the server type to be uploaded, a target storage space corresponding to the server type is determined according to the server type and the corresponding relation between the preset server type and the storage space, the log to be uploaded is stored in the target storage space, when the log uploading condition is met, the log to be uploaded is read from the target storage space, and the log to be uploaded is uploaded to the target server according to the server type corresponding to the log to be uploaded. The electronic equipment can determine the target storage space for storing the logs to be uploaded and upload the logs to be uploaded to the corresponding target servers according to the types of the servers contained in the logs to be uploaded and the corresponding relation between the preset server types and the storage space, so that the logs to be uploaded of different types can be uploaded only by one set of log uploading system, one set of log uploading system does not need to be specially written for each type of logs to be uploaded, and the code quantity of the log uploading system can be reduced. And when other types of logs to be uploaded are expanded subsequently, the storage and uploading of the logs to be uploaded can be realized only by adding the corresponding relation between the type of the server to be uploaded and the storage space in the corresponding relation, a new log uploading system is not required to be developed, and the storage and uploading of the logs to be uploaded of other types are convenient to expand.

When the server monitors and analyzes the behaviors of the electronic device and the user using the electronic device, the server needs to acquire logs to be uploaded generated by a plurality of embedded points which are arranged in the electronic device in advance, so that the electronic device can acquire the logs to be uploaded generated by each embedded point and upload the logs to the corresponding server, and the server can monitor and analyze the behaviors of the electronic device and the user using the electronic device.

The embedded point can be used for monitoring events occurring in the running process of the electronic equipment and judging the events, and when the events concerned by the embedded point occur, the embedded point can capture the events to generate logs to be uploaded. And because the server types corresponding to each embedded point are different, for example, the server type corresponding to the recommended embedded point is a recommended server, the server type corresponding to the monitored embedded point is a monitoring server, and the server corresponding to the traceless embedded point is a server for analyzing user attributes and habits, when the embedded point generates a log to be uploaded, the log to be uploaded can carry the server type to be uploaded, so that the subsequent electronic equipment processes the log to be uploaded.

When the server needs to monitor and analyze the behaviors of the terminal and the terminal user, the log to be uploaded can be a log generated by the terminal, and when the server providing the business service needs to monitor and analyze, the log to be uploaded can be a log generated by the server, and the log to be uploaded is not particularly limited.

After obtaining the log to be uploaded, the electronic device may execute the step S102 to store the log to be uploaded. When the electronic device stores the log to be uploaded, the electronic device can allocate a corresponding storage space for the log to be uploaded because the size, importance, time delay requirements and fault tolerance requirements of the log to be uploaded to different server types may be different. The storage space may include a hard disk, a memory, a database, a floppy disk, an optical disk, etc., which are not particularly limited herein.

In one embodiment, hard disk storage is suitable for scenes with low requirements on timeliness and large log amounts, memory storage is suitable for scenes with high requirements on timeliness but low requirements on fault tolerance, and database storage is suitable for scenes with medium requirements on timeliness and high requirements on fault tolerance. Therefore, the logs to be uploaded which need to be uploaded to the real-time recommendation server have high requirements on time delay and low requirements on fault tolerance, so that the logs to be uploaded can be stored by being distributed with memory, the logs to be uploaded which need to be uploaded to the monitoring server have medium requirements on time delay and high requirements on fault tolerance, so that the logs to be uploaded can be distributed with a database for storage, and the logs to be uploaded which need to be uploaded to a server for analyzing user attributes and habits have low requirements on time delay and large data volume, so that the logs to be uploaded can be distributed with a hard disk for storage.

The time delay requirements corresponding to the logs to be uploaded to different server types can be divided according to the speed requirements of uploading the logs to be uploaded to the corresponding servers. As an implementation mode, the log to be uploaded needs to be uploaded to the corresponding server at a second-level speed, so that the time ductility requirement is high, the log to be uploaded can be regarded as being waiting for the time ductility requirement when being uploaded to the corresponding server on the generating day, and the log to be uploaded can be regarded as being low when being uploaded to the corresponding server on the generating week.

For fault tolerance requirements corresponding to logs to be uploaded, which are uploaded to different server types, the maximum loss rate allowed in the process of uploading the logs to be uploaded to the corresponding servers can be divided. As an implementation mode, the maximum allowable loss rate of the log to be uploaded in the process of uploading to the corresponding server is 2%, so that the requirement of fault tolerance is considered to be high, the maximum allowable loss rate of the log to be uploaded in the process of uploading to the corresponding server is 5%, so that the requirement of fault tolerance is considered to be medium, the maximum allowable loss rate of the log to be uploaded in the process of uploading to the corresponding server is considered to be 20%, and the requirement of fault tolerance is considered to be low. The specific division criteria for the latency requirement and the fault tolerance requirement can be set correspondingly according to the actual application requirement, and are not limited in detail herein.

In order to facilitate the storage of the log to be uploaded, a corresponding relation between the server type and the storage space can be pre-established, so that after the electronic device obtains the log to be uploaded carrying the server type to be uploaded, the target storage space for storing the log to be uploaded can be determined according to the server type carried by the log to be uploaded and the corresponding relation between the preset server type and the storage space, and the target storage space is stored.

After the electronic device stores the log to be uploaded in the target storage space, in the step S104, when the log uploading condition is satisfied, the electronic device may read the log to be uploaded from the target storage space, and upload the log to be uploaded to the target server according to the server type corresponding to the log to be uploaded. The electronic device may periodically determine whether the log uploading condition is satisfied, or may determine whether the log uploading condition is satisfied according to other rules, for example, may determine whether the log uploading condition is satisfied at regular time, which is not limited herein.

In one embodiment, in the process of arranging the embedded point by the electronic device, the server may send the server type, the server address, etc. to the electronic device, and the electronic device may pre-establish a corresponding relationship between the server type and the server address according to the above content, so that after reading the log to be uploaded, the electronic device may determine the server address according to the server type to be uploaded by the log to be uploaded and the preset corresponding relationship between the server type and the server address, and further may upload the log to be uploaded to the target server according to the server address.

In order to facilitate the storage and uploading of the log to be uploaded, the electronic device may have a storage management unit, where the storage management unit is configured to manage the storage and uploading operations of the log to be uploaded to different server types, and when the storage management unit is initialized, the storage management unit may pre-store configuration information corresponding to the server, for example, name: type of server, url: address of server, time: uploading time interval of the log to be uploaded to the server, type: storage space for storing the log to be uploaded to the server, and so on, and may also store a callback function for calling a callback interface. In one embodiment, as shown in fig. 2, the storage management unit may receive and store the server configuration information sent by the server during initialization, and in another embodiment, it is reasonable to use the storage management unit to receive and store the server configuration information actively set by the technician, which is not limited herein specifically.

According to the scheme provided by the embodiment of the invention, the electronic equipment can determine the target storage space for storing the logs to be uploaded and upload the logs to be uploaded to the corresponding target servers according to the types of the servers contained in the logs to be uploaded and the corresponding relation between the preset types of the servers and the storage space, so that the logs to be uploaded of different types can be uploaded only by one set of log uploading system, one set of log uploading system does not need to be specially written for each type of logs to be uploaded, and the code quantity of the log uploading system can be reduced. And when other types of logs to be uploaded are expanded subsequently, the storage and uploading of the logs to be uploaded can be realized only by adding the corresponding relation between the type of the server to be uploaded and the storage space in the corresponding relation, a new log uploading system is not required to be developed, and the storage and uploading of the logs to be uploaded of other types are convenient to expand.

As an implementation manner of the embodiment of the present invention, the step of determining, according to the server type and a preset correspondence between the server type and a storage space, a target storage space corresponding to the server type may include:

In the process of arranging the buried point in the electronic equipment, the server can send the server type of the server and the storage space corresponding to the log generated by the buried point to the electronic equipment, and the electronic equipment can send the corresponding relation to the manager corresponding to each storage space, and each manager stores the corresponding relation.

For the case that the storage space includes a hard disk, a memory and a database, in one embodiment, as shown in fig. 3, after the electronic device obtains the log to be uploaded, the log to be uploaded may be distributed to the managers corresponding to each storage space, the manager corresponding to each storage space determines whether the corresponding storage space is used for storing the log to be uploaded, and the storage space for storing the log to be uploaded is determined as the target storage space.

In another embodiment, as shown in fig. 4, after the electronic device obtains the log to be uploaded, the log to be uploaded may be sent to a manager corresponding to the hard disk first, the manager corresponding to the hard disk determines whether the hard disk is used for storing the log to be uploaded, if yes, the hard disk is determined to be a target storage space, if no, the log to be uploaded is sent to a manager corresponding to the memory, the manager corresponding to the memory is determined to be used for storing the log to be uploaded, if no, the log to be uploaded is sent to the manager corresponding to the database, the manager corresponding to the database is determined to be used for storing the log to be uploaded, if yes, the database is determined to be a target storage space, and if no, the current storage space is determined to be unavailable for storing the log to be uploaded.

It should be understood that, although fig. 4 only shows the manner in which the log to be uploaded is sent to the manager corresponding to the hard disk first, then sent to the manager corresponding to the memory, and finally sent to the manager corresponding to the database, in practice, the order in which the log to be uploaded is sent to the managers corresponding to the storage spaces may be adjusted according to the actual requirement, which is not limited to the manner shown in fig. 4.

In the scheme provided by the embodiment of the invention, the electronic equipment can send the log to be uploaded to the manager corresponding to each storage space, the manager corresponding to each storage space determines the target storage space, the pressure of the electronic equipment can be relieved to a certain extent, and meanwhile, when the manager determines that the storage space corresponding to the manager is the target storage space, the log to be uploaded can be directly stored, so that the log to be uploaded can be quickly stored.

As shown in fig. 5, the method may further include:

s501, judging whether the data volume in the storage space is larger than a first preset threshold value according to a first preset time interval;

In order to avoid the overlarge data amount in the storage space, the electronic device may detect the data amount in the storage space according to a first preset time interval, determine whether the data amount in the storage space is greater than a first preset threshold, and the first preset thresholds corresponding to the storage spaces may be the same or different, which is not limited specifically herein.

S502, if so, deleting the data in the storage space by adopting a preset deletion mode corresponding to the storage space;

If the judgment result is yes, the fact that the data volume in the storage space is too large possibly affects the storage of the log to be uploaded, so that the data in the storage space needs to be deleted, and at the moment, the electronic equipment can delete the data in the storage space according to a deletion mode corresponding to the storage space.

In one embodiment, the electronic device may store the deletion modes corresponding to the storage spaces in advance, for example, for the memory, the temporary storage function is implemented, and the stored data has a high latency requirement, so the deletion mode corresponding to the memory may be to delete the data whose time period expires. Those skilled in the art may correspondingly design the deletion mode corresponding to each storage space according to the storage characteristics of the storage space and/or the characteristics of the stored data, which is not limited herein.

S503, judging whether the data amount in the deleted storage space is larger than the first preset threshold value;

s504, if so, deleting the data in the storage space according to the storage time so that the data volume in the storage space is not larger than the first preset threshold value.

The deletion method corresponding to the storage space can delete specific data, for example, the deletion method corresponding to the memory can delete the data with the time-lapse, but the data volume in the storage space after deleting the data according to the deletion method cannot be ensured to be not larger than a first preset threshold, for example, the data volume occupied by the data with the time-lapse of the memory is small, and at the moment, the data volume in the memory cannot be obviously reduced by deleting the data with the time-lapse. Therefore, after deleting the data in the storage space in a deleting manner corresponding to the storage space, the electronic device can detect the data amount in the storage space again, judge whether the data amount in the storage space is greater than a first preset threshold, if not, indicate that the current data amount in the storage space meets the requirement, and then detect the storage space at intervals of a first preset time interval. If yes, the data quantity in the storage space still does not meet the requirement, and further deletion is needed, at this time, the electronic device can delete the data in the storage space in a deleting mode which is common to all the storage spaces, that is, the data with the earliest storage time in the storage space is preferentially deleted according to the sequence of the storage time until the data quantity in the storage space is not greater than a first preset threshold value.

According to the scheme provided by the embodiment of the invention, the electronic equipment can detect the data amount in the storage space according to the first preset time interval, and delete the data in the storage space when the data amount in the storage space is larger than the first preset threshold value, so that the storage space is always provided with an idle position, and further the electronic equipment can successfully store the log to be uploaded in the storage space when storing the log to be uploaded. When the electronic equipment deletes the data in the storage space, the deleting mode corresponding to each storage space can be adopted for deleting, and when the deleting mode can not meet the requirement, the deleting mode universal to each storage space is adopted for deleting, so that the influence of the data deleting operation on the normal data in the storage space is reduced.

As shown in fig. 6, before the step of reading the log to be uploaded from the target storage space and according to the server type corresponding to the log to be uploaded, the method may further include:

s601, judging whether the data volume in the target storage space reaches a second preset threshold value according to a second preset time interval;

S602, if so, determining that the log uploading condition is met.

In order to determine whether the log to be uploaded reaches a preset log uploading condition, the electronic device may detect the data amount in the target storage space according to a second preset time interval, and determine whether the data amount in the target storage space reaches a second preset threshold, where the second time interval may be uniformly set by the electronic device for the log to be uploaded to each server type, or may be set by the electronic device for the log to be uploaded to each server type, for example, when a buried point is arranged in the electronic device by the server, the uploading time interval of the log to be uploaded corresponding to the server type may be sent to the electronic device, and the electronic device uses the uploading time interval corresponding to the server type as the second preset time interval corresponding to the log to be uploaded to the server type.

In one embodiment, as shown in fig. 7, the electronic device may first detect whether the corresponding application is started, for example, when the electronic device is a terminal, whether an APP in the terminal is started, and when the electronic device is a server, whether the program in the server is started. This is reasonable and is not particularly limited herein.

When the electronic device detects that the corresponding application program is started, in order to not occupy the starting time of the application program and ensure the smoothness of the starting of the application program, the electronic device can delay for a certain time, for example, 10 seconds, then send wake-up information to a log sending thread in the electronic device, then judge whether the log sending thread is connected with a network, if the log sending thread is not connected with the network, then interval a second preset time interval, for example, 30 seconds, send wake-up information to the log sending thread in the electronic device again, and if the log sending thread is connected with the network, then further judge whether the data volume in the target storage space reaches a second preset threshold, for example, judge whether the data volume in the target storage space is larger than 100M.

If the data volume in the target storage space reaches the second preset threshold, the current target storage space is larger in data volume, and the uploading step of the log to be uploaded can be performed, namely when the data volume in the target storage space reaches the second preset threshold, the log uploading condition can be determined to be met.

If the data amount in the target storage space does not reach the second preset threshold, the fact that the data amount in the current target storage space is smaller is indicated, at this time, a log uploading step to be uploaded is not needed, and a log to be uploaded generated later can be uploaded after being stored in the target storage space, so that in order to avoid frequent uploading occupying resources of the electronic device, the electronic device does not carry out the uploading step when determining that the data amount in the storage space does not reach the second preset threshold, and resends wakeup information at intervals of the second preset time interval, and judgment on whether network connection exists or not and the data amount in the target storage space is carried out again.

According to the scheme provided by the embodiment of the invention, the electronic equipment can determine whether the log uploading condition is met by judging whether the data volume in the target storage space reaches the second preset threshold value, so that the log to be uploaded is uploaded only when the data volume of the log to be uploaded in the target storage space reaches a certain quantity, and the problem that resources of the electronic equipment are occupied due to frequent uploading is solved. Meanwhile, in the process, the electronic equipment can also delay the wake-up log sending thread and judge whether the log sending thread is connected with a network or not so as to ensure the smoothness of starting an application program and the successful uploading of the log to be uploaded.

As shown in fig. 8, after the step of uploading the log to be uploaded to the target server, the method may further include:

s801, judging whether the log to be uploaded is successfully uploaded;

because there may be a failure of the server to receive the log to be uploaded, or a disconnection condition in the uploading process of the log to be uploaded, the electronic device may further determine whether the log to be uploaded is successfully uploaded after uploading the log to the target server, for example, after uploading the log to be uploaded to the target server, may determine whether response information sent by the target server is received, if yes, then determine that the log to be uploaded is successfully uploaded, and if no, determine that the log to be uploaded is not successfully uploaded.

S802, deleting the log to be uploaded from the target storage space if the uploading is successful;

After the log is successfully uploaded to the server, the log to be uploaded stored in the target storage space is lost, so that the electronic equipment can delete the log to be uploaded from the target storage space after the log to be uploaded is successfully uploaded, and space cleaning of the target storage space is realized.

In one embodiment, as shown in fig. 7, after the electronic device deletes the log to be uploaded from the target storage space, wake-up information may be sent to the log sending thread again at intervals of a second preset time interval, and the determination of whether there is a network connection and the data amount in the target storage space is performed again.

S803, if the uploading fails, determining a target log processing mode corresponding to the server type according to the server type and the corresponding relation between the preset server type and the log processing mode;

The log processing mode comprises deleting and retransmitting logs.

S804, processing the log to be uploaded according to the target log processing mode.

Because the characteristics of the logs to be uploaded to different server types are different, the log processing mode to be performed after the logs to be uploaded to different server types are failed to upload is also different, for example, the logs to be uploaded to the real-time recommendation server have high time delay requirements, so that the condition that the logs to be uploaded expire may exist after the logs to be uploaded are failed to upload, and the electronic equipment can delete the logs to be uploaded when the logs to be uploaded are failed to upload. For example, the log to be uploaded to the server for analyzing the user attribute and habit has a low requirement for latency, but the data amount of the log is relatively large, so that the log to be uploaded may be retransmitted the next time the log uploading condition is satisfied without deleting the log to be uploaded after the uploading failure. Therefore, when uploading of the log to be uploaded fails, the electronic device can determine whether to delete or retransmit the log to be uploaded according to the type of the server carried by the log to be uploaded and the corresponding relation between the preset type of the server and the log processing mode, and execute the corresponding operation.

In one embodiment, as shown in fig. 7, if uploading of the log to be uploaded fails, the electronic device may determine whether to delete the log to be uploaded, if yes, delete the log to be uploaded, resend the wakeup information at intervals of a second preset time interval, and determine whether to have network connection and the data amount in the target storage space again, if no, resend the wakeup information at intervals of the second preset time interval, and determine whether to have network connection and the data amount again.

In the scheme provided by the embodiment of the invention, when the uploading of the log to be uploaded is successful, the electronic equipment can delete the log to be uploaded from the target storage space, and when the uploading of the log to be uploaded fails, the processing mode of the log to be uploaded can be determined according to the type of the server to be uploaded of the log to be uploaded, so that the cleaning of the target storage space is realized, and the successful uploading of the log to be uploaded is also ensured to a certain extent.

As shown in fig. 9, the step of obtaining the log to be uploaded may include:

s901, obtaining original logs generated by each preset buried point;

s902, converting the original log into a log to be uploaded in a target format according to a preset data processing mode.

The electronic equipment performs unified storage operation and uploading operation on the logs generated by the preset buried points, and the formats and contents of the logs generated by the different preset buried points are different, so that in order to facilitate unified processing on the logs generated by the different preset buried points, the electronic equipment can firstly acquire original logs generated by the preset buried points, then convert the original logs generated by the preset buried points into logs to be uploaded in a target format according to a preset data processing mode, unified log formats is realized, and subsequent processing on the logs to be uploaded is facilitated.

In one embodiment, the electronic device may represent information of the original log in a general field, so as to convert the original log into a log to be uploaded in a target format, for example, the system time generated by the original log is represented by t, the system time is specified to be accurate to ms, v represents a version number of an application program generating the original log, the version number is specified to be represented by a string type, c represents a server type to be uploaded by the original log, d represents a serial self-increasing ID, d represents an order of the ID for identifying the original log generated in a preset embedded point, so as to be convenient for tracking log loss and order of the serial log, uid represents a unique device ID for identifying the unique electronic device, and other data is transmitted in the form of an on object, wherein the general field is indicated by "_head" so as to avoid conflict with service data.

According to the scheme provided by the embodiment of the invention, the electronic equipment can convert the original logs generated by each preset buried point into the logs to be uploaded in the target format, so that the unification of the log formats is realized, and the electronic equipment can conveniently perform unified processing on the logs from different preset buried points.

As shown in fig. 10, before the step of uploading the log to be uploaded to the target server according to the server type corresponding to the log to be uploaded, the method may further include:

s1001, determining a target callback interface corresponding to the server type according to the server type and the corresponding relation between the preset server type and the callback interface;

S1002, calling the callback interface to convert the log to be uploaded from a target format into a log format corresponding to the server type.

Because the preset embedded points are arranged in the electronic equipment in advance by the server, the format of the log processed by the server corresponds to the format of the original log generated by the preset embedded points, but when the electronic equipment acquires the log to be uploaded, the original log generated by each preset embedded point is converted into the log to be uploaded in a target format, so that the server needs to convert the format of the log to be uploaded after receiving the log to be uploaded in the target format, and then the operations of reading and processing the log to be uploaded can be performed. Therefore, in order to directly read and process the log to be uploaded by the server, the electronic device can convert the format of the log to be uploaded into the log format corresponding to the server type before uploading the log to be uploaded to the server.

In an embodiment, when a preset embedding point is arranged in the electronic device by the server, the server type and the corresponding callback interface of the server can be sent to the electronic device, and the electronic device can pre-establish a corresponding relation between the server type and the callback interface according to the content, wherein the callback interface corresponding to each server type can convert a log to be uploaded from a target format to a log format corresponding to the server type, so that before the log to be uploaded is uploaded by the electronic device, the electronic device can call the corresponding callback interface according to the server type to be uploaded by the log to be uploaded and the corresponding relation between the preset server type and the callback interface, and convert the format of the log to be uploaded.

In the scheme provided by the embodiment of the invention, before uploading the log to be uploaded, the electronic equipment can convert the target format of the log to be uploaded into the log format corresponding to the server type to be uploaded of the log to be uploaded according to the pre-configured callback interface, so that the server can directly process the log to be uploaded.

Referring to fig. 11, a description will be given of an example of a log uploading method provided by an embodiment of the present invention, where, as shown in fig. 11, a preset embedded point in an electronic device may send a log to be uploaded to a memory in the electronic device, and a memory space for storing the log to be uploaded is determined by the memory, and storage space configuration information may be configured in the memory space, for example, a correspondence between a server type and a storage space, etc. The electronic device may send the server configuration information to the electronic device according to the server type to be uploaded by the log to be uploaded, for example, the service server or the monitoring server, where each server has the server configuration information, and in the process that the electronic device arranges the preset embedded point, the server may send the configuration information of the server to the electronic device, for example, the identifier of the server, the address of the server, the uploading time interval of the log uploaded to the server, the storage space of the log, the callback interface corresponding to the server, and so on are sent to the electronic device.

Corresponding to the log uploading method, the embodiment of the invention also provides a log uploading device. The following describes a log uploading device provided by the embodiment of the invention.

As shown in fig. 12, a log uploading device, the device includes:

the log obtaining module 1210 is configured to obtain a log to be uploaded, where the log to be uploaded carries a server type to be uploaded;

a storage space determining module 1220, configured to determine a target storage space corresponding to the server type according to the server type and a preset correspondence between the server type and the storage space;

a log storage module 1230, configured to store the log to be uploaded to the target storage space;

and a log uploading module 1240, configured to read the log to be uploaded from the target storage space when the log uploading condition is satisfied, and upload the log to be uploaded to a target server according to a server type corresponding to the log to be uploaded.

As an implementation manner of the embodiment of the present invention, the storage space determining module 1220 may be specifically configured to:

As an implementation manner of the embodiment of the present invention, the apparatus may further include:

As an implementation manner of the embodiment of the present invention, the log obtaining module 1210 may include:

As an implementation manner of the embodiment of the present invention, the foregoing apparatus may further include:

As an implementation manner of the embodiment of the invention, the storage space may at least include a hard disk, a memory and a database.

The embodiment of the present invention further provides an electronic device, as shown in fig. 13, including a processor 1301, a communication interface 1302, a memory 1303 and a communication bus 1304, where the processor 1301, the communication interface 1302, and the memory 1303 complete communication with each other through the communication bus 1304,

A memory 1303 for storing a computer program;

processor 1301, when executing the program stored in memory 1303, implements the following steps:

The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the terminal and other devices.

The memory may include random access memory (Random Access Memory, RAM) or may include non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The Processor may be a general-purpose Processor, including a central processing unit (Central Processing Unit, CPU), a network Processor (Network Processor, NP), a digital signal Processor (DIGITAL SIGNAL Processor, DSP), an Application-specific integrated Circuit (ASIC), a Field-Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, or discrete hardware components.

In yet another embodiment of the present invention, a computer readable storage medium is provided, where a computer program is stored, where the computer program, when executed by a processor, implements the log uploading method according to any of the foregoing embodiments.

In yet another embodiment of the present invention, a computer program product containing instructions that, when run on a computer, cause the computer to perform the log uploading method of any of the above embodiments is also provided.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus, electronic device, computer readable storage medium, and computer program product embodiments, the description is relatively simple, as relevant to the method embodiments being referred to in the section of the description of the method embodiments.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A log uploading method, applied to an electronic device, comprising:

Acquiring original logs generated by preset buried points, wherein the preset buried points are arranged in the electronic equipment in advance by a server, and the format of the log processed by the server corresponds to the format of the original log generated by the preset buried points arranged in advance by the server for each server;

according to a preset data processing mode, converting the original log into a log to be uploaded in a target format so as to realize unification of the log formats, wherein the log to be uploaded carries the type of a server to be uploaded;

storing the log to be uploaded to the target storage space;

When the log uploading condition is met, the log to be uploaded is read from the target storage space, a target callback interface corresponding to the server type is determined according to the corresponding relation between the server type and a preset server type and the callback interface, the callback interface is called to convert the log to be uploaded from a target format to a log format corresponding to the server type, and the log to be uploaded is uploaded to a target server according to the server type corresponding to the log to be uploaded.

2. The method according to claim 1, wherein the step of determining the target storage space corresponding to the server type according to the server type and the preset correspondence between the server type and the storage space includes:

3. The method according to claim 2, wherein the method further comprises:

4. The method of claim 1, wherein prior to the step of reading the log to be uploaded from the target storage space and according to the server type to which the log to be uploaded corresponds, the method further comprises:

If so, it is determined that the log upload condition is satisfied.

5. The method of claim 1, wherein after the step of uploading the log to be uploaded to a target server, the method further comprises:

judging whether the log to be uploaded is successfully uploaded;

6. The method of any of claims 1-5, wherein the storage space comprises at least a hard disk, a memory, and a database.

7. A log uploading apparatus, applied to an electronic device, the apparatus comprising:

The system comprises a log acquisition module, a data processing module and a data processing module, wherein the log acquisition module is used for acquiring original logs generated by each preset embedded point, converting the original logs into logs to be uploaded in a target format according to a preset data processing mode so as to realize unification of log formats, wherein the preset embedded points are arranged in the electronic equipment in advance by a server, and for each server, the format of the logs processed by the server corresponds to the format of the original logs generated by the preset embedded points arranged in advance by the server, and the logs to be uploaded carry the type of the server to be uploaded;

the log uploading module is used for reading the log to be uploaded from the target storage space when the log uploading condition is met, and uploading the log to be uploaded to a target server according to the server type corresponding to the log to be uploaded;

the apparatus further comprises:

The callback interface determining module is used for determining a target callback interface corresponding to the server type according to the server type and the corresponding relation between the preset server type and the callback interface before the log uploading module uploads the log to be uploaded to the target server;

8. The apparatus of claim 7, wherein the storage space determination module is specifically configured to:

9. The apparatus of claim 8, wherein the apparatus further comprises:

10. The apparatus of claim 7, wherein the apparatus further comprises:

11. The apparatus of claim 7, wherein the apparatus further comprises:

12. The apparatus of any of claims 7-11, wherein the storage space comprises at least a hard disk, a memory, and a database.

13. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for carrying out the method steps of any one of claims 1-6 when executing a program stored on a memory.

14. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-6.