CN116483578A - Link time consumption analysis method and device, storage medium and electronic equipment - Google Patents

Abstract

The invention provides a method and a device for analyzing time consumption of a link, a storage medium and electronic equipment, wherein the method comprises the following steps: when a data access request sent by a target client is received, adding the data access request into a first queue, and determining the enqueue time when the data access request is added into the first queue; transmitting the data access request to target storage equipment included in the storage equipment cluster through the first queue so that the target storage equipment executes the data access request to obtain a target request result of the data access request; receiving a target request result returned by the target storage device, adding the target request result into a second queue, and determining the dequeue time when the target request result comes out of the second queue to be sent to the target client; and determining the middle layer time-consuming result of the data access request by adopting the enqueue time and the dequeue time. The embodiment of the invention can construct a storage system supporting the link time consumption analysis so as to realize the requested link time consumption analysis.

Description

Link time-consuming analysis method, device, storage medium and electronic equipment

technical field

The invention relates to the field of computer technology, in particular to a link time-consuming analysis method, device, storage medium and electronic equipment.

Background technique

At present, a storage system composed of various storage devices in a storage device cluster has been widely used. Among them, the storage device can be a memory object cache system (Memcache) and Gridstore (a storage device that can increase storage blocks according to needs), etc., and Memcache is a set of high-performance memory object cache system, which is generally used in dynamic web (World Wide Web, World Wide Web) applications to reduce the load on the database. By caching data and objects in memory, the number of times to read the database can be reduced, thereby improving the ability to dynamically access the database. Taking the storage device as the memory object cache system as an example, the memory object cache system does not have the slow query logging function. The existing storage system can only obtain the cache (cache) hit rate of the entire storage device cluster, the QPS (Query Per Second, query rate per second) and the average time consumption of various commands through memcache-top (an auxiliary tool for memcache), but cannot provide detailed time-consuming information of each request. It can be seen that in the existing technology, the storage system has insufficient ability to analyze the time consumption of links. Based on this, how to build a storage system that supports link time-consuming analysis to realize link time-consuming analysis of requests has become a research hotspot.

Contents of the invention

In view of this, embodiments of the present invention provide a link time-consuming analysis method, device, storage medium, and electronic equipment to solve the problem that existing storage systems do not support link time-consuming analysis of requests, that is, embodiments of the present invention can build a storage system that supports link time-consuming analysis, so as to implement link time-consuming analysis of requests.

According to one aspect of the present invention, a method for analyzing link time consumption is provided, the method is applied to an intermediate proxy device in a storage system, and the storage system includes at least one client, the intermediate proxy device, and a storage device cluster managed by the intermediate proxy device; the method includes:

When receiving the data access request sent by the target client, adding the data access request to the first queue, and determining the enqueue time when the data access request is added to the first queue, the first queue is used to store requests received from the client;

Send the data access request to a target storage device included in the storage device cluster through the first queue, so that the target storage device executes the data access request to obtain a target request result of the data access request, and the target storage device is any storage device in the storage device cluster;

receiving the target request result returned by the target storage device, adding the target request result to a second queue, and determining the dequeue time for the target request result to come out of the second queue to be sent to the target client, and the second queue is used to store the request result to be sent to the client;

Using the enqueue time and the dequeue time, determine the time-consuming result of the middle layer of the data access request.

According to another aspect of the present invention, another link time-consuming analysis method is provided. The method is applied to a target instance proxy device in a storage system. The storage system includes at least one client, an intermediate proxy device, a storage device cluster managed by the intermediate proxy device, and an instance proxy device corresponding to each storage device in the storage device cluster. The target instance proxy device supports monitoring and capturing information of the intermediate proxy device for detecting messages received and sent by the target storage device. The target storage device is a storage device corresponding to the target instance proxy device in the storage device cluster; the method includes:

When it is detected that the target storage device receives the first message data in the data access request, record the receiving time of the first message data, the data access request includes at least one message data, and the receiving time of the first message data is earlier than the receiving time of any message data in the at least one message data except the first message data;

When it is detected that the response message of each message data in the at least one message data is sent, record the sending time of the response message of each message data, and the response message of one message data is the execution result of the corresponding message data by the storage device;

Using the receiving time of the first message data and the sending time of the response message of the second message data to determine the time-consuming result of the instance layer of the data access request, the sending time of the response message of the second message data is later than the sending time of the response messages of other message data in the at least one message data.

According to another aspect of the present invention, a link time-consuming analysis device is provided, the device runs on an intermediate proxy device in a storage system, and the storage system includes at least one client, the intermediate proxy device, and a storage device cluster managed by the intermediate proxy device; the device includes:

The processing unit is configured to add the data access request to the first queue when receiving the data access request sent by the target client, and determine the enqueue time when the data access request is added to the first queue, and the first queue is used to store requests received from the client;

A sending unit, configured to send the data access request to a target storage device included in the storage device cluster through the first queue, so that the target storage device executes the data access request to obtain a target request result of the data access request, and the target storage device is any storage device in the storage device cluster;

a receiving unit, configured to receive the target request result returned by the target storage device;

The processing unit is further configured to add the target request result to a second queue, and determine the dequeue time for the target request result to come out of the second queue to be sent to the target client, and the second queue is used to store the request result to be sent to the client;

The processing unit is further configured to use the enqueue time and the dequeue time to determine the time-consuming result of the middle layer of the data access request.

According to another aspect of the present invention, another link time-consuming analysis device is provided. The device runs on a target instance proxy device in a storage system. The storage system includes at least one client, an intermediate proxy device, a storage device cluster managed by the intermediate proxy device, and an instance proxy device corresponding to each storage device in the storage device cluster. The target instance proxy device supports monitoring and capturing information of the intermediate proxy device for detecting messages received and sent by the target storage device. The target storage device is a storage device corresponding to the target instance proxy device in the storage device cluster; the device includes:

A processing unit, configured to record the receiving time of the first message data when detecting that the target storage device receives the first message data in the data access request, the data access request includes at least one message data, and the receiving time of the first message data is earlier than the receiving time of any message data in the at least one message data except the first message data;

The processing unit is further configured to record the sending time of the response message of each message data when detecting that the response message of each message data in the at least one message data is sent, and the response message of one message data is the execution result of the corresponding message data by the storage device;

The processing unit is further configured to use the receiving time of the first message data and the sending time of the response message of the second message data to determine the time-consuming result of the instance layer of the data access request, and the sending time of the response message of the second message data is later than the sending time of the response messages of other message data in the at least one message data.

According to another aspect of the present invention, an electronic device is provided, and the electronic device includes a processor and a memory storing a program, wherein the program includes instructions, and the instructions cause the processor to perform the above-mentioned method when executed by the processor.

According to another aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the above-mentioned method.

In the embodiment of the present invention, when the intermediate proxy device receives the data access request sent by the target client, it can add the data access request to the first queue, and determine the enqueue time when the data access request is added to the first queue; then, the intermediate proxy device can send the data access request to the target storage device through the first queue, so that the target storage device executes the data access request, and obtains the target request result of the data access request; The dequeue time sent to the target client is used to determine the time-consuming result of the middle layer of the data access request by using the queue-entry time and the dequeue time, so that the storage system based on the intermediate proxy device can support link time-consuming analysis. Correspondingly, the constructed storage system may further include an instance agent device corresponding to each storage device in the storage device cluster. For the target instance agent device, the target instance agent device may record the receiving time of the first message data when detecting that the target storage device receives the first message data in the data access request. When the file is sent, record the sending time of the response message of each message data, so as to use the receiving time of the first message data and the sending time of the response message of the second message data to determine the time-consuming result of the instance layer of the data access request, so as to obtain a more accurate time-consuming result. It can be seen that the embodiment of the present invention can build a storage system that supports link time-consuming analysis, so as to realize link time-consuming analysis of requests from at least one aspect, especially when the storage device is a memory object caching system, it can implement link time-consuming analysis of requests for accessing the memory object caching system, so as to solve the problem that the memory object caching system does not support link time-consuming analysis of corresponding requests, thereby solving problems related to the storage system including the memory object caching system.

Description of drawings

Further details, features and advantages of the invention are disclosed in the following description of exemplary embodiments with reference to the accompanying drawings in which:

Figure 1a shows a schematic structural diagram of a storage system according to an exemplary embodiment of the present invention;

Figure 1b shows a schematic structural diagram of another storage system according to an exemplary embodiment of the present invention;

Fig. 2 shows a schematic flowchart of a link time consumption analysis method according to an exemplary embodiment of the present invention;

Fig. 3a shows a schematic structural diagram of another storage system according to an exemplary embodiment of the present invention;

Fig. 3b shows a schematic structural diagram of another storage system according to an exemplary embodiment of the present invention;

FIG. 4 shows a schematic flowchart of another link time consumption analysis method according to an exemplary embodiment of the present invention;

Fig. 5 shows a schematic diagram of communication according to an exemplary embodiment of the present invention;

FIG. 6 shows a schematic flowchart of another method for link time consumption analysis according to an exemplary embodiment of the present invention;

Fig. 7a shows a schematic block diagram of a link time consumption analysis device according to an exemplary embodiment of the present invention;

Fig. 7b shows a schematic block diagram of another link time consumption analysis device according to an exemplary embodiment of the present invention;

FIG. 8 shows a structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present invention.

Detailed ways

Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the invention. It should be understood that the drawings and embodiments of the present invention are for exemplary purposes only, and are not intended to limit the protection scope of the present invention.

It should be understood that the various steps described in the method implementation manners of the present invention may be executed in different orders, and/or executed in parallel. Additionally, method embodiments may include additional steps and/or omit performing illustrated steps. The scope of the invention is not limited in this regard.

As used herein, the term "comprise" and its variations are open-ended, ie "including but not limited to". The term "based on" is "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one further embodiment"; the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms will be given in the description below. It should be noted that concepts such as "first" and "second" mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the sequence or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

The names of messages or information exchanged between multiple devices in the embodiments of the present invention are used for illustrative purposes only, and are not used to limit the scope of these messages or information.

In an embodiment of the present invention, as shown in FIG. 1a, it relates to a storage system that supports link time-consuming analysis. The storage system includes at least one client (i.e., a terminal) 101, an intermediate proxy device (Router-server) 102, and at least one storage device 103. At least one storage device 103 constitutes a storage device cluster in the storage system, and the intermediate proxy device 102 can also be called an intermediate proxy layer; At least one storage device 103 is strung together to form a distributed cluster, thereby breaking through the memory bottleneck of a single storage device, and avoiding the cost of maintaining the storage device list on the user side (ie, the client 101 ). Wherein, the storage device 103 may be a memory object cache system (ie, a memory object cache system instance) or a NAS (Network Attached Storage, network attached storage), etc., which is not limited in the present invention; when the storage device 103 is a memory object cache system, the embodiment of the present invention may also refer to the storage system as a memory object cache system cluster. Correspondingly, the intermediate proxy device 102 may be a terminal (that is, a client) or a server, which is not limited in the present invention.

Specifically, the intermediate proxy device 102 can receive a request (such as a data access request) sent by any client 101, and send the received request to the corresponding storage device 103; based on this, the storage device 103 that receives the request can execute the corresponding request, obtain the request result, and send the request result to the intermediate proxy device 102; further, the intermediate proxy device 102 can send the request result to the corresponding client 101. It can be seen that the intermediate proxy device 102 belongs to the middleware proxy of the storage device 103, so that each request of any client 101 will be forwarded by the intermediate proxy device 102, and the request result of the storage device 103 will be returned to the corresponding client 101.

It should be noted that, as shown in FIG. 1b, the storage system may also include instance proxy devices 104 corresponding to each storage device 103, and one instance proxy device 104 may support monitoring and capturing information of the intermediate proxy device 102, so as to detect messages received and sent by the corresponding storage device 103. Wherein, the example proxy device 104 may be a terminal or a server, which is not limited in the present invention.

It should be noted that the terminals mentioned here may include, but are not limited to: smart phones, tablet computers, notebook computers, desktop computers, smart watches, smart voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, and so on. The server mentioned here can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network), and big data and artificial intelligence platforms.

Based on the above description, an embodiment of the present invention proposes a link time-consuming analysis method. The link time-consuming analysis method may be executed by an intermediate proxy device in a storage system. The storage system includes at least one client, an intermediate proxy device, and a storage device cluster managed by the intermediate proxy device; as shown in FIG. 2, the link time-consuming analysis method may include the following steps S201-S204:

S201. When receiving the data access request sent by the target client, add the data access request to the first queue, and determine the enqueue time when the data access request is added to the first queue, and the first queue is used to store requests received from the client.

Wherein, the target client is any client in the above at least one client; it should be understood that the intermediate proxy device can receive the request sent by each client in the at least one client, and the request mentioned here can be a read request or a write request, which is not limited in the embodiment of the present invention.

Correspondingly, the above-mentioned target client may send at least one request to the intermediate proxy device, then the above-mentioned data access request is any request in the at least one request.

S202. Send the data access request to the target storage device included in the storage device cluster through the first queue, so that the target storage device executes the data access request, and obtains a target request result of the data access request.

Wherein, the target storage device is any storage device in the storage device cluster. It should be understood that the storage device cluster may include at least one shard, and each shard is composed of a storage device, as shown in FIG. 3a; based on this, the intermediate proxy device may send a data access request to the shard formed by the target storage device. In a specific implementation, the storage device may be an in-memory object cache system, in this case, each slice may consist of one in-memory object cache system instance.

In the embodiment of the present invention, the intermediate proxy device can map and forward the data access request to the corresponding storage device according to the hash value of the key (keyword) in the data access request; In this case, the intermediate proxy device may use the storage device indicated by the hash value of the keyword as the target storage device. It can be seen that the intermediate proxy device can hash (hash) map the received request to different storage devices according to the key for processing.

In other embodiments, the data access request may also carry a storage device identifier, then the intermediate proxy device may determine the storage device identifier from the data access request, and use the storage device indicated by the determined storage device identifier as the target storage device, and then send the data access request to the target storage device. Wherein, the storage device identifier may be a digital identifier or a character identifier, which is not limited in the present invention.

In a specific implementation, the intermediate proxy device can send the data access request to the third queue through the first queue, and the third queue is used to store requests to be sent to the storage device; further, the intermediate proxy device can use the third queue to send the data access request to the target storage device included in the storage device cluster. Among them, the first queue and the third queue can interact through QDT (Queue-Dealer-Thread, internal queue processing thread), and the QDT can be used to handle the work of data transfer between different queues.

S203. Receive the target request result returned by the target storage device, add the target request result to the second queue, and determine the dequeue time for the target request result to come out of the second queue to be sent to the target client, and the second queue is used to store the request result to be sent to the client.

It should be noted that, in the process of interaction between the intermediate proxy device and the client, the intermediate proxy device can interact with the client through CDT (Client-Dealer-Thread, client-side interactive thread), that is, the CDT can be used to process the operation of the interaction part between the intermediate proxy device and the user request; specifically, the intermediate proxy device can receive the request sent by the client through the CDT (such as receiving the data access request sent by the target client), and the intermediate proxy device can send the request result to the client through the CDT (such as sending the target request result to the target client).

Correspondingly, in the process of interaction between the intermediate proxy device and the storage device, the intermediate proxy device can interact with the storage device through SDT (Server-Dealer-Thread, server-side interactive thread), that is, the SDT can be used to process the operation of the interaction between the intermediate proxy device and the storage device; specifically, the intermediate proxy device can send a request to the storage device through the SDT (such as sending a data access request to the target storage device), and the intermediate proxy device can receive the request result sent by the storage device through the SDT (such as receiving the target request result sent by the target storage device).

It should be noted that the storage device may include at least one intermediate proxy device. In this case, each intermediate proxy device can interact with each client in the storage system through the CDT, and each intermediate proxy device can interact with each storage device in the storage system through the SDT. For ease of illustration, the following descriptions will be made by taking the storage system including an intermediate proxy device as an example.

In a specific implementation, when adding the target request result to the second queue, the intermediate proxy device may add the target request result to the fourth queue, and the fourth queue is used to store the request result received from the storage device; further, the target request result may be sent to the second queue through the fourth queue, so that the target request result is added to the second queue.

Based on this, the request sent by the user can be received by CDT, and enter the first queue in the intermediate proxy device for queuing. The first queue can also be called queue-1 (queue-1); then, the intermediate proxy device can take out the request of the head of the first queue through QDT, and then insert it into the tail of the third queue. , then the intermediate proxy device can receive the request result from the storage device (i.e. the return result processed by the storage device) through SDT, and insert it into the tail of the fourth queue, which can also be called queue-3; based on this, the intermediate proxy device can take out the request result from the head of the fourth queue and process it into the tail of the second queue, so that the request result from the head of the second queue can be returned to the client through CDT, and the second queue can also be called queue-4. It should be understood that the intermediate proxy device respectively stores data interacting with a device (such as a client or a storage device) through the above four queues. For example, the first queue is used to store requests involved in the interaction between the intermediate proxy device and the client, and the third queue is used to store requests involved in the interaction between the intermediate proxy device and the storage device, which can avoid the coupling relationship between the sender and the receiver of the request, so that the interaction process between the intermediate proxy device and the client and the interaction process between the intermediate proxy device and the storage device do not affect each other, thereby saving transmission time.

Based on this, after receiving the data access request sent by the target client, the intermediate proxy device can add the data access request to the first queue, and send the data access request to the third queue through the first queue, and use the third queue to send the data access request to the target storage device included in the storage device cluster; further, the intermediate proxy device can add the target request result returned by the target storage device to the fourth queue, and send the target request result to the second queue through the fourth queue, so as to send the target request result to the target client through the second queue.

S204. Determine the time-consuming result of the middle layer of the data access request by using the queue entry time and the queue exit time.

In a specific implementation, the intermediate agent device can perform a difference operation on the queue-entry time and the queue-exit time to obtain the difference operation result, and use the difference operation result as the intermediate layer time-consuming result of the data access request, that is, the intermediate agent device can use the difference between the queue-out time and the queue-entry time as the intermediate layer time-consuming result; In another specific implementation, the intermediate proxy device may use the above enqueue time and dequeue time as the time-consuming result of the intermediate layer for the data access request. In this case, the intermediate layer time-consuming result may include the enqueue time and the dequeue time, so that the time-consuming data access request is reflected by the enqueue time and the dequeue time, and so on.

In the embodiment of the present invention, the intermediate proxy device may include M queue groups, where M is a positive integer, and a queue group corresponds to a processing pool (i.e., a pool), which may also be referred to as a pool, and the data involved in a request is transmitted through each queue in the same queue group (that is, the data involved in a request is only processed in the same processing pool), then the first queue and the second queue are both queues in the target queue group, and the target queue group is any queue group in the M queue groups. It should be understood that when both the first queue and the second queue are queues in the target queue group, the third queue that performs data transmission with the first queue and the fourth queue that performs data transmission with the second queue are also queues in the target queue group.

Based on this, the intermediate proxy device can receive multiple requests and distribute the multiple requests to the processing pools corresponding to the N queue groups in the M queue groups, where N is a positive integer less than or equal to M; correspondingly, the requests in each queue group can be processed in parallel through each of the N queue groups, so that the request results of multiple requests and the time-consuming results of the intermediate layer can be obtained in parallel through the corresponding storage devices. It can be seen that the embodiment of the present invention can split multiple requests into different queue groups as much as possible, thereby making full use of the advantages of multi-thread processing, reducing the queuing time, and improving request efficiency.

Exemplarily, as shown in Figure 3b, the interaction between the intermediate proxy device and the client or storage device will be split into different pools (such as pool 1, pool 2...pool M), and each processing pool can be split into multiple queues (such as the above-mentioned 4 queues); wherein, the splitting logic of the processing pool can be related to the configuration parameters deployed by the intermediate proxy device, and the configuration parameters can be CPU (Central Processing Unit, central processing unit) core or memory size, etc. For example, if the configuration parameter is 4 cores/8GB, 1 processing pool can be allocated; if the configuration parameter is 8 cores/16GB, 2 processing pools can be allocated, and so on.

In the embodiment of the present invention, when the intermediate proxy device receives the data access request sent by the target client, it can add the data access request to the first queue, and determine the enqueue time when the data access request is added to the first queue. The request result is added to the second queue, and the dequeue time for the target request result to be sent from the second queue to be sent to the target client is determined. The second queue is used to store the request result to be sent to the client, thereby using the enqueue time and dequeue time to determine the time-consuming result of the middle layer of the data access request, so that the storage system built based on the intermediate proxy device can support link time-consuming analysis. It can be seen that in the embodiment of the present invention, a storage system that supports link time-consuming analysis can be constructed to provide the requested link time-consuming analysis function through an intermediate proxy device, so that the requested link time-consuming analysis can be realized; and, especially when the storage device included in the storage system is a memory object caching system, link time-consuming analysis of requests to access any memory object caching system in the storage system can be implemented, so as to solve the problem that the memory object caching system does not support the link time-consuming analysis of the corresponding request, thereby solving the problems involved in the storage system that includes the memory object caching system. Insufficiency and other issues.

Based on the above description, the embodiment of the present invention also proposes a more specific link time-consuming analysis method. The link time-consuming analysis method may be executed by the intermediate agent device in the storage system mentioned above. The storage system includes at least one client, the intermediate agent device and a storage device cluster managed by the intermediate agent device; please refer to FIG. 4, the link time-consuming analysis method may include the following steps S401-S404:

S401. When receiving the data access request sent by the target client, add the data access request to the first queue, and determine the enqueue time when the data access request is added to the first queue, and the first queue is used to store requests received from the client.

In the embodiment of the present invention, the target storage device may include at least one probe group, any probe group includes a start probe and at least one end probe, and during a link execution process, only one start probe and one end probe are executed in any probe group; wherein, a probe is provided with a corresponding probe identifier, and the probe identifier may be a text identifier or a digital identifier (that is, a specific parameter), which is not limited in the present invention.

S402. Send the data access request to the target storage device included in the storage device cluster through the first queue, so that the target storage device executes the data access request according to the corresponding link, and when executing the target probe group in at least one probe group, print the probe ID of the target start probe executed in the target probe group and the probe ID of the target end probe executed in the target probe group.

Wherein, the printing time of the probe ID in the target starting probe and the printing time of the probe ID in the target ending probe are used to determine the local time-consuming result when the data access request is executed according to the link monitored by the target probe group.

It should be understood that when the target queue group where the first queue is located includes the third queue, the intermediate proxy device can send the data access request to the third queue through the first queue; further, through the third queue, send the data access request to the target storage device included in the storage device cluster, so that the target storage device executes the data access request according to the corresponding link, and when executing the target probe group in at least one probe group, identify the probes in the target start probes executed in the target probe group and the probes in the target termination probes executed in the target probe group to print.

It should be noted that the processing process of each user command (that is, request) inside the storage device is actually divided into multiple processes, which include but are not limited to: sending commands, parsing commands, executing commands, and returning results, etc.; therefore, when slow query requests (ie slow queries, slowlog) appear, usually the cause cannot be effectively located, and the response time of the storage device can only be sensed. Among them, when the processing speed of the storage system is slow to a certain extent, it will become a slow query, slowing down the processing speed of the entire storage system.

Based on this, it is necessary to obtain the specific time-consuming in each stage; in the embodiment of the present invention, the storage device has undergone secondary development and transformation, that is, the storage device in the storage system can be a customized developed storage device. Correspondingly, eBPF (Extended Berkeley Packet Filter, an extension of the mechanism for safely injecting code when kernel events and user program events occur) technology can be used to set USDT (Userland Statically Defined Tracing) probes (a probe based on eBPF technology) on core key links. The source code is embedded in the feature code and compiled into the program, that is, at least one link in the customized storage device can be monitored by the corresponding probe set. Specifically, DTRACE_PROBE and DTRACE_PROBE1 macros can be added to the place where the probe needs to be implanted in the storage device to mark the code area that needs to trigger time-consuming result statistics. The code area can also be called the execution program, link or function stage. Optionally, the kernel version of the deployed storage device may not be lower than a preset version, such as 4.18, and the specific value of the preset version is not limited in the present invention. In other embodiments, BPF technology may also be used to perform probe settings on the storage device, etc.; the present invention is not limited thereto.

Exemplarily, conn_new_command (accepting new commands), conn_waiting (waiting for network data), conn_read (data reading), conn_parse_cmd (command parsing), conn_write (writing data to message buffer), conn_mwrite (writing data to router-server), conn_closing (closing connection) and other key links in the storage device can add USDT probes, so that data access requests can be executed according to the corresponding link , the probe IDs in the probes involved in the link can be printed, etc.

In a specific implementation, when the storage device prints the probe IDs in the target start probes executed in the target probe group and the probe IDs in the target end probes executed in the target probe group, a specific kernel script can be loaded outside the storage device, so that when the target probe group is executed inside the storage device, the probe IDs in the target start probes and the probe IDs in the target end probes are printed out from the kernel script, thereby obtaining the processing time of the link monitored by the target probe group (that is, the local time-consuming result).

In the embodiment of the present invention, the local time-consuming results when the data access request is executed according to the link monitored by the target probe group may include: the printing time of the probe ID in the target start probe and the printing time of the probe ID in the target end probe, and may also include: the printing time of the probe ID in the target end probe, and the printing time difference between the probe ID in the target start probe, etc.; the present invention is not limited to this. Wherein, the target termination probe is any termination probe in the target probe group.

S403. Receive the target request result returned by the target storage device, add the target request result to the second queue, and determine the dequeue time for the target request result to come out of the second queue to be sent to the target client, and the second queue is used to store the request result to be sent to the client.

S404. Determine the time-consuming result of the middle layer of the data access request by using the queue entry time and the queue exit time.

In the embodiment of the present invention, the intermediate proxy device may store the time-consuming result of the intermediate layer of the data access request. In one embodiment, the intermediate agent device may store the time-consuming result of the intermediate layer in the first time-consuming data, so as to add a piece of stored data including the time-consuming result of the intermediate layer to the first time-consuming data, and the first time-consuming data includes the time-consuming result of the intermediate layer of each request received by the intermediate agent device.

In another embodiment, if the time-consuming result of the middle layer is longer than the first preset duration, the time-consuming result of the middle layer can be stored in the second time-consuming data, so as to add a piece of stored data including the time-consuming result of the middle layer to the second time-consuming data, and the second time-consuming data includes the time-consuming result of the middle layer of each first slow query request in at least one first slow query request. sm etc., the present invention is not limited to this.

In yet another embodiment, if the corresponding duration of the time-consuming result of the middle layer is longer than the second preset duration, the data access request is stored in the slow query data to add a piece of stored data including the data access request to the slow query data, and the slow query data includes at least one second slow query request, and the second slow query request refers to a request whose corresponding duration of the time-consuming result of the middle layer is longer than the second preset duration, etc.; wherein, the second preset duration can be 500ms or 600ms, etc. The duration is different, which is not limited in the present invention.

Among them, if the time-consuming result of the middle layer is represented by two time points, the corresponding duration of the time-consuming result of the middle layer is the difference between the two time points in the time-consuming result of the middle layer; if the time-consuming result of the middle layer is the difference between the two time points, the corresponding duration of the time-consuming result of the middle layer is the time-consuming result of the middle layer itself.

It should be noted that each queue in a processing pool can have a corresponding time-consuming buried point. In this case, for each queue in a processing pool, the intermediate proxy device can record the data involved in the data access request (that is, the data access request or the target request result) in each queue. Enqueue timestamp (ie, queue entry time) and queue exit timestamp (ie, queue exit time), so as to obtain the intermediate layer time-consuming result of the data access request (such as the result of the dequeue time of the target request result for the second queue minus the data access request for the first queue. The result of the queue time) , the time-consuming result of the middle layer can also be called client-cost (client-cost).

Correspondingly, for any stored data in the first time-consuming data and the second time-consuming data, the content included in the stored data may be: [time stamp, client-ip (Internet Protocol, protocol for interconnection between networks), storage device-ip, request (such as data access request), size of sent bytes, size of returned bytes, time-consuming of the first queue, time-consuming of the second queue, time-consuming of the third queue, time-consuming of the fourth queue, time-consuming result of the middle layer]; or, the content of any stored data may be: [client -ip, storage device-ip, request, intermediate layer time-consuming result, intermediate proxy device-ip], etc.; the present invention does not limit the specific content included in the stored data. At the same time, for any stored data in the slow query data, the content included in the stored data may be: [client-ip, storage device-ip, second slow query request]; or, the content included in any stored data may be: [client-ip, storage device-ip, second slow query request, intermediate layer time-consuming result], etc.; the present invention is not limited to this. Wherein, client-ip is the ip of the client, and storage device-ip is the ip of the storage device processing the corresponding request.

It should be understood that if the intermediate proxy device stores data according to the second time-consuming data and slow query data, if and only if the data access request is a slow query request (such as the first slow query request or the second slow query request), the intermediate proxy device can generate the data to be stored for the data access request, and store the data to be stored.

In one embodiment, the intermediate proxy device can store the target storage data in the log of the intermediate proxy device, and the target storage data is any data in the first time-consuming data, the second time-consuming data, and the slow query data; in this case, when the intermediate proxy device needs to store the data to be stored in the data access request, it can serialize the content that the data to be stored needs to include, obtain the data to be stored, and send the data to be stored to the log thread (Log-Collector-Thread, LCT) of the intermediate proxy device to store the data to be stored to the intermediate proxy device In the log, the target storage data includes the data to be stored. Exemplary, if the stored data includes the ip (i.e. the intermediate proxy device identification) of the slow query request and the intermediate proxy device, and the data access request is a slow query request, then the intermediate proxy device can serialize the data access request and the ip of the intermediate proxy device; optionally, the intermediate proxy device can be sent to the LCT by UDP (UserDatagram Protocol, User Datagram Protocol) communication mode, as shown in Figure 5.

It should be noted that when the intermediate proxy device sends the data to be stored to the log thread of the intermediate proxy device through the client interaction thread, once the CDT sends the request (that is, the data to be stored), it can return immediately. That is to say, it can no longer pay attention to whether the LCT receives or returns the result after processing, which can effectively prevent the CDT from blocking due to waiting for the result to return, thereby avoiding blocking the user request (that is, the request sent by the client).

In another embodiment, the intermediate proxy device can store the target storage data in a data list (list) in the local space. In this case, when the intermediate proxy device needs to store the data to be stored in the data access request, it can add the data to be stored to the data list, so that the target storage data includes the data to be stored. Correspondingly, the intermediate proxy device can include an operation and maintenance interface, so the operation and maintenance personnel can view the target storage data through the operation and maintenance interface; optionally, the operation and maintenance personnel can view the target storage data when they have the viewing authority, and the intermediate proxy device can use the user name, password, and ip to identify the viewing authority.

Further, the intermediate proxy device can determine the expiration time, and delete any storage data when it detects that the storage time of any storage data in the target storage data is longer than the expiration time, and the target storage data is any one of the first time-consuming data, the second time-consuming data, and the slow query data; wherein, the storage time of any storage data refers to the difference between the storage time and the current system time. It should be noted that the expiration time may be 10 minutes, or 1 hour, etc.; the present invention is not limited thereto.

Exemplarily, assuming that the expiration time is 10 minutes, and the target storage data stores at least one slow query request storage data, such as the time-consuming result of the intermediate layer corresponds to a slow query request with a duration greater than 50 ms, then when the storage duration of any storage data is longer than 10 minutes, the intermediate proxy device can delete any storage data, so that the target storage data no longer includes the any storage data. In this case, when the operation and maintenance personnel need to temporarily go online to query, they can obtain the slow query requests in the last 10 minutes through the operation and maintenance interface, and use it to temporarily assist in troubleshooting the latest slow query problems online.

In the embodiment of the present invention, the intermediate agent device can perform online slow query warning detection on the data access request; further, when the data access request is a slow query request (such as the first slow query request or the second slow query request, etc.), the intermediate agent device can output slow query warning information for the data access request to prompt the operation and maintenance personnel. The slow query warning information can be audio information or text information, which is not limited in the present invention. It can be seen that the intermediate proxy device can have the function of automatically recording slow queries, and can be used for online slow query warning detection and collection.

It should be noted that, in other embodiments, the determination of the enqueue time of the data access request being added to the first queue in step S401 may not be performed, the dequeue time of determining the target request result from the second queue to be sent to the target client in step S403 may not be performed, and step S404 may not be performed. The device determines the local time-consuming result when the data access request is executed according to the link monitored by the target probe set based on the print time of the probe ID in the target start probe and the print time of the probe ID in the target end probe, and so on.

In the embodiment of the present invention, when the intermediate proxy device receives the data access request sent by the target client, it can add the data access request to the first queue, and determine the enqueue time when the data access request is added to the first queue; then, through the first queue, send the data access request to the target storage device included in the storage device cluster, so that the target storage device executes the data access request according to the corresponding link, and when the target probe group in at least one probe group is executed, the probe identifier in the target initial probe that is executed in the target probe group and the probe ID that is executed in the target probe group The probe ID in the target termination probe is printed; based on this, the time-consuming state of running the corresponding link inside the storage device can be obtained through the printing time of the probe ID in the target start probe and the printing time of the probe ID in the target termination probe, so that the storage device is no longer a black box for the operation and maintenance personnel, which facilitates the tracking and analysis of time-consuming problems. That is to say, the operation and maintenance personnel can capture the local time-consuming results of key links in the storage device on demand. Further, when receiving the target request result returned by the target storage device, the intermediate proxy device can add the target request result to the second queue, and determine the dequeue time for the target request result to be sent to the target client from the second queue, thereby using the enqueue time and dequeue time to determine the time-consuming result of the middle layer of the data access request; further, the data to be stored involved in the data access request can be stored based on the time-consuming result of the middle layer, so that the operation and maintenance personnel can query the stored data within a certain time range, so that the operation and maintenance personnel can query the storage device status is analyzed. Moreover, the intermediate proxy device can also monitor the slow query request to output slow query warning information, thereby improving the ability to perceive slow query problems.

Based on the above description, the embodiment of the present invention also proposes a more specific link time-consuming analysis method. The link time-consuming analysis method can be executed by the target instance proxy device in the storage system mentioned above. The storage system includes at least one client, an intermediate proxy device, a storage device cluster managed by the intermediate proxy device, and an instance proxy device corresponding to each storage device in the storage device cluster. The target instance proxy device supports monitoring and capturing information of the intermediate proxy device for detecting messages received and sent by the target storage device. The target storage device is a storage device in the storage device cluster corresponding to the target instance proxy device; Referring to FIG. 6, the link time consumption analysis method may include the following steps S601-S603:

S601. When it is detected that the target storage device receives the first message data in the data access request, record the receiving time of the first message data, the data access request includes at least one message data, and the receiving time of the first message data is earlier than the receiving time of any message data except the first message data in the at least one message data.

It should be understood that the intermediate proxy device can only count the time-consuming results of the intermediate layer. However, considering the network transmission overhead from the intermediate proxy device to the storage device level, it is necessary to further ignore the influence of network transmission to obtain the real time-consuming results of the underlying command processing of the storage device. However, the industry often uses tools such as tcpdump or wireshark to capture and analyze TCP (Transmission Control Protocol) messages, which is cumbersome and inconvenient. In the embodiment of the present invention, the response time of each request in the storage device (that is, the time-consuming result of the instance layer) can be captured through the instance agent device (perf-agent) corresponding to the storage device, so as to support statistics and analysis at the operation and maintenance level. It should be noted that the instance proxy device corresponding to a storage device may be located on the corresponding storage device (that is, the instance proxy device is also deployed on the storage device), or it may be located outside the corresponding storage device, which is not limited in the present invention.

In the embodiment of the present invention, an instance agent device can be an independent perf-agent (an agent realized based on libpcap technology) developed by relying on libpcap (a network packet capture function package under a unix/linux operating system), and each storage device corresponds to an instance agent device. The time-consuming result of the instance layer of the request). Among them, the instance proxy device can start the main thread based on libpcap, and separately monitor and capture the information of the intermediate proxy device to obtain the TCP messages received and sent by the storage device process.

In a specific implementation, the request received by the storage device can be split into at least one packet data, and since the requests received by a storage device are usually at the mass level, when the instance proxy device counts the processing time of a single request of the storage device, the instance proxy device can maintain a hash table to store the request, the receiving time of the first packet data, and the sending time of the response message of each packet data. Wherein, the packet data received by the storage device may also be called an incoming packet, and the outgoing packet (that is, a response packet to any packet data in at least one packet data) sent by the storage device may also be called an outgoing packet. In other embodiments, the instance proxy device may also store the request, the receiving time of the first message data, and the sending time of the response message of each message data in an array, etc.; the present invention is not limited thereto.

S602. When it is detected that the response message of each message data in at least one message data is sent, record the sending time of the response message of each message data. The response message of one message data is the execution result of the corresponding message data by the storage device.

It should be noted that the response packets of the above-mentioned packet data may constitute the target request result of the data access request. It should be understood that, for the same request (such as a data access request), when the target instance proxy device stores data by maintaining a hash table, each time the target storage device sends a response message, the target instance proxy device will update the sending time, and can delete the original packet output record (that is, the sending time of the response message) from the hash table.

Exemplarily, assuming that the sending time of the response message recorded in the hash table is time A, and within a time period after time A, the target instance proxy device detects that the target storage device sends a response message at time B, then the target instance proxy device can update time A in the hash table to time B.

S603. Using the receiving time of the first message data and the sending time of the response message of the second message data to determine the time-consuming result of the instance layer of the data access request, the sending time of the response message of the second message data is later than the sending time of the response messages of other message data in at least one message data.

Based on this, the target instance proxy device can start to calculate (IN, input) from the first message data that the target storage device receives from the intermediate proxy device, until the target storage device returns the response message (OUT, output) of the second message data (that is, the last TCP message) to the intermediate proxy device, and the time difference between the two (that is, the receiving time of the first message data and the sending time of the response message of the second message data) is the real storage device response time.

It should be noted that the target instance proxy device may use the receiving time of the first message data and the sending time of the response message of the second message data as the instance-level time-consuming result of the data access request. At this time, the instance-level time-consuming result includes the receiving time of the first message data and the sending time of the response message of the second message data.

In the embodiment of the present invention, the storage system may further include a display system, and the target instance proxy device may determine the storage device identifier of the target storage device, and serialize the instance layer time-consuming result and the storage device identifier to obtain the serialized time-consuming data; and send the serialized time-consuming data to the display system, so that the display system can display the time-consuming data within a specified time range. Exemplarily, the storage device identifier may be the ip of the storage device, and so on.

Specifically, after the display system receives the serialized time-consuming data, it can persistently store the received time-consuming data; correspondingly, when the display system detects a time-consuming data viewing request, it can display the time-consuming data within a specified time range, and the time-consuming data viewing request can carry a corresponding specified time range.

Further, the serialized time-consuming data may also include a data access request; in this case, the target instance proxy device may obtain at least one message data in the data access request from the target storage device to obtain the data access request, thereby performing serialization processing on the data access request, the time-consuming result of the instance layer, and the storage device identifier of the target storage device to obtain the serialized time-consuming data.

Optionally, the above-mentioned display system may include an open-source ElasticSearch (a distributed search and analysis engine) for storing the serialized time-consuming data remotely written by any instance agent device, as shown in Figure 3a; in this case, the target instance agent device can remotely write the above-mentioned serialized time-consuming data into the ElasticSearch storage medium. Correspondingly, the above display system may also include kibana (an open-source analysis and visualization platform). Based on this, the operation and maintenance personnel can retrieve the time-consuming fluctuation details of the corresponding storage device cluster according to the time dimension by viewing kibana, that is, they can retrieve the instance-level time-consuming results of each request processed by the corresponding storage device cluster according to the time dimension.

In the embodiment of the present invention, when the target instance proxy device detects that the target storage device has received the first message data in the data access request, it can record the receiving time of the first message data. The data access request includes at least one message data, and the receiving time of the first message data is earlier than the receiving time of any message data in the at least one message data except the first message data; and when it detects that the response message of each message data in the at least one message data is sent, it can record the sending time of the response message of each message data; then correspondingly, The receiving time of the first message data and the sending time of the response message of the second message data can be used to determine the time-consuming result of the instance layer of the data access request. The sending time of the response message of the second message data is later than the sending time of the response messages of other message data in at least one message data. It can be seen that the target instance proxy device can be used to perceive the time-consuming result of the instance layer at the storage device level. That is to say, the embodiment of the present invention can obtain the accurate time-consuming result of each request received by the target storage device in real time through the target instance proxy device, thereby avoiding the impact on the time-consuming result when the network delay between the intermediate proxy device and the storage device is high; and the target instance proxy device can send the serialized time-consuming data to the display system, so that the display system can persistently store the serialized time-consuming data, so that operation and maintenance personnel can view the time-consuming data in the display system according to the time dimension to search.

Based on the description of the related embodiments of the above-mentioned link time-consuming analysis method, an embodiment of the present invention also proposes a link time-consuming analysis device. The link time-consuming analysis device may be a computer program (including program code) running on an electronic device, where the electronic device refers to an intermediate proxy device in a storage system, and the storage system includes at least one client, an intermediate proxy device, and a storage device cluster managed by the intermediate proxy device; as shown in FIG. The link time-consuming analysis device can execute the link time-consuming analysis method shown in Figure 2 or Figure 4, that is, the link time-consuming analysis device can run the above units:

The processing unit 702 is configured to add the data access request to the first queue when receiving the data access request sent by the target client, and determine the enqueue time when the data access request is added to the first queue, and the first queue is used to store requests received from the client;

The sending unit 703 is configured to send the data access request to a target storage device included in the storage device cluster through the first queue, so that the target storage device executes the data access request to obtain a target request result of the data access request, and the target storage device is any storage device in the storage device cluster;

a receiving unit 701, configured to receive the target request result returned by the target storage device;

The processing unit 702 is further configured to add the target request result to a second queue, and determine the dequeue time for the target request result to come out of the second queue to be sent to the target client, and the second queue is used to store the request result to be sent to the client;

The processing unit 702 is further configured to determine the time-consuming result of the middle layer of the data access request by using the enqueue time and the dequeue time.

In one embodiment, when the sending unit 703 sends the data access request to the target storage device included in the storage device cluster through the first queue, it may be specifically configured to:

Send the data access request to a third queue through the first queue, and the third queue is used to store requests to be sent to the storage device;

Using the third queue, sending the data access request to a target storage device included in the storage device cluster;

When the processing unit 702 adds the target request result to the second queue, it may be specifically configured to:

adding the target request result to a fourth queue, where the fourth queue is used to store the request result received from the storage device;

Send the target request result to the second queue through the fourth queue, so that the target request result is added to the second queue.

In another embodiment, the intermediate proxy device includes M queue groups, M is a positive integer, one queue group corresponds to one processing pool, and the data involved in a request is transmitted through each queue in the same queue group, then the first queue and the second queue are queues in the target queue group, and the target queue group is any queue group in the M queue groups; the receiving unit 701 can also be used for:

receive multiple requests;

The processing unit 702 can also be used for:

Distributing the multiple requests to processing pools corresponding to N queue groups in the M queue groups, where N is a positive integer less than or equal to M;

The requests in the respective queue groups are processed in parallel through each of the N queue groups, so that the request results and the time-consuming results of the middle layer of the multiple requests are obtained in parallel through corresponding storage devices.

In another implementation manner, the processing unit 702 may also be used to:

storing the time-consuming result of the intermediate layer in the first time-consuming data, so as to add a piece of stored data including the time-consuming result of the intermediate layer to the first time-consuming data, and the first time-consuming data includes the time-consuming result of the intermediate layer of each request received by the intermediate proxy device; or,

If the time-consuming result corresponding to the intermediate layer is longer than the first preset duration, store the time-consuming result of the intermediate layer in the second time-consuming data to add a piece of storage data including the time-consuming result of the intermediate layer to the second time-consuming data, and the second time-consuming data includes the time-consuming result of the intermediate layer for each first slow query request in at least one first slow query request, and the first slow query request refers to a request for which the corresponding duration of the time-consuming result of the intermediate layer is longer than the first preset duration; or,

If the corresponding duration of the time-consuming result of the middle layer is longer than the second preset duration, the data access request is stored in the slow query data to add a piece of stored data including the data access request to the slow query data, and the slow query data includes at least one second slow query request, and the second slow query request refers to a request whose corresponding duration of the time-consuming result of the middle layer is longer than the second preset duration.

In another implementation manner, the processing unit 702 may also be used to:

Determine the expiration time, and when it is detected that the storage time of any storage data in the target storage data is longer than the expiration time, delete the any storage data, and the target storage data is any data in the first time-consuming data, the second time-consuming data and the slow query data;

Wherein, the storage duration of any stored data refers to the difference between the storage time and the current system time.

In another embodiment, the target storage device includes at least one probe group, and any probe group includes a start probe and at least one stop probe, and during a link execution process, only one start probe and one stop probe are executed in any probe group; wherein, a probe is set with a corresponding probe identifier; when the processing unit 702 sends the data access request to the target storage device included in the storage device cluster, so that the target storage device executes the data access request and obtains a target request result of the data access request, it can be specifically used for:

Sending the data access request to the target storage device included in the storage device cluster, so that the target storage device executes the data access request according to the corresponding link, and when the target probe group in the at least one probe group is executed, print the probe ID of the target start probe executed in the target probe group and the probe ID of the target end probe executed in the target probe group;

Wherein, the printing time of the probe identification in the target start probe and the printing time of the probe identification in the target termination probe are used to determine the local time-consuming result when the data access request is executed according to the link monitored by the target probe group.

On the other hand, the embodiment of the present application also proposes another link time-consuming analysis device. The link time-consuming analysis device may be a computer program (including program code) running on an electronic device, and the electronic device here refers to: a target instance proxy device in a storage system. The storage system includes at least one client, an intermediate proxy device, a storage device cluster managed by the intermediate proxy device, and an instance proxy device corresponding to each storage device in the storage device cluster. A storage device corresponding to the proxy device of the target instance in the device cluster; as shown in FIG. 7 b , the apparatus for link time consumption analysis may include a processing unit 704 . The link time-consuming analysis device can execute the link time-consuming analysis method shown in Figure 6, that is, the link time-consuming analysis device can run the above units:

The processing unit 704 is configured to record the receiving time of the first message data when detecting that the target storage device receives the first message data in the data access request, the data access request includes at least one message data, and the receiving time of the first message data is earlier than the receiving time of any message data in the at least one message data except the first message data;

The processing unit 704 is further configured to, when it is detected that a response message of each message data in the at least one message data is sent, record the sending time of the response message of each message data, and a response message of one message data is the execution result of the corresponding message data by the storage device;

The processing unit 704 is further configured to use the receiving time of the first message data and the sending time of the response message of the second message data to determine the time-consuming result of the instance layer of the data access request, and the sending time of the response message of the second message data is later than the sending time of the response messages of other message data in the at least one message data.

In one embodiment, the storage system further includes a display system, and a processing unit 704, which can also be used for:

Determine the device identifier of the target storage device, and serialize the instance layer time-consuming result and the device identifier to obtain serialized time-consuming data;

The above link time-consuming analysis device may also include a sending unit 705, configured to:

The serialized time-consuming data is sent to the display system, so that the display system displays the time-consuming data within a specified time range.

According to an embodiment of the present invention, each step involved in the method shown in FIG. 2 or FIG. 4 can be executed by each unit in the link time consumption analysis device shown in FIG. 7a. According to another embodiment of the present invention, each step involved in the method shown in FIG. 6 can be executed by each unit in the link time consumption analysis device shown in FIG. 7b.

According to another embodiment of the present invention, each unit in the link time-consuming analysis device shown in Fig. 7a and Fig. 7b can be respectively or all combined into one or several other units to form, or one (some) units can be further divided into functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effect of the embodiments of the present invention. The above-mentioned units are divided based on logical functions. In practical applications, the functions of one unit may also be realized by multiple units, or the functions of multiple units may be realized by one unit. In other embodiments of the present invention, any link time consumption analysis device may also include other units, and in practical applications, these functions may also be implemented with the assistance of other units, and may be implemented cooperatively by multiple units.

According to another embodiment of the present invention, a computer program (including program code) capable of executing the steps involved in the corresponding methods as shown in FIG. 2 or FIG. 4 can be run on a general-purpose electronic device such as a computer that includes processing elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). RAM), read-only storage medium (ROM) and other processing elements and storage elements such as computer general-purpose electronic equipment run computer programs (including program codes) that can execute the steps involved in the corresponding methods as shown in Figure 6, to construct the link time-consuming analysis device as shown in Figure 7b, and to realize the link time-consuming analysis method of the embodiment of the present invention. The computer program may be recorded in, for example, a computer storage medium, loaded into the above-mentioned electronic device through the computer storage medium, and run therein.

In the embodiment of the present invention, when the intermediate proxy device receives the data access request sent by the target client, it can add the data access request to the first queue, and determine the enqueue time when the data access request is added to the first queue; then, the intermediate proxy device can send the data access request to the target storage device through the first queue, so that the target storage device executes the data access request, and obtains the target request result of the data access request; The dequeue time sent to the target client is used to determine the time-consuming result of the middle layer of the data access request by using the queue-entry time and the dequeue time, so that the storage system based on the intermediate proxy device can support link time-consuming analysis. Correspondingly, the constructed storage system may further include an instance agent device corresponding to each storage device in the storage device cluster. For the target instance agent device, the target instance agent device may record the receiving time of the first message data when detecting that the target storage device receives the first message data in the data access request. When the file is sent, record the sending time of the response message of each message data, so as to use the receiving time of the first message data and the sending time of the response message of the second message data to determine the time-consuming result of the instance layer of the data access request, so as to obtain a more accurate time-consuming result. It can be seen that the embodiment of the present invention can build a storage system that supports link time-consuming analysis, so as to realize link time-consuming analysis of requests from at least one aspect, especially when the storage device is a memory object caching system, it can implement link time-consuming analysis of requests for accessing the memory object caching system, so as to solve the problem that the memory object caching system does not support link time-consuming analysis of corresponding requests, thereby solving problems related to the storage system including the memory object caching system.

Based on the descriptions of the method embodiment and the device embodiment above, an exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, and when executed by the at least one processor, the computer program is used to cause the electronic device to execute the method according to the embodiment of the present invention.

Exemplary embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method according to the embodiment of the present invention.

Exemplary embodiments of the present invention also provide a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method according to the embodiment of the present invention.

Referring to FIG. 8 , a structural block diagram of an electronic device 800 that can serve as a server or a client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. Electronic device is intended to mean various forms of digital electronic computing equipment, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the inventions described and/or claimed herein.

As shown in FIG. 8 , the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the device 800 can also be stored. The computing unit 801 , ROM 802 , and RAM 803 are connected to each other through a bus 804 . An input/output (I/O) interface 805 is also connected to the bus 804 .

Multiple components in the electronic device 800 are connected to the I/O interface 805 , including: an input unit 806 , an output unit 807 , a storage unit 808 and a communication unit 809 . The input unit 806 can be any type of device capable of inputting information to the electronic device 800. The input unit 806 can receive input numeric or character information, and generate key signal input related to user settings and/or function control of the electronic device. The output unit 807 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 808 may include, but is not limited to, a magnetic disk and an optical disk. The communication unit 809 allows the electronic device 800 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and/or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and/or the like.

The computing unit 801 may be various general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of computing units 801 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any suitable processors, controllers, microcontrollers, etc. The computing unit 801 executes the various methods and processes described above. For example, in some embodiments, the link timing analysis method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 808 . In some embodiments, part or all of the computer program may be loaded and/or installed on the electronic device 800 via the ROM 802 and/or the communication unit 809 . In some embodiments, the calculation unit 801 may be configured to execute the link time consumption analysis method in any other appropriate manner (for example, by means of firmware).

Program codes for implementing the methods of the present invention may be written in any combination of one or more programming languages. These program codes may be provided to processors or controllers of general-purpose computers, special purpose computers, or other programmable data processing devices, so that the program codes cause the functions/operations specified in the flowcharts and/or block diagrams to be implemented when executed by the processors or controllers. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include one or more wire-based electrical connections, a portable computer disk, a hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or means (e.g., magnetic disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and/or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, voice input, or tactile input.

The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.

A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other.

And, it should be understood that what is disclosed above is only a preferred embodiment of the present invention, which certainly cannot limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.

Claims (12)

1. A link time-consuming analysis method, characterized in that, the method is applied to an intermediate proxy device in a storage system, and the storage system includes at least one client, the intermediate proxy device, and a storage device cluster managed by the intermediate proxy device; the method includes: When receiving the data access request sent by the target client, adding the data access request to the first queue, and determining the enqueue time when the data access request is added to the first queue, the first queue is used to store requests received from the client; Send the data access request to a target storage device included in the storage device cluster through the first queue, so that the target storage device executes the data access request to obtain a target request result of the data access request, and the target storage device is any storage device in the storage device cluster; receiving the target request result returned by the target storage device, adding the target request result to a second queue, and determining the dequeue time for the target request result to come out of the second queue to be sent to the target client, and the second queue is used to store the request result to be sent to the client; Using the enqueue time and the dequeue time, determine the time-consuming result of the middle layer of the data access request. 2. The method according to claim 1, wherein the sending the data access request to the target storage device included in the storage device cluster through the first queue comprises: Send the data access request to a third queue through the first queue, and the third queue is used to store requests to be sent to the storage device; Using the third queue, sending the data access request to a target storage device included in the storage device cluster; The adding the target request result to the second queue includes: adding the target request result to a fourth queue, where the fourth queue is used to store the request result received from the storage device; Send the target request result to the second queue through the fourth queue, so that the target request result is added to the second queue. 3. The method according to claim 1 or 2, wherein the intermediate proxy device includes M queue groups, M is a positive integer, a queue group corresponds to a processing pool, and the data involved in a request is transmitted through each queue in the same queue group, then the first queue and the second queue are queues in the target queue group, and the target queue group is any queue group in the M queue groups; the method also includes: receiving multiple requests, and distributing the multiple requests to processing pools corresponding to N queue groups in the M queue groups, where N is a positive integer less than or equal to M; The requests in the respective queue groups are processed in parallel through each of the N queue groups, so that the request results and the time-consuming results of the middle layer of the multiple requests are obtained in parallel through corresponding storage devices. 4. method according to claim 1 or 2, is characterized in that, described method also comprises: storing the time-consuming result of the intermediate layer in the first time-consuming data, so as to add a piece of stored data including the time-consuming result of the intermediate layer to the first time-consuming data, and the first time-consuming data includes the time-consuming result of the intermediate layer of each request received by the intermediate proxy device; or, If the time-consuming result corresponding to the intermediate layer is longer than the first preset duration, store the time-consuming result of the intermediate layer in the second time-consuming data to add a piece of storage data including the time-consuming result of the intermediate layer to the second time-consuming data, and the second time-consuming data includes the time-consuming result of the intermediate layer for each first slow query request in at least one first slow query request, and the first slow query request refers to a request for which the corresponding duration of the time-consuming result of the intermediate layer is longer than the first preset duration; or, If the corresponding duration of the time-consuming result of the middle layer is longer than the second preset duration, the data access request is stored in the slow query data to add a piece of stored data including the data access request to the slow query data, and the slow query data includes at least one second slow query request, and the second slow query request refers to a request whose corresponding duration of the time-consuming result of the middle layer is longer than the second preset duration. 5. method according to claim 4, is characterized in that, described method also comprises: Determine the expiration time, and when it is detected that the storage time of any storage data in the target storage data is longer than the expiration time, delete the any storage data, and the target storage data is any data in the first time-consuming data, the second time-consuming data and the slow query data; Wherein, the storage duration of any stored data refers to the difference between the storage time and the current system time. 6. The method according to claim 1 or 2, wherein the target storage device includes at least one probe group, any probe group includes a start probe and at least one end probe, and during a link execution process, only one start probe and one end probe are executed in any probe group; wherein, a probe is provided with a corresponding probe identifier; the sending the data access request to the target storage device included in the storage device cluster, so that the target storage device executes the data access request, and obtains a target request result of the data access request, comprising: Sending the data access request to the target storage device included in the storage device cluster, so that the target storage device executes the data access request according to the corresponding link, and when the target probe group in the at least one probe group is executed, print the probe ID of the target start probe executed in the target probe group and the probe ID of the target end probe executed in the target probe group; Wherein, the printing time of the probe identification in the target start probe and the printing time of the probe identification in the target termination probe are used to determine the local time-consuming result when the data access request is executed according to the link monitored by the target probe group. 7. A link time-consuming analysis method, wherein the method is applied to a target instance proxy device in a storage system, the storage system includes at least one client, an intermediate proxy device, a storage device cluster managed by the intermediate proxy device, and an instance proxy device corresponding to each storage device in the storage device cluster, the target instance proxy device supports monitoring and capturing information of the intermediate proxy device for detecting messages received and sent by the target storage device, and the target storage device is a storage device corresponding to the target instance proxy device in the storage device cluster; the method includes: When it is detected that the target storage device receives the first message data in the data access request, record the receiving time of the first message data, the data access request includes at least one message data, and the receiving time of the first message data is earlier than the receiving time of any message data in the at least one message data except the first message data; When it is detected that the response message of each message data in the at least one message data is sent, record the sending time of the response message of each message data, and the response message of one message data is the execution result of the corresponding message data by the storage device; Using the receiving time of the first message data and the sending time of the response message of the second message data to determine the time-consuming result of the instance layer of the data access request, the sending time of the response message of the second message data is later than the sending time of the response messages of other message data in the at least one message data. 8. The method according to claim 7, wherein the storage system further comprises a display system, and the method further comprises: Determine the storage device identifier of the target storage device, and serialize the instance layer time-consuming result and the storage device identifier to obtain serialized time-consuming data; The serialized time-consuming data is sent to the display system, so that the display system displays the time-consuming data within a specified time range. 9. A link time-consuming analysis device, characterized in that the device operates on an intermediate proxy device in a storage system, and the storage system includes at least one client, the intermediate proxy device, and a storage device cluster managed by the intermediate proxy device; the device includes: The processing unit is configured to add the data access request to the first queue when receiving the data access request sent by the target client, and determine the enqueue time when the data access request is added to the first queue, and the first queue is used to store requests received from the client; A sending unit, configured to send the data access request to a target storage device included in the storage device cluster through the first queue, so that the target storage device executes the data access request to obtain a target request result of the data access request, and the target storage device is any storage device in the storage device cluster; a receiving unit, configured to receive the target request result returned by the target storage device; The processing unit is further configured to add the target request result to a second queue, and determine the dequeue time for the target request result to come out of the second queue to be sent to the target client, and the second queue is used to store the request result to be sent to the client; The processing unit is further configured to use the enqueue time and the dequeue time to determine the time-consuming result of the middle layer of the data access request. 10. A link time-consuming analysis device, characterized in that the device runs on a target instance proxy device in a storage system, and the storage system includes at least one client, an intermediate proxy device, a storage device cluster managed by the intermediate proxy device, and an instance proxy device corresponding to each storage device in the storage device cluster, and the target instance proxy device supports monitoring and capturing information of the intermediate proxy device for detecting messages received and sent by the target storage device, and the target storage device is a storage device corresponding to the target instance proxy device in the storage device cluster; the device includes: A processing unit, configured to record the receiving time of the first message data when detecting that the target storage device receives the first message data in the data access request, the data access request includes at least one message data, and the receiving time of the first message data is earlier than the receiving time of any message data in the at least one message data except the first message data; The processing unit is further configured to record the sending time of the response message of each message data when detecting that the response message of each message data in the at least one message data is sent, and the response message of one message data is the execution result of the corresponding message data by the storage device; The processing unit is further configured to use the receiving time of the first message data and the sending time of the response message of the second message data to determine the time-consuming result of the instance layer of the data access request, and the sending time of the response message of the second message data is later than the sending time of the response messages of other message data in the at least one message data. 11. An electronic device, characterized in that it comprises: processor; and memory for storing programs, Wherein, the program includes instructions, which when executed by the processor cause the processor to perform the method according to any one of claims 1-6; or, when executed by the processor, the instructions cause the processor to perform the method according to claim 7 or 8. 12. A non-transitory computer-readable storage medium stored with computer instructions, wherein the computer instructions are used to cause a computer to perform the method according to any one of claims 1-6; or, the computer instructions are used to cause a computer to perform the method according to claim 7 or 8.