CN107276857A - A kind of method and device for monitoring flow - Google Patents

Abstract

The invention discloses a method for monitoring traffic, the method comprising: receiving a request sent by a client; judging whether the request is a read-write request; if so, performing traffic monitoring on the request through a read-write request processing thread, And store the monitored traffic data and the request in a storage structure corresponding to the client. This method provides a corresponding storage structure for the client, and at the same time judges the sent request, provides a dedicated processing thread for the read and write request, and then monitors the read and write flow data, and stores the flow data and the request in the The storage structure corresponding to the above client. It facilitates the management of the traffic data of the client's read and write requests, and at the same time provides a basis for controlling the traffic data of the client's read and write requests in the future and realizing traffic restrictions. Correspondingly, a flow monitoring device disclosed in the present invention also has the above-mentioned technical effects.

Description

A method and device for monitoring flow

technical field

The invention relates to flow monitoring of a distributed storage system, more specifically, to a method and device for monitoring flow.

Background technique

With the rapid development of computer technology and the Internet industry, the cluster computer network has become mature, and the shared resources in it are also increasing. The Network File System (NFS), as a way of sharing data in a network environment, greatly facilitates the use of shared resources by users. Network File System (NFS) allows a system to share directories and files with others on the network, and users can access files on remote systems as if they were accessing local files. Network File System (NFS) has been widely used in various business scenarios in the field of distributed storage because of its excellent stability and scalability.

At present, in the field of distributed storage, data services are usually provided to all users through the network file system (NFS) service. Therefore, during the peak period of business, high pressure is likely to cause network congestion and excessive pressure on cluster services, and user requests cannot be processed in time. It causes a backlog or delay of requests, which may lead to service system failure or paralysis of the entire service system in severe cases. Since the traffic usage behavior of each user is not recorded on the server side, service managers are helpless to deal with network congestion during peak business hours and cannot solve the congestion in time.

Therefore, how to monitor users' traffic usage behavior in real time, analyze and manage traffic data, and then realize accurate detection and control of traffic is a problem to be solved by those skilled in the art.

Contents of the invention

The purpose of the present invention is to provide a method and device for monitoring traffic, so as to realize real-time monitoring of traffic usage behavior of users, and analyze and manage traffic data.

In order to achieve the above object, the embodiment of the present invention provides the following technical solutions:

A method of monitoring traffic, comprising:

S11. Receive the request sent by the client;

S12. Determine whether the request is a read-write request;

S13. If yes, monitor the flow of the request through the read and write request processing thread, and store the monitored flow data and the request in a storage structure corresponding to the client.

Wherein, after receiving the request sent by the client, it also includes:

judging whether the client is sending the request for the first time;

If yes, create a storage structure corresponding to the client, and execute S12;

If not, execute S12.

Wherein, the judging whether the client sends the request for the first time includes:

Determine whether the client sends the request for the first time according to the IP of the client sending the request.

Wherein, the flow monitoring of the request through the read-write request processing thread, and storing the monitored flow data and the request in a storage structure corresponding to the client includes:

If the request is a read request, the read and write request processing thread corresponding to the client will monitor the flow of the read request, and store the monitored read flow data and the read request to the client. The corresponding read request storage structure;

If the request is a write request, monitor the flow of the write request through the read and write request processing thread corresponding to the client, and store the monitored write flow data and the write request to the client The corresponding write request storage structure.

Wherein, if the said yes, monitor the traffic of the request through the read and write request processing thread, and store the monitored traffic data and the request in the storage structure corresponding to the client, it also includes:

Through the flow monitoring thread and timer, the total flow value of all clients is counted in seconds and stored;

If the total flow value exceeds a preset threshold, an alarm is issued.

A device for monitoring flow comprising:

The receiving module is used to receive the request sent by the client;

A first judging module, configured to judge whether the request is a read-write request;

A monitoring module, configured to monitor the flow of the request through a read-write request processing thread when the request is a read-write request, and store the monitored flow data and the request in a storage structure corresponding to the client .

Among them, also include:

The second judging module is used to judge whether the client sends a request for the first time;

A creating module, configured to create a storage structure corresponding to the client when the client sends a request for the first time.

Wherein, the second judging module is specifically configured to: judge whether the client sends the request for the first time according to the IP of the client sending the request.

Wherein, the monitoring module includes:

A first monitoring unit, configured to monitor the flow of the read request through a read and write request processing thread corresponding to the client when the request is a read request;

A first storage unit, configured to store the monitored read traffic data and the read request in a read request storage structure corresponding to the client;

The second monitoring unit is configured to monitor the flow of the write request through the read and write request processing thread corresponding to the client when the request is a write request;

The second storage unit is configured to store the monitored write traffic data and the write request in a write request storage structure corresponding to the client.

Among them, also include:

The total flow value monitoring module is used to count and store the total flow value of all clients in seconds through the flow monitoring thread and timer;

An alarm module, configured to issue an alarm when the total flow value exceeds a preset threshold.

It can be seen from the above scheme that the embodiment of the present invention provides a method for monitoring traffic, the method includes: receiving the request sent by the client; judging whether the request is a read-write request; Traffic monitoring is performed on the request, and the monitored traffic data and the request are stored in a storage structure corresponding to the client. In this way, the server can monitor the traffic data generated by the client's read and write requests in real time, and perform statistical analysis, which provides an effective basis for managing and controlling traffic.

It can be seen that this method provides a corresponding storage structure for each client, and at the same time judges the sent request, provides a dedicated processing thread for the read and write request, and then monitors the read and write traffic data, and compares the traffic data with the request stored in a storage structure corresponding to the client. By monitoring and counting the read and write flow data of each client, it is convenient to manage the flow data of each client's read and write requests, and at the same time control the flow data of each client's read and write requests in the future to achieve flow limitation provides the basis.

Correspondingly, a device for monitoring flow provided by an embodiment of the present invention also has the above-mentioned technical effect.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a flow chart of a method for monitoring flow disclosed by an embodiment of the present invention;

FIG. 2 is a schematic diagram of a classification structure of read and write requests disclosed in FIG. 1 in a method for monitoring traffic;

FIG. 3 is a flow chart of another method for monitoring flow disclosed in the embodiment of the present invention;

FIG. 4 is a schematic diagram of a classification structure of read and write requests disclosed in FIG. 3 in a method for monitoring traffic;

Fig. 5 is a schematic diagram of a flow monitoring device disclosed in an embodiment of the present invention.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The embodiment of the present invention discloses a method and device for monitoring traffic, so as to realize real-time monitoring of user traffic usage behavior and analyze and manage traffic data.

Referring to Figure 1, a method for monitoring traffic provided by an embodiment of the present invention includes:

S11. Receive the request sent by the client;

Specifically, in the field of distributed storage, when a client needs to access a network file, it will send a request to the server. Usually, the request is transmitted in the form of a message, so after receiving the request, the server needs to analyze and process the request.

S12. Determine whether the request is a read-write request;

Specifically, after the server parses the request, it will obtain information such as the specific content of the request, the sender, and the transmission path, and then the server will judge whether the request is a read-write request based on the specific content of the request. , so that read and write requests are handled differently from requests such as creation and transfer.

S13. If yes, monitor the flow of the request through the read and write request processing thread, and store the monitored flow data and the request in a storage structure corresponding to the client;

It should be noted that since the server allocates a dedicated thread pool for read and write requests, after judging that the request is a read and write request, the server will monitor the flow of the request through the read and write request processing thread, and The monitored traffic data and the request are stored in a storage structure corresponding to the client.

Referring to FIG. 2, FIG. 2 is a schematic diagram of the classification structure of read and write requests in this embodiment. The read and write request processing thread monitors the flow of the request, and stores the monitored flow data and the request to the client. The storage structure corresponding to the end, including:

If the request is a read request, the read and write request processing thread corresponding to the client will monitor the flow of the read request, and store the monitored read flow data and the read request to the client. The corresponding read request storage structure;

If the request is a write request, monitor the flow of the write request through the read and write request processing thread corresponding to the client, and store the monitored write flow data and the write request to the client The corresponding write request storage structure.

Specifically, in this embodiment, the server allocates a thread pool for processing read and write requests for each client, that is, each client has a corresponding thread pool for processing read and write requests; and the service The terminal provides each client with a storage structure for storing data related to read and write requests, that is, each client has a corresponding storage structure.

When the server receives the read and write request sent by the client, it will automatically hand over the request to the thread in the thread pool corresponding to the client for traffic monitoring and traffic data storage; if the request is a read request, then the relevant thread When the traffic is monitored, the monitored traffic data and the read request will be automatically stored in the read request storage structure corresponding to the client. If the request is a write request, when the relevant thread monitors its traffic, it will automatically store the monitored traffic data and the write request in the write request storage structure corresponding to the client. That is to say, the system automatically stores the traffic data of read requests and write requests of the same client separately, so as to facilitate statistical analysis and management of read traffic data and write traffic data in the future. Of course, the storage structure also stores corresponding client information, including information such as IP addresses and physical addresses.

S14. If not, process through other threads.

Specifically, if the request is other requests, such as creation and transfer requests, etc., the server will automatically perform related processing through other threads, and return the processing results to the client in time.

It can be seen that this embodiment provides a method for monitoring traffic. After receiving the request sent by the client, the method judges the type of the sent request, provides a dedicated processing thread for the read and write requests, and then monitors the read and write traffic data. , and store the traffic data of the read request and the traffic data of the write request in the corresponding read request storage structure and write request storage structure respectively. In this way, by monitoring and storing the read and write flow data of each client separately, it is convenient to manage the flow data of each client's read and write requests, and at the same time control the flow data of each client's read and write requests in the future to realize the flow Limits provide the basis.

Referring to Fig. 3, another method for monitoring traffic provided by an embodiment of the present invention includes:

S31. Receive a request sent by the client;

S32. Determine whether the client sends the request for the first time;

It should be noted that after the server receives the request sent by the client, it first analyzes the request to obtain information such as the IP of the client that sent the request and the specific content of the request, and then the server will Judging whether the client is sending the request for the first time based on the client information of the request, that is, judging whether the client is accessing the server for the first time.

In this embodiment, the judging whether the client sends the request for the first time includes:

Determine whether the client sends the request for the first time according to the IP of the client sending the request.

Specifically, since the server provides each client with a storage structure for storing client information and reading and writing traffic data, the information of clients with access history records, such as IP addresses, etc., have been stored in the server middle. When a client accesses the server for the first time, the server obtains the IP address of the client, and then searches for the corresponding IP address in the system. If the corresponding IP address cannot be found, it is judged that the client is newly added. The client is sending a request for the first time. Of course, it is also possible to determine whether the client is sending a request for the first time according to the physical address of the client. The detailed process is similar to the judgment using the IP address, so it will not be repeated here.

S33. If yes, create a storage structure corresponding to the client, and execute S34; if not, execute S34;

Specifically, after judging that the client sends the request for the first time according to the IP of the client that sent the request, the server creates a corresponding storage structure for the client that sends the request for the first time to store the client's read and write requests, read Write traffic data and information about the client, etc.

S34. Determine whether the request is a read-write request;

S35. If so, monitor the flow of the request through the read and write request processing thread, and store the monitored flow data and the request in a storage structure corresponding to the client;

S36. If not, process through other threads.

Specifically, if the client is sending a request for the first time, after creating a corresponding storage structure for it, it is necessary to determine whether the request type sent by the client is a read-write request; if it is a read-write request, then pass the read-write request The processing thread performs related processing.

It should be noted that since the server allocates a dedicated thread pool for read and write requests, after judging that the request is a read and write request, the server will monitor the traffic of the request through the read and write request processing thread, and will monitor The traffic data and the request are stored in a storage structure corresponding to the client.

Referring to Fig. 4, Fig. 4 is a schematic diagram of the classification structure of read and write requests in this embodiment. The server allocates a dedicated thread pool for read and write requests in the system, that is, the threads in the thread pool are used to process the read and write requests of all clients . Similar to the above embodiment, this embodiment also provides each client with a corresponding storage structure, including a read request storage structure and a write request storage structure; the traffic data of the read request and the traffic data of the write request are stored in the read request storage The structure and write request structure, that is, the client storage structure provided by this embodiment is the same as the client storage structure in the foregoing embodiments. Compared with the above embodiment, in this embodiment, all clients share read and write request processing threads, that is, the server allocates a dedicated thread pool for processing read and write requests to all clients in the system.

Specifically, when the server receives the read and write request sent by the client, the server automatically monitors the traffic through the idle thread in the read and write request thread pool, and stores the monitored traffic data and the request to the The storage structure corresponding to the end. That is, the thread pool for processing read and write requests in this embodiment does not correspond to each client, but all clients share the same thread pool. Of course, the clients can also be divided according to the number or other information, and two clients share one thread pool, or three or five clients share one thread pool, which is not specifically limited in this embodiment.

It can be seen that in the method for monitoring traffic provided by this embodiment, after receiving the request sent by the client, the method first judges whether the sending end is sending the request for the first time, and if so, judges the type of request sent; Provide a dedicated processing thread for the read and write requests of the client, that is, all clients share the same thread pool to process the read and write requests; then monitor the read and write traffic data, and store the traffic data of the read request and the traffic data of the write request in the corresponding Read request storage structure and write request storage structure. In this way, by monitoring and storing the read and write flow data of each client separately, it is convenient to manage the flow data of each client's read and write requests, and at the same time control the flow data of each client's read and write requests in the future to realize the flow Limits provide the basis.

Based on any of the above embodiments, it should be noted that when monitoring the read and write traffic data of each client in real time, the server counts the read and write traffic data of all clients.

Specifically, if the said if, monitors the traffic of the request through the read and write request processing thread, and stores the monitored traffic data and the request in the storage structure corresponding to the client, it also includes:

Through the flow monitoring thread and timer, the total flow value of all clients is counted in seconds and stored;

If the total flow value exceeds a preset threshold, an alarm is issued.

The server system has a dedicated thread for traffic value statistics, and it cooperates with timers to count traffic values. Update the read and write traffic values of all clients in seconds and store them in real time. Based on statistical analysis of historical read and write traffic, the server presets a security threshold to alert managers of network congestion. If the total flow value exceeds the preset safety threshold, the system will perform alarm processing. Administrators can limit the flow of certain clients to reduce network congestion and avoid system crashes.

Based on any of the above embodiments, after monitoring the flow of read and write requests of each client, and obtaining the monitored flow data, the flow data can also be returned to the client in time, so that the user can know the flow data of the current operation in time. If a flow-limiting strategy is formulated to limit the flow of each client, then after the user knows the flow data that has been used, he can reduce it as appropriate and reasonably control the number of visits.

A device for monitoring flow rate provided by an embodiment of the present invention is introduced below. The device for monitoring flow rate described below and the method for monitoring flow rate described above may refer to each other.

Referring to Figure 5, a flow monitoring device provided by an embodiment of the present invention includes:

A receiving module 501, configured to receive a request sent by a client;

A first judging module 502, configured to judge whether the request is a read-write request;

The monitoring module 503 is configured to monitor the flow of the request through a read-write request processing thread when the request is a read-write request, and store the monitored flow data and the request in a storage corresponding to the client structure.

Among them, also include:

The second judging module is used to judge whether the client sends a request for the first time;

A creating module, configured to create a storage structure corresponding to the client when the client sends a request for the first time.

Wherein, the second judging module is specifically configured to: judge whether the client sends the request for the first time according to the IP of the client sending the request.

Wherein, the monitoring module includes:

A first monitoring unit, configured to monitor the flow of the read request through a read and write request processing thread corresponding to the client when the request is a read request;

A first storage unit, configured to store the monitored read traffic data and the read request in a read request storage structure corresponding to the client;

The second monitoring unit is configured to monitor the flow of the write request through the read and write request processing thread corresponding to the client when the request is a write request;

The second storage unit is configured to store the monitored write traffic data and the write request in a write request storage structure corresponding to the client.

Among them, also include:

The total flow value monitoring module is used to count and store the total flow value of all clients in seconds through the flow monitoring thread and timer;

An alarm module, configured to issue an alarm when the total flow value exceeds a preset threshold.

It can be seen that the device for monitoring traffic provided by this embodiment includes: a receiving module, a first judging module, and a monitoring module. First, the receiving module receives the request sent by the client; then the first judging module judges whether the request is a read-write request; when the request is a read-write request, the monitoring module processes the request through the read-write request processing thread monitor, and store the monitored traffic data and the request in a storage structure corresponding to the client. In this way, the division of labor and cooperation among the various modules, each performing its own duties, jointly realize the real-time monitoring of the read and write traffic of each client; at the same time, through the monitoring and statistics of the read and write traffic data of each client, it is convenient The management of read and write requests also provides a basis for controlling the read and write requests of each client in the future and realizing traffic limitation.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. a kind of method for monitoring flow, it is characterised in that including：

S11, the request for receiving client transmission；

S12, judge it is described request whether be read-write requests；

S13, if so, then handle thread by read-write requests carries out flow monitoring to the request, and by the data on flows of monitoring Stored with the request to storage organization corresponding with the client.

2. the method for monitoring flow according to claim 1, it is characterised in that the request that the reception client is sent it Afterwards, in addition to：

Whether judge the client is to send request first；

If so, then creating storage organization corresponding with the client, and perform S12；

If it is not, then performing S12.

3. the method for monitoring flow according to claim 2, it is characterised in that described to judge whether the client is first Secondary send is asked, including：

Client ip according to the request is sent judges whether the client is to send request first.

4. the method for monitoring flow according to claim 1, it is characterised in that described to handle thread pair by read-write requests The request carries out flow monitoring, and the data on flows of monitoring and the request are stored to storage corresponding with the client Structure, including：

If the request is read request, handles thread by read-write requests corresponding with the client and the read request is entered Row flow monitoring, and the reading data on flows and the read request of monitoring are stored to read request corresponding with client storage Structure；

If the request is write request, handles thread by read-write requests corresponding with the client and the write request is entered Row flow monitoring, and by monitoring write data on flows and the write request is stored to write request corresponding with the client and stored Structure.

5. the method for the monitoring flow according to claim 1-4 any one, it is characterised in that described if so, by reading Write request handle thread to it is described request carry out flow monitoring, and by the data on flows of monitoring and the request store to it is described After the corresponding storage organization of client, in addition to：

By flow monitoring thread and timer, the total flow value of all clients is counted in seconds, and is stored；

If the total flow value exceedes the threshold value pre-set, alarmed.

6. a kind of device for monitoring flow, it is characterised in that including：

Receiving module, the request for receiving client transmission；

First judge module, for judging whether the request is read-write requests；

Monitoring modular, for when the request is read-write requests, handling thread by read-write requests and being flowed to the request Amount monitoring, and the data on flows of monitoring and the request are stored to storage organization corresponding with the client.

7. the device of monitoring flow according to claim 6, it is characterised in that also include：

Second judge module, for judging whether the client is to send request first；

Creation module, for when the client sends request first, creating storage organization corresponding with the client.

8. the device of monitoring flow according to claim 7, it is characterised in that

Second judge module specifically for：Client ip according to the request is sent judges whether the client is first It is secondary to send request.

9. the device of monitoring flow according to claim 6, it is characterised in that the monitoring modular includes：

First monitoring unit, for when the request is read request, being handled by read-write requests corresponding with the client Thread carries out flow monitoring to the read request；

First memory cell, reads for the reading data on flows and the read request of monitoring to be stored to corresponding with the client Ask storage organization；

Second monitoring unit, for when the request is write request, being handled by read-write requests corresponding with the client Thread carries out flow monitoring to the write request；

Second memory cell, for by monitoring write data on flows and the write request is stored to corresponding with the client and write Ask storage organization.

10. the device of the monitoring flow according to claim 6-9 any one, it is characterised in that also include：

Total flow value monitoring modular, for by flow monitoring thread and timer, all clients to be counted in seconds Total flow value, and stored；

Alarm module, for when the total flow value exceedes the threshold value pre-set, being alarmed.