CN114201659A - A message track transmission query method, device and system - Google Patents
A message track transmission query method, device and system Download PDFInfo
- Publication number
- CN114201659A CN114201659A CN202111368224.4A CN202111368224A CN114201659A CN 114201659 A CN114201659 A CN 114201659A CN 202111368224 A CN202111368224 A CN 202111368224A CN 114201659 A CN114201659 A CN 114201659A
- Authority
- CN
- China
- Prior art keywords
- message
- information
- terminal
- track
- track information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 51
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000004519 manufacturing process Methods 0.000 claims abstract description 40
- 238000003860 storage Methods 0.000 claims abstract description 17
- 238000007639 printing Methods 0.000 claims abstract description 7
- 238000009826 distribution Methods 0.000 abstract description 3
- 238000004806 packaging method and process Methods 0.000 abstract description 3
- 230000002688 persistence Effects 0.000 abstract description 3
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 230000006854 communication Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000013024 troubleshooting Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a message track transmission query method, a device and a system, comprising the following steps: a server receives a network message sent by multiple parties, wherein the network message comprises a transmission message and sending end track information; acquiring the track information of the sending ends according to the network information, and storing the track information of the multiple sending ends into a server log; the method comprises the steps of setting unique identification information for track information of each sending end so as to enable inquiry and/or identification, respectively printing information in a production stage, a consumption stage and a storage stage to a server log and storing the information in the server log, collecting and storing data to the server log by a method of packaging SDK, realizing data disk-dropping persistence, and ensuring reasonable distribution and safe storage of the data.
Description
Technical Field
The present invention relates to the field of network communication technologies, and in particular, to a message trace transmission query method, apparatus, and system.
Background
The message track refers to the consumption processing of a message from a producer to a consumer, the whole process comprises three roles of the producer, a server and the consumer, relevant information is added in a track link in the process of processing the message by each role, and the information is gathered to obtain the current state of any message, so that powerful data support is provided for problem troubleshooting in a production environment
The message tracks adopted in the market at present mainly comprise tracks for tracking message sending and message consumption, namely, logs for recording each processing link of the messages in detail, and mainly solve the following three problems in design:
1. a message track data format is specified;
2. how to record message traces;
3. how to store message trace data;
the information which the server needs to obtain can be specified in a message track data format; for how to record the message track, a Hook mechanism can be adopted, a Hook function is defined through two interfaces, and then the message track can be recorded before and after message sending and message consumption by implementing the two interfaces; in terms of how the message trace data is stored, it is optional to store the message trace data in the Broker server.
At present, most of methods adopted in the market in the aspect of storing message track data are stored in a Broker server, and firstly, the problem to be considered is how to avoid the mixing of the data of the message track and normal service data, if a new Broker server is added, the utilization rate of resources is not high, and the loss of the data of the message track is also prevented.
Disclosure of Invention
In view of the above drawbacks or shortcomings, an object of the present invention is to provide a method, an apparatus, and a system for inquiring message trace transmission.
In order to achieve the above purpose, the technical scheme of the invention is as follows:
a message track transmission query method comprises the following steps:
a server receives a network message sent by multiple parties, wherein the network message comprises a transmission message and sending end track information;
acquiring the track information of the sending ends according to the network information, and storing the track information of the multiple sending ends into a server log;
and setting unique identification information for each transmitting terminal track information for inquiring and/or identifying.
The receiving, by the server, the network message sent by multiple parties specifically includes:
a network message of the producing terminal and/or the consuming terminal is received.
Before sending the track information of the sending end, the production terminal is attached to a Header of a transmission message; the transmitting end track information includes: an IP address of the production terminal, a request date/time, a requested web page, an HTTP code, a number of bytes provided, a user agent, and/or a reference address.
After the consumption terminal receives the network message sent by the production terminal, storing the consumed track information of the sending terminal into a server log; the consumed transmitting end track information comprises: an IP address of the consuming terminal, a consumption time, an HTTP code, a number of bytes provided, a user agent, and/or a reference address.
The storing of the plurality of sending end track information into the server end log comprises:
classifying the transmitting terminal track information in the production stage, the consumption stage and the storage stage according to the production terminal and the consumption terminal;
and respectively printing the classified transmitting end track information into a server log.
The setting of the unique identification information for each transmitting end track information specifically includes:
setting a unique information ID for each transmitting terminal track information;
and sets an index for each information ID for use in searching for the desired information.
The setting of the unique information ID for each piece of sender track information includes: an index is set to the unique information ID using the ELK tool.
A message track transmission inquiry device comprises a receiving unit, a storage unit and an identification unit, wherein the receiving unit is used for receiving a message track;
the receiving unit is used for receiving network messages sent by multiple parties, wherein the network messages comprise transmission messages and sending end track information;
the storage unit is used for acquiring the sending end track information according to the network message and storing a plurality of sending end track information into the server log;
the identification unit is used for setting unique identification information for each transmitting end track information for inquiry and/or identification.
A message track transmission inquiry system comprises a production terminal, a server and a consumption terminal, wherein the production terminal is connected with the server;
the production terminal is used for sending a network message of the production terminal to the server, wherein the network message comprises a transmission message and sending terminal track information;
the consumption terminal is used for sending a network message after the consumption result is delivered to the server, and the network message comprises a transmission message and sending end track information;
the server is used for receiving the network message, acquiring the track information of the sending end according to the network message and storing the track information of the sending ends into a server log; finally, unique identification information is set for each transmitting end track information for inquiry and/or identification.
Before sending the track information of the sending end, the production terminal is attached to a Header of a transmission message; the transmitting end track information includes: an IP address of the production terminal, a request date/time, a requested web page, an HTTP code, a number of bytes provided, a user agent, and/or a reference address; after the consumption terminal receives the network message sent by the production terminal, storing the consumed track information of the sending terminal into a server log; the consumed transmitting end track information comprises: an IP address of the consuming terminal, a consumption time, an HTTP code, a number of bytes provided, a user agent, and/or a reference address.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a message track transmission query method, a device and a system, which respectively print information in a production stage, a consumption stage and a storage stage to a server log and store the information in the server log, collect and store data in the server log by a method of packaging an SDK (software development kit), realize data disk-dropping persistence, ensure reasonable distribution and safe storage of the data, and further, obtain message track data by indexing the server log, thereby saving server resources, ensuring the integrity and higher readability of the data.
Drawings
FIG. 1 is a flow chart of a message trace transmission query method of the present invention;
FIG. 2 is a block diagram of a message trace transmission query device according to the present invention;
FIG. 3 is a block diagram of a message trace transmission query system according to the present invention;
fig. 4 is a transmission diagram of the message track transmission query system of the present invention.
Detailed Description
The present invention will now be described in detail with reference to the drawings, wherein the described embodiments are only some, but not all embodiments of the invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, belong to the scope of the present invention.
The message track refers to the consumption processing of a message from a producer to a consumer, the whole process comprises three roles of the producer, a server and the consumer, relevant information is added in a track link in the process of processing the message by each role, and the information is gathered to obtain the current state of any message, so that powerful data support is provided for problem troubleshooting in a production environment. Most of the methods currently used in the market for storing message trace data are stored in a Broker server. First, the problem to be considered is how to avoid mixing the data of the message track with the normal service data, which would result in low resource utilization rate if a Broker server is added, and also prevent the loss of the message track data. Therefore, the invention collects and stores the data to the service end log and realizes the data disk-dropping persistence by a method of packaging SDK (Software Development Kit) which is generally a Development tool set when some Software engineers build application Software for specific Software packages, Software frameworks, hardware platforms, operating systems and the like), thereby ensuring the reasonable distribution and safe storage of the data.
Example one
As shown in fig. 1, the present invention provides a message trace transmission query method, including:
s1: a server receives a network message sent by multiple parties, wherein the network message comprises a transmission message and sending end track information;
in the present invention, the receiving, by the server, the network message sent by multiple parties specifically includes:
a network message of the producing terminal and/or the consuming terminal is received.
Before sending the track information of the sending end, the production terminal is attached to a Header of a transmission message; the transmitting end track information includes: an IP address of the production terminal, a request date/time, a requested web page, an HTTP code, a number of bytes provided, a user agent, and/or a reference address.
The process specifically attached to the Header of the transmission message specifically includes:
after the consumption terminal receives the network message sent by the production terminal, storing the consumed track information of the sending terminal into a server log; the consumed transmitting end track information comprises: an IP address of the consuming terminal, a consumption time, an HTTP code, a number of bytes provided, a user agent, and/or a reference address.
S2, acquiring the sending end track information according to the network message, and storing a plurality of sending end track information into a server log;
the network messages are mainly information generated in the whole internet communication process of the network, wherein relevant information is added in a track link in the information transmission process, and track information needs to be separated according to requirements.
The storing of the plurality of sending end track information into the server end log comprises:
classifying the transmitting terminal track information in the production stage, the consumption stage and the storage stage according to the production terminal and the consumption terminal;
and respectively printing the classified transmitting end track information into a server log.
It should be noted that the trace information of the sending end includes traces for tracking message sending and message consuming, that is, logs for recording each processing link of the message in detail. The method also comprises a message track data format, a message track recording mode and a message track data storage mode. The information which the server needs to obtain can be specified in a message track data format; for how to record the message track, a Hook mechanism can be adopted, a Hook function is defined through two interfaces, and then the message track can be recorded before and after message sending and message consuming by implementing the two interfaces. Finally, respectively printing the information of the production stage and the storage stage into a server log, and after receiving a consumption result delivery of a Consumer by a Broker, printing consumption related information to a server log; and printing consumption related information to a server log.
In addition, the invention also provides an information track query method:
s3: and setting unique identification information for each transmitting terminal track information for inquiring and/or identifying.
The setting of the unique identification information for each transmitting end track information specifically includes:
setting a unique information ID for each transmitting terminal track information;
and sets an index for each information ID for use in searching for the desired information.
The setting of the unique information ID for each piece of sender track information includes: an index is set to the unique information ID using the ELK tool. ELK (ElasticSearch, Logstash, Kibana) consisting of three open source tools, ElasticSearch, Logstash and Kiabana. The Elasticissearch is an open source distributed search engine and is characterized in that: distributed, zero configuration, automatic discovery, index automatic fragmentation, index copy mechanism, restful style interface, multiple data sources, automatic search load, etc. Logstash is a completely open source tool that he can collect, filter, and store your log for later use (e.g., searching). Kibana is also an open source and free tool, and can provide a log analysis friendly Web interface for Logstash and ElasticSearch, and can help you to gather, analyze and search important data logs. The logs mainly comprise system logs, application program logs and safety logs. The system operation and development personnel can know the software and hardware information of the server through the log, and check errors in the configuration process and the reasons of the errors. The load and the performance safety of the server can be known by frequently analyzing the logs, so that measures can be taken in time to correct errors.
And when receiving a query instruction of the suspected problematic message information, collecting the suspected problematic message information, including the sending time of the MessageID. And establishing a query task according to the existing information, querying the track of the related information, and then viewing the result and analyzing and judging the reason.
Example two
In the present invention, based on the message track transmission query method in the first embodiment, a corresponding message track transmission query device is also provided, as shown in fig. 2, including a receiving unit 1, a storage unit 2, and an identification unit 3, where;
the receiving unit 1 is configured to receive a network message sent by multiple parties, where the network message includes a transmission message and sending end trajectory information;
the storage unit 2 is configured to obtain the sending end trajectory information according to the network message, and store the multiple pieces of sending end trajectory information in the server log;
the identification unit 3 is configured to set unique identification information for each sender trace information for querying and/or identifying.
It should be noted that, the message track transmission query device in the present invention may be a Broker server, may be multiple servers, or may be one server, and it is only necessary to collect and gather the completed logs.
EXAMPLE III
As shown in fig. 3, a message track transmission query system includes a production terminal 4, a server 5, and a consumption terminal 6, wherein;
the production terminal 4 is used for sending a network message of the production terminal to the server, wherein the network message comprises a transmission message and sending terminal track information;
the consumption terminal 5 is used for sending a network message after the consumption result is delivered to the server, wherein the network message comprises a transmission message and sending end track information;
the server 6 is configured to receive a network message, acquire sender track information according to the network message, and store a plurality of sender track information in a server log; finally, unique identification information is set for each transmitting end track information for inquiry and/or identification.
Specifically, the server 6 includes a receiving unit 1, a storage unit 2, and an identification unit 3, where;
the receiving unit 1 is configured to receive a network message sent by multiple parties, where the network message includes a transmission message and sending end trajectory information;
the storage unit 2 is configured to obtain the sending end trajectory information according to the network message, and store the multiple pieces of sending end trajectory information in the server log;
the identification unit 3 is configured to set unique identification information for each sender trace information for querying and/or identifying.
In this embodiment, the production terminal attaches to a Header of a transmission message before sending the trace information of the sending end; the transmitting end track information includes: an IP address of the production terminal, a request date/time, a requested web page, an HTTP code, a number of bytes provided, a user agent, and/or a reference address; after the consumption terminal receives the network message sent by the production terminal, storing the consumed track information of the sending terminal into a server log; the consumed transmitting end track information comprises: an IP address of the consuming terminal, a consumption time, an HTTP code, a number of bytes provided, a user agent, and/or a reference address.
It will be appreciated by those skilled in the art that the above embodiments are merely preferred embodiments of the invention, and thus, modifications and variations may be made in the invention by those skilled in the art, which will embody the principles of the invention and achieve the objects and objectives of the invention while remaining within the scope of the invention.
Claims (10)
1. A message track transmission query method is characterized by comprising the following steps:
a server receives a network message sent by multiple parties, wherein the network message comprises a transmission message and sending end track information;
acquiring the track information of the sending ends according to the network information, and storing the track information of the multiple sending ends into a server log;
and setting unique identification information for each transmitting terminal track information for inquiring and/or identifying.
2. The message track transmission query method according to claim 1, wherein the receiving, by the server, the network message sent by multiple parties specifically comprises:
a network message of the producing terminal and/or the consuming terminal is received.
3. The message track transmission query method according to claim 2, wherein the production terminal attaches to a Header of a transmission message before transmitting the transmitting end track information; the transmitting end track information includes: an IP address of the production terminal, a request date/time, a requested web page, an HTTP code, a number of bytes provided, a user agent, and/or a reference address.
4. The message track transmission query method according to claim 2, wherein the consumption terminal stores consumed sending terminal track information to a server log after receiving a network message sent by a production terminal; the consumed transmitting end track information comprises: an IP address of the consuming terminal, a consumption time, an HTTP code, a number of bytes provided, a user agent, and/or a reference address.
5. The message track transmission query method of claim 2, wherein the storing the plurality of sender track information into the server log further comprises:
classifying the transmitting terminal track information in the production stage, the consumption stage and the storage stage according to the production terminal and the consumption terminal;
and respectively printing the classified transmitting end track information into a server log.
6. The message track transmission query method according to claim 1, wherein the setting of the unique identification information for each piece of sender track information specifically includes:
setting a unique information ID for each transmitting terminal track information;
and sets an index for each information ID for use in searching for the desired information.
7. The message track transmission query method according to claim 6, wherein the setting of the unique information ID for each sending-end track information includes: an index is set to the unique information ID using the ELK tool.
8. A message track transmission inquiry device is characterized by comprising a receiving unit, a storage unit and an identification unit, wherein the receiving unit is used for receiving a message track;
the receiving unit is used for receiving network messages sent by multiple parties, wherein the network messages comprise transmission messages and sending end track information;
the storage unit is used for acquiring the sending end track information according to the network message and storing a plurality of sending end track information into the server log;
the identification unit is used for setting unique identification information for each transmitting end track information for inquiry and/or identification.
9. A message track transmission inquiry system is characterized by comprising a production terminal, a server and a consumption terminal, wherein the production terminal is connected with the server;
the production terminal is used for sending a network message of the production terminal to the server, wherein the network message comprises a transmission message and sending terminal track information;
the consumption terminal is used for sending a network message after the consumption result is delivered to the server, and the network message comprises a transmission message and sending end track information;
the server is used for receiving the network message, acquiring the track information of the sending end according to the network message and storing the track information of the sending ends into a server log; finally, unique identification information is set for each transmitting end track information for inquiry and/or identification.
10. The message track transmission query system according to claim 9, wherein the production terminal attaches to a Header of a transmission message before transmitting the transmitting end track information; the transmitting end track information includes: an IP address of the production terminal, a request date/time, a requested web page, an HTTP code, a number of bytes provided, a user agent, and/or a reference address; after the consumption terminal receives the network message sent by the production terminal, storing the consumed track information of the sending terminal into a server log; the consumed transmitting end track information comprises: an IP address of the consuming terminal, a consumption time, an HTTP code, a number of bytes provided, a user agent, and/or a reference address. .
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111368224.4A CN114201659A (en) | 2021-11-18 | 2021-11-18 | A message track transmission query method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111368224.4A CN114201659A (en) | 2021-11-18 | 2021-11-18 | A message track transmission query method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114201659A true CN114201659A (en) | 2022-03-18 |
Family
ID=80647951
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111368224.4A Pending CN114201659A (en) | 2021-11-18 | 2021-11-18 | A message track transmission query method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114201659A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115396530A (en) * | 2022-07-21 | 2022-11-25 | 拉卡拉支付股份有限公司 | Message management method, system, electronic device, storage medium, and program product |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109687994A (en) * | 2018-11-30 | 2019-04-26 | 北京奇艺世纪科技有限公司 | A kind of message treatment method, system, device and equipment |
| CN109739657A (en) * | 2018-11-30 | 2019-05-10 | 北京奇艺世纪科技有限公司 | A kind of message treatment method, system, device and equipment |
| CN110362721A (en) * | 2018-04-08 | 2019-10-22 | 阿里巴巴集团控股有限公司 | Processing method, system, device and the electronic equipment of message traces information |
-
2021
- 2021-11-18 CN CN202111368224.4A patent/CN114201659A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110362721A (en) * | 2018-04-08 | 2019-10-22 | 阿里巴巴集团控股有限公司 | Processing method, system, device and the electronic equipment of message traces information |
| CN109687994A (en) * | 2018-11-30 | 2019-04-26 | 北京奇艺世纪科技有限公司 | A kind of message treatment method, system, device and equipment |
| CN109739657A (en) * | 2018-11-30 | 2019-05-10 | 北京奇艺世纪科技有限公司 | A kind of message treatment method, system, device and equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115396530A (en) * | 2022-07-21 | 2022-11-25 | 拉卡拉支付股份有限公司 | Message management method, system, electronic device, storage medium, and program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112765103B (en) | File analysis method, system, device and equipment | |
| CN105159964A (en) | Log monitoring method and system | |
| CN109783330B (en) | Log processing method, log display method, and related device and system | |
| CN111241049B (en) | Distributed operation log realization system based on micro-service architecture | |
| CN110858192A (en) | Log query method and system, log checking system and query terminal | |
| CN108228322B (en) | Distributed link tracking and analyzing method, server and global scheduler | |
| CN111177094B (en) | Log data processing method and device, electronic equipment and storage medium | |
| CN108228432A (en) | A kind of distributed link tracking, analysis method and server, global scheduler | |
| CN117194156A (en) | Unified monitoring operation and maintenance management method and system for multi-cloud platform | |
| CN114780335A (en) | Correlation method, apparatus, computer equipment and storage medium for monitoring data | |
| US7664848B2 (en) | System and method for monitoring networked devices employing RSS functionality | |
| CN110888985A (en) | Alarm information processing method and device, electronic equipment and storage medium | |
| CN101667932A (en) | Method of network element equipment log management and device | |
| CN114201659A (en) | A message track transmission query method, device and system | |
| CN112579406A (en) | Log call chain generation method and device | |
| CN107820270B (en) | GPRS interface monitoring system based on GSM-R network | |
| CN103297477A (en) | Data collecting and reporting system, data processing method and proxy server | |
| WO2025123878A1 (en) | Data synchronization verification method and apparatus, and electronic device | |
| CN113381907A (en) | Log collection method and device, electronic equipment and storage medium | |
| CN112181929A (en) | Cloud management platform log processing method and device, electronic device and storage medium | |
| CN117632445A (en) | Request processing method and device, task execution method and device | |
| CN110399357A (en) | A method, device and system for centralized management of big data component logs | |
| WO2011137678A1 (en) | Method and system for processing multi-user parallel signaling tracking in client | |
| CN101764707A (en) | Processing method, conversion device and processing system for network configuration event notification message | |
| CN100365617C (en) | A usage statistics device and usage statistics method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |