CN114244908B

CN114244908B - RTSP stream media transmission method crossing network domain

Info

Publication number: CN114244908B
Application number: CN202111304071.7A
Authority: CN
Inventors: 汪月林; 谭彰; 陈吉平; 关威
Original assignee: Zhejiang Lanzhuo Industrial Internet Information Technology Co ltd
Current assignee: Zhejiang Lanzhuo Industrial Internet Information Technology Co ltd
Priority date: 2021-11-05
Filing date: 2021-11-05
Publication date: 2024-08-23
Anticipated expiration: 2041-11-05
Also published as: CN114244908A

Abstract

The invention provides a method for transmitting RTSP streaming media of a cross-network domain, which comprises the following steps: the RTSP request of the client is acquired and forwarded to the media source proxy through the command transfer server; replacing a port of a client in the RTSP protocol with a port of a media source proxy based on a first mapping relation, and sending an RTSP request to a media source through the replaced RTSP protocol; acquiring a response of a media source to an RTSP request, and acquiring streaming media information in an RTSP protocol at the moment when the response is sent to a media source proxy; replacing the port of the media source in the RTSP protocol with the port of the client agent based on the second mapping relation, and restoring the port of the media source agent in the RTSP protocol into the port of the client based on the first mapping relation; and sending the streaming media information to the client through the client agent by using the replaced RTSP protocol. The invention provides a method for cross-domain direct access for media sources and clients in RTSP protocol, and reduces bandwidth occupation.

Description

RTSP stream media transmission method crossing network domain

Technical Field

The invention belongs to the technical field of streaming media transmission, and particularly relates to a cross-network RTSP streaming media transmission method.

Background

With the development of society, further improvement of industrial intelligence is more and more important in intelligent analysis and mining of various streaming media data, and most of the streaming media data come from physical devices such as cameras and microphones. In order to more efficiently and timely transmit streaming media data for later intelligent analysis, a real-time streaming protocol is generally adopted to realize the transmission of the streaming media data.

The real-time streaming Protocol (REAL TIME STREAMING Protocol, RTSP) is an application layer Protocol in the TCP/IP Protocol architecture that defines how efficiently a one-to-many application program transmits multimedia data over an IP network. RTSP is architecturally located above RTP and RTCP, and uses UDP or TCP to complete data transmission.

However, there are drawbacks in the multi-domain environment of the industrial enterprise, because the media source and the client of the industrial enterprise are mostly located in different domains, the servers of the different domains cannot directly realize the transmission of the streaming media information based on the RTSP protocol. Therefore, the existing RTSP streaming media forwarding mode mostly cannot realize cross-domain forwarding, or a multi-level proxy needs to be set when the cross-domain forwarding is needed, and the forwarding of the multi-level proxy occupies more bandwidth, and the deployment difficulty of the multi-level proxy is higher, so that the multi-level proxy is inconvenient to expand.

Disclosure of Invention

In order to solve the defects and shortcomings in the prior art, the invention provides a method for transmitting RTSP streaming media of a cross-network domain, which comprises the following steps:

the RTSP request of the client is acquired and forwarded to the media source proxy through the command transfer server;

Determining a first mapping relation between a port of a media source proxy and a port of a client, replacing the port of the client in an RTSP protocol with the port of the media source proxy based on the first mapping relation, and sending an RTSP request to a media source through the replaced RTSP protocol;

Acquiring a response of a media source to an RTSP request, and acquiring streaming media information in an RTSP protocol at the moment when the response is sent to a media source proxy;

Determining a second mapping relation between a port of a media source and a port of a client agent, replacing the port of the media source in the RTSP protocol with the port of the client agent based on the second mapping relation, and restoring the port of the media source agent in the RTSP protocol to the port of the client based on the first mapping relation;

And sending the streaming media information to the client through the client agent by using the replaced RTSP protocol.

Optionally, the determining the first mapping relationship between the port of the media source agent and the port of the client includes:

When the media source agent receives the RTSP request, determining an RTP/RTCP port pair opened by a client in the current RTSP protocol, opening a first local UDP port pair of the media source agent, and establishing a first mapping relation between the RTP/RTCP port pair and the first local UDP port pair.

Optionally, when determining the first mapping relationship between the port of the media source proxy and the port of the client, the media source proxy opens a p2p session of ice, and forwards address information of the p2p session to the client proxy through the command forwarding server.

Optionally, the replacing the port of the client in the RTSP protocol with the port of the media source proxy based on the first mapping relationship sends the RTSP request to the media source through the replaced RTSP protocol, including:

Based on the first mapping relation, replacing an RTP/RTCP port pair opened by a client in an RTSP protocol with a first local UDP port pair used for transmitting streaming media data;

analyzing the RTSP request based on an address list pre-stored in a media source agent;

and based on the replaced RTSP, sending the analysis result of the RTSP request to the media source.

Optionally, the generating process of the address list includes:

The media source agent acquires an RTSP address list of a media source in a local network, wherein the RTSP address list contains RTSP url of the media source;

encoding each media source to generate a locally unique identifier;

and storing the corresponding relation between the local unique identification of the same media source and RTSP url into an address list, and uploading the address list to the command transfer server through a TCP protocol between the media source proxy and the command transfer server.

Optionally, the parsing the RTSP request based on an address list pre-stored in the media source proxy includes:

The media source proxy analyzes the local unique identifier from the original url of the RTSP request, finds the RTSP url corresponding to the local unique identifier in a pre-stored address list, and takes the RTSP url as an analysis result.

Optionally, the determining the second mapping relationship between the port of the media source and the port of the client agent includes:

When the client agent receives the return of the RTSP request, determining an RTP/RTCP port pair of which the media source is open in the current RTSP protocol, opening a second local UDP port pair of the client agent, and establishing a second mapping relation between the RTP/RTCP port pair and the second local UDP port pair.

Optionally, when determining the second mapping relationship between the port of the media source and the port of the client agent, the client agent opens a p2p session of ice, and forwards address information of the p2p session to the media source agent through the command forwarding server.

Optionally, the replacing the port of the media source in the RTSP protocol with the port of the client agent based on the second mapping relationship includes:

Based on the second mapping relation, replacing the RTP/RTCP port pair opened by the media source in the RTSP protocol with a second local UDP port pair used for transmitting stream media data.

Optionally, the restoring the port of the media source proxy in the RTSP protocol to the port of the client based on the first mapping relationship includes:

and based on the first mapping relation, restoring the first local UDP port pair in the RTSP protocol into an RTP/RTCP port pair opened by the client.

The technical scheme provided by the invention has the beneficial effects that:

(1) The invention provides a method for directly accessing media source and client in RTSP protocol across network domain, which makes RTSP transmission between media source and client unnecessary to do any conversion work of streaming media protocol, only needs to replace ports between client and media source agent and between media source and client agent, and can achieve transmission effect equivalent to local direct current, as long as media source and client conforming to standard RTSP protocol can be accessed according to the method provided by the invention, thereby greatly improving the convenience of access.

(2) The client agent and the media source agent are used as intermediate nodes, even if the media sources of the streaming media are scattered in a plurality of different private networks, unified management can be carried out through the client agent, the media source agent and the command transfer server without setting a multi-level proxy service, so that excessive bandwidth is occupied, and the deployment difficulty of the proxy server is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a method for transmitting RTSP streaming media across network domains according to an embodiment of the present invention;

Fig. 2 is a schematic diagram of a proposed cross-domain RTSP transmission system according to an embodiment;

Fig. 3 is a url replacement schematic of an RTSP request.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein.

It should be understood that, in various embodiments of the present invention, the sequence number of each process does not mean that the execution sequence of each process should be determined by its functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

It should be understood that in the present invention, "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements that are expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that in the present invention, "plurality" means two or more. "and/or" is merely an association relationship describing an association object, and means that three relationships may exist, for example, and/or B may mean: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. "comprising A, B and C", "comprising A, B, C" means that all three of A, B, C are comprised, "comprising A, B or C" means that one of A, B, C is comprised, "comprising A, B and/or C" means that any 1 or any 2 or 3 of A, B, C are comprised.

It should be understood that in the present invention, "B corresponding to a", "a corresponding to B", or "B corresponding to a" means that B is associated with a, from which B can be determined. Determining B from a does not mean determining B from a alone, but may also determine B from a and/or other information. The matching of A and B is that the similarity of A and B is larger than or equal to a preset threshold value.

As used herein, the term "if" may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to a detection", depending on the context.

The technical scheme of the invention is described in detail below by specific examples. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.

Example 1

As shown in fig. 1, this embodiment proposes a method for transmitting RTSP streaming media across network domains, where the method for transmitting RTSP streaming media involves a client and a client proxy in a first network domain, and a media source proxy in a second network domain, and the method includes:

S1: the RTSP request of the client is acquired and forwarded to the media source proxy through the command transfer server;

S2: determining a first mapping relation between a port of a media source proxy and a port of a client, replacing the port of the client in an RTSP protocol with the port of the media source proxy based on the first mapping relation, and sending an RTSP request to a media source through the replaced RTSP protocol;

s3: acquiring a response of a media source to an RTSP request, and acquiring streaming media information in an RTSP protocol at the moment when the response is sent to a media source proxy;

S4: determining a second mapping relation between a port of a media source and a port of a client agent, replacing the port of the media source in the RTSP protocol with the port of the client agent based on the second mapping relation, and restoring the port of the media source agent in the RTSP protocol to the port of the client based on the first mapping relation;

s5: and sending the streaming media information to the client through the client agent by using the replaced RTSP protocol.

The embodiment provides a method for cross-domain direct access for media sources and clients in an RTSP protocol, so that the RTSP transmission between the media sources and the clients does not need to do any conversion work of a streaming media protocol, the transmission effect equivalent to local direct current is achieved, and only the media sources and the clients conforming to the standard RTSP protocol can be accessed according to the method provided by the invention, thereby greatly improving the convenience of access.

In this embodiment, the client and the media source are respectively located in different network domains, and a specific cross-network RTSP transmission system is shown in fig. 2, and includes the following modules:

a. the client (RTSP CLIENT, RC), in this embodiment, a client supporting RTSP protocol, which is self-developed or from a third party, is in an independent Network C;

b. A media source (RTSP SERVER, RS), any hardware device or server that can provide RTSP protocol access, in another independent Network domain Network B that is different from the RC;

c. The client agent (RTSP CLIENT AGENT, RCA) is deployed in a Network C where the RC is located to simulate the function of an RTSP media source server, and the RC can directly access the RCA through an RTSP protocol, namely, the RC can realize the transmission of RTP/RTCP protocol through respective local UDP ports;

d. the media source agent (RTSP SERVER AGENT, RSA) is deployed in a Network B where the RS is located, simulates the function of an RTSP client, and RSA can directly access RTSP service of the RS through an RTSP protocol, namely, the RTSP client can realize transmission of RTP/RTCP protocol through respective local UDP ports;

e. Command transfer service (Command TRANSFER SERVICE, CTS) may be deployed in any Network domain different from RC and RS, and in this embodiment, the deployment is in Network a, but it is necessary to ensure that RCA and RSA can connect to the TCP service of CTS through which Command data is transferred between RCA and RSA.

The foregoing description is merely a description of functional modules, and does not refer to a specific implementation, for example, RSA and RS may be different sub-modules in the same process, or different processes in the same physical device, or run on different physical devices.

In this embodiment, the RCA and RSA respectively establish a command channel of the TCP long connection with the CTS, and the command protocol may use any protocol, such as HTTP protocol or protobuf private protocol. The CTS uniquely identifies each attached RCA and RSA, namely RCAID for RCA and RSAID for RSAS. RCA and RSA and CTS need to send heartbeat commands at regular intervals to maintain the whole login cycle. Meanwhile, p2p communication is established between RCA and RSA through a ice framework, so that the transmission of streaming media information is realized. The RC starts the playing flow according to the standard RTSP protocol, and the request address is the local service address of the RCA. All RTSP packets are packaged by RCA command protocol and then forwarded to CTS, then forwarded to RSA by CTS, finally RSA analyzes the command protocol to obtain real RTSP protocol, and initiates RTSP request to real media source. The RTSP protocol returned by the media source also transmits RSA- > CTS- > RCA- > RC in reverse in the same manner.

In the process of maintaining login period between RCA and RSA and CTS, RSA obtains RTSP address of media source in local network, i.e. RTSP url of media source by means of onvif searching or manual configuration. And then each RS is encoded to generate a Local unique identifier (LSID) of the unique identifier. The RSA stores the corresponding relation between the local unique identifier of the RS and the RTSP url into an address list, on one hand, the address list is stored in the RSA, and the address list pre-stored in the RSA is as follows:

{

“LSID1”:”rtsp://192.168.1.1:554/stream1”,

“LSID1”:”rtsp://192.168.1.1:554/stream2”,

“LSID1”:”rtsp://192.168.1.1:554/stream3”,

……

}；

On the other hand, the address list is synchronously uploaded to the command transfer server through a TCP protocol between RSA and CTS, and in the future RC can inquire all media sources globally through CTS, and the global media source address list stored by CTS is as follows:

Where "RSAID1", "RSAID2", "RSAID" represent the IDs of the different RSAs, respectively, followed by the LSIDs of the media sources managed under each RSA.

When the RC needs to acquire the streaming media information of a certain media source, the RC starts a playing flow according to a standard RTSP protocol, so that an RTSP request is generated, the format of url of the RTSP request in the embodiment is' RTSP:// [ RCAIP ]: [ RCA local port ]/[ RSAID ]/[ LSID ] ", and RCA corresponding to RCAIP analyzes RSAID and LSID through url and creates a context for the session.

In this embodiment, the RS returns a REDIRECT message for which RSA needs to analyze the new url to automatically create a new LSID and upload the CTS, while updating the path in the REDIRECT message url to [ RSAID ]/[ LSID ]. When the RCA receives the REDIRECT message, the ip and port of the url in the message need to be replaced as the local address, and finally the RC ends the old session TEARDOWN and connects the new url.

In this embodiment, an RTSP request of a client is acquired, and the RTSP request is forwarded to a media source proxy through a command forwarding server, specifically: the RTSP request is sent by the RC, sent to the RCA through the local network, and then sent to the corresponding RSA RSAID through the CTS by the TCP protocol.

At this time, a first mapping relation between a port of the media source proxy and a port of the client is determined, an open RTP/RTCP port pair of the client in the current RTSP protocol is determined, a first local UDP port pair of the media source proxy is opened, and a first mapping relation between the RTP/RTCP port pair and the first local UDP port pair is established. The purpose of determining the first mapping relation is to enable the RS in Network B to communicate with the RTP/RTCP port pair of the RC in Network C and transmit media data in the future, and finally to save the first mapping relation locally on the media source proxy.

In this embodiment, when determining the first mapping relationship between the port of the media source proxy and the port of the client, the media source proxy opens the p2p session of ice, and forwards the address information of the p2p session to the client proxy through the command forwarding server. The description format of the ice protocol generally adopts a standard session description protocol (Session Description Protocol, SDP), which is a description standard of information format. The local address is acquired through the network card information, the external address is acquired through the STUN server or the TURN server, and a person skilled in the art should know how to implement p2p communication based on the ice protocol, which is not described here.

The embodiment uses the first mapping relation to replace the RTP/RTCP port pair which is related to the RTSP protocol and is opened by the client by the first local UDP port pair, so that RTSP requests of RC can be received by RSs of different network domains, specifically:

Based on the first mapping relation, replacing the RTP/RTCP port pair opened by the client in the RTSP protocol with a first local UDP port pair, in the RTSP protocol, indicating the streaming media to be transmitted in a way through a SETUP request, and when the SETUP request is sent, the client designates two ports, one port is used for transmitting RTP data and the other port is used for transmitting RTCP data, wherein the SETUP operation must be executed before each streaming media is played. In this embodiment, after the SETUP request of the RTSP request reaches the RSA, the RSA parses the client_port=port1-port2 of the Transport in the RTSP protocol header, where port1-port2 represents a pair of RTP-RTCP ports of the client, and client_port represents a port pair that is opened by the RC before the SETUP request is initiated and is used for receiving streaming media data in the future. At this time, RSA opens two other local UDP ports, namely the first local UDP port to replace port1 and port2. Because the RS at the Network B can identify the UDP port of the RSA, the RS can identify the replaced RTSP protocol, and further forward the replaced RTSP request to a media source corresponding to the RTSP url based on the replaced RTSP protocol.

And analyzing the RTSP request based on an address list pre-stored in the media source proxy, and transmitting an analysis result of the RTSP request to the media source based on the replaced RTSP protocol. For example, RSA finds the RTSP url of the RS based on the locally saved address list, replaces the original url of the RTSP request with the RTSP url, e.g., as shown in FIG. 3, the original url of the RTSP request from RCA is "RTSP://172.16.1.0:554/RSAID/LSID1", where "172.16.1.0" is the local ip of RCA, "554" is the local port of RCA, RSAID is the ID allocated by CTS for RSA, and LSID1 is the ID allocated by RSA for RS in the local network. By looking up the address list, RSAID corresponds to a plurality of LSIDs shown in fig. 3, and the original url of the RTSP request is replaced by a specific url of LSID1, it is known that the url of the RTSP request after replacement is "RTSP://192.168.1.1:554/Channels/101transportmode =unicast".

And then, based on the replaced RTSP protocol, sending an analysis result of the RTSP request to a media source through a local network, sending the RTSP request to an RS corresponding to 'RTSP:// 192.168.1.1:554/Channels/101transportmode =unicast', and acquiring streaming media information of the RS. Because the RS and the RSA are in the Network B, the streaming media information can be directly sent to the RSA through the local Network, and the SETUP return flow is entered at the moment.

When the SETUP returns, first, a second mapping relationship of the port between the media source and the client agent is determined, and the same as the determination of the first mapping relationship is specifically: when the client agent receives the return of the RTSP request, namely, the SETUP returns, determining an RTP/RTCP port pair of which the media source is open in the current RTSP protocol, opening a second local UDP port pair of the client agent, and establishing a second mapping relation between the RTP/RTCP port pair and the second local UDP port pair; and locally saving the second mapping relation on the client agent.

And the same as the SETUP request, when determining the second mapping relation between the port of the media source and the port of the client agent, the client agent starts the p2p session of the ice and forwards the address information of the p2p session to the media source agent through the command transfer server. Thus, a p2p session between RSA and RCA is established for transmitting streaming media information.

Since the RTP/RTCP port pair of the RC in the RTSP protocol has been replaced with the first local UDP port pair for the RS to transfer media data with the RSA in the SETUP request phase, the replaced client_port needs to be restored when the SETUP returns, that is, the first local UDP port pair in the RTSP protocol is restored to the RTP/RTCP port pair opened by the client based on the first mapping relationship, so that the first local UDP port pair in the RTSP protocol is restored to the client_port at this time, so that the RC can receive the media data. Subsequently, for the RCA to transfer media data with the RC in future, based on the second mapping relation, the RTP/RTCP port pair opened by the media source in the RTSP protocol is replaced with the second local UDP port pair, for example, when the SETUP of the RTSP request returns, the RCA parses the server_port=port3-port4 of the Transport in the RTSP header, and the port3-port4 represents a pair of RTP-RTCP ports representing the media source, and the server_port represents a port pair opened by the RS for transferring streaming media data in future. At this point the RCA opens two more local UDP ports, the second local UDP port, to replace port3 and port4. Because the RC at the Network C can identify the UDP port of the RCA, the RC can identify the replaced RTSP protocol, and then based on the replaced RTSP protocol, stream media information of the media source responding to the RTSP request is sequentially sent to the client through the media source agent and the client agent.

Therefore, on one hand, the embodiment unifies the addresses of a plurality of media sources through the pre-established address list, namely generates the LSID of the unique identifier, so that the RC can acquire the streaming media information of different media sources across network domains. On the other hand, the RTSP protocol is processed through the first mapping relation and the second mapping relation, so that RC and RS among a plurality of different network domains can carry out streaming media transmission based on the RTSP protocol.

In this embodiment, for security of RTSP transmission, when the client agent obtains an RTSP request of the client, it is further required to authenticate the RTSP request according to the Authorization Header information in the Header of the RTSP request, and when the Authorization Header information defines an authentication mode as Basic authentication, the authentication is completed by performing Base64 encoding on the username field and the password field in the RTSP request, and when the Authorization Header information defines an authentication mode as Digest authentication, the authentication is completed by resetting the username field and the response Header field according to a preset authentication rule of RTSP.

Meanwhile, when the RTSP protocol returns, CSeq and Session in the Header need to be recorded for subsequent Session, for example, when the RCA is abnormally disconnected, RSA may attach CSeq and Session to perform sending TearDown to close the Session.

The various numbers in the above embodiments are for illustration only and do not represent the order of assembly or use of the various components.

The foregoing is illustrative of the present invention and is not to be construed as limiting thereof, but rather, the present invention is to be construed as limited to the appended claims.

Claims

1. The method for transmitting the RTSP streaming media across the network domain is characterized by comprising the following steps of:

Transmitting the streaming media information to the client through the client proxy by using the replaced RTSP protocol;

the determining a first mapping relationship between the port of the media source agent and the port of the client comprises:

when a media source agent receives an RTSP request, determining an RTP/RTCP port pair opened by a client in a current RTSP protocol, opening a first local UDP port pair of the media source agent, and establishing a first mapping relation between the RTP/RTCP port pair and the first local UDP port pair;

the determining the second mapping relationship between the port of the media source and the port of the client agent includes:

When the client agent receives the return of the RTSP request, determining an RTP/RTCP port pair of which the media source is open in the current RTSP protocol, opening a second local UDP port pair of the client agent, and establishing a second mapping relation between the RTP/RTCP port pair and the second local UDP port pair;

When a first mapping relation between a port of a media source agent and a port of a client is determined, the media source agent starts a p2p session of ice, and address information of the p2p session is forwarded to the client agent through a command transfer server;

Based on information definition authentication mode, request authentication is carried out on the RTSP request acquired by the client agent;

In the process that a client agent, a media source agent and a command transfer service maintain a login period, the media source agent acquires an RTSP address of a local media source, codes the RTSP address to generate a unique media source identifier, stores a corresponding relation between the unique media source identifier and the RTSP address of the media source to an address list of the media source agent, and synchronously uploads the address list to the command transfer service end in real time based on a TCP protocol between the media source agent and the command transfer service.

2. The method for transmitting RTSP streaming media across network domains according to claim 1, wherein the replacing the port of the client in the RTSP protocol with the port of the media source proxy based on the first mapping relation, and transmitting the RTSP request to the media source through the replaced RTSP protocol, includes:

3. The method for transmitting RTSP streaming media across network domain according to claim 2, wherein the parsing the RTSP request based on the address list pre-stored in the media source proxy comprises:

4. The method for transmitting RTSP streaming media across network domains according to claim 1, wherein when determining the second mapping relationship between the port of the media source and the port of the client proxy, the client proxy opens the p2p session of ice and forwards the address information of the p2p session to the media source proxy through the command forwarding server.