CN1750651A - A multimedia monitoring system - Google Patents

Abstract

This invention discloses multimedia monitor system, which contains client side with account information in plurality of area and having monitor terminal and monitored terminal, positioning server set in internet for receiving and storing account and address information transmitted by client side, register server set in internet for client side logging, registering and certificating according to positioning server, proxy server set in internet for receiving client side request and communicating among client sides, and among client side, register server and positioning server. Comparing with current technology, said invention has advantages of strong client side mobility, convenient use and fine system extensibility.

Description

A multimedia monitoring system

technical field

The invention relates to a multimedia monitoring system based on image and video monitoring.

technical background

With the development of multimedia technology, multimedia monitoring system is more and more used in the fields of safety operation, public security prevention, static monitoring and so on, and plays an important role. At present, real-time monitoring systems through the transmission of video or audio have been implemented in most units and places, such as in some office buildings with corresponding monitoring systems installed, or some living quarters with good safety management systems, or even some private Inside the house, etc.

The current monitoring system is mainly divided into analog monitoring and digital monitoring. Because analog monitoring uses analog signals, it has line exclusiveness, high bandwidth occupation, point-to-point technology, short transmission distance, poor manageability, poor image quality, no intelligence, small application occasions, narrow range, and locality Strong and other disadvantages, so it is gradually being replaced by digital monitoring.

The characteristics of digital monitoring are: the use of digital coding, greatly reducing the occupied bandwidth, high image quality, reduced storage space, monitoring information can be sent through the network, strong real-time performance, and easy to use.

Most of the current digital monitoring systems use video encoding card plus PC, using C/S (client and server) structure or B/S (browser and server) structure. Poor performance, poor management, due to the use of encoding cards, high cost, limited application occasions, etc. As shown in Figure 1.

In the solution disclosed in Chinese patent application CN00101634, the remote monitoring terminal can obtain the monitoring information and alarm information of the monitoring area through the network through the browser used by it, and display it on the monitoring terminal, as shown in FIG. 2 .

Chinese patent application CN02129616.2 discloses a warning type multimedia automatic monitoring system based on Ethernet, which manages the network cameras in the receiving module through a management center, and the monitoring terminal users use computers, mobile phones, etc. in any In other places, access the web server through the Internet to access the IP address of the management center server, send request information to the server and receive the requested multimedia information, and display the multimedia information, such as images, sounds, motion pictures, etc. on the monitor , that is to realize remote monitoring with unlimited distance, as shown in Figure 3.

However, the current network monitoring system has the following problems:

The first problem is that there are relatively few monitoring points, and most of them are limited to the local area network. The scalability and management are poor. It can only be deployed in a small area, and cannot be used in a large area, a large number of users at the same time, and cannot solve the problem of managing a large number of terminals.

The second problem is that the current monitoring system is mainly developed for specific applications and is not universal, so it does not have issues such as grouping users, management of rights, and confidentiality of content; as a general , a monitoring system platform that can be used on a large scale and can be operated commercially needs to provide different services to different users; the privacy of users must also be considered, that is, each user can only monitor their own corresponding monitored points, and Other monitored points cannot be monitored.

The third problem is that in the existing monitoring system, the monitored terminal can only be fixed in a certain place and cannot be moved frequently or in real time. The main reason is that its address cannot be changed dynamically, or there is no function to support mobility. With the continuous development of the development and the continuous expansion of the application field of monitoring, the monitoring system is required to have great flexibility, such as the monitored and monitored points are constantly changing, even moving in real time.

The fourth problem is that the cameras at the monitoring points cannot form an effective interaction with the management server. The cameras at the monitoring points can only be passively searched, and cannot transmit some of their own information to the outside world.

The fifth problem is that the current monitoring system cannot realize the cross-gateway function. The camera at the monitoring point must have a public network IP address. If it has an internal network IP address, external users will not be able to access it. .

The sixth problem is that the whole system cannot be easily upgraded.

Contents of the invention

The purpose of the present invention is to provide a multimedia monitoring system with reasonable structure, strong functionality and strong expandability for the deficiencies of the prior art.

In order to solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a multimedia monitoring system, which includes a client, a registration server, a positioning server and a proxy server, wherein:

The client is set in multiple domains, including a monitored terminal and a monitoring terminal, wherein the monitored terminal is used to collect images in its monitored area, and the monitoring terminal is used to display the images sent by the monitored terminal to For user monitoring, each client has its account information; the positioning server is set in the Internet, which receives and stores the account information and address information sent by the client;

The registration server is set in the Internet, and is used to register, log in and authenticate the client according to the positioning server;

The proxy server is set in the Internet, and it receives the request sent by the client, so as to realize the communication between the clients and between the client and the registration server and the location server according to the request content and the address information provided by the registration server and the location server .

Preferably, the proxy server can receive requests from clients in various domains, the location server stores account information and address information of clients in all domains, and the proxy server receives requests from clients to call other clients , according to the account information of other clients included in the request, query the corresponding address information from the positioning server, so as to forward the call request to the called client.

Preferably, a proxy server, a registration server and a location server may be set in each of the domains, and the proxy server of each domain receives the request of the client in this domain and the information forwarded by the proxy servers of other domains; the location server of each domain stores The account information and address information of the client in the domain; the registration server of each domain registers, logs in and authenticates the client in the domain according to the location server in the domain; the proxy server receives a client calling another client After the call request from the terminal, according to the account information of the called client included in the request, it will inquire whether the called client belongs to the local domain from the positioning server, and if so, obtain its address information, thereby forwarding the call request to The called client; if not, forward the call request to a proxy server in another domain.

Further, the positioning server can also be used to store the system information of the client, the client uses the account information to send a registration and login authentication request to the registration server through the proxy server based on the Internet, and the registration server queries the positioning server, so as to The stored account information authenticates it, and updates the current address information, account information and system information of the client after the authentication is passed.

Further, the proxy server can also be used for client grouping and authority management. The location server stores client grouping and authority management information. After receiving the request from the client, the proxy server will first The terminal account information queries the grouping and authority management information of the client from the positioning server, so as to determine the grouping status and authority scope of the client.

Further, a redirection server may also be included, which is set in the Internet and is used to determine the location information of proxy servers in all domains, and send a redirection message to the client.

Further, a database may also be included, which is set in the Internet and used to store account information of clients in various domains and provide it to the redirection server for query.

Preferably, the monitored terminal may include a camera module, an image processing module, a control unit, a storage module and a network interface, the account information of the monitored terminal is stored in the storage module, and the control unit is provided with a client The application program is used to realize that the monitored terminal automatically sends a registration and login authentication request to the registration server, and sends its address information to the positioning server at regular intervals.

Further, a media server may also be included, which is used to receive and store the media stream and data from the monitored terminal, and at the same time provide the monitoring terminal with the query of the stored information.

Further, a cross-gateway server may also be included, which is used to realize the data communication across the gateway by the client located in the gateway.

Preferably, the monitored terminal may be a network camera.

Further, the account information may include an account name and a password, and the account name adopts a structure of username@domain name, and the address information includes an IP address and a port.

Compared with the prior art, the present invention has the following advantages: due to the adoption of the above technical scheme, the present invention effectively manages the client through the domain due to the adoption of the domain concept, and manages and manages the account information and address information of the client through the registration and location server. update, so as to ensure that the proxy server can smoothly route the signaling and messages between intra-domain and inter-domain clients. Therefore, compared with the prior art, the system client provided by the present invention has strong mobility, convenient and flexible use, and good system scalability , The monitoring field is easy to expand. Furthermore, the present invention realizes the cross-gateway communication of the client through the cross-gateway server, thereby breaking the limitation of the network environment on the system application.

Description of drawings

Accompanying drawing 1 is the first kind of monitoring system structural diagram of prior art;

Accompanying drawing 2 is the structural block diagram of the monitoring system of the second prior art;

Accompanying drawing 3 is the structural block diagram of the monitoring system of the third prior art;

Accompanying drawing 4 is the system block diagram of embodiment one of multimedia monitoring system of the present invention;

Accompanying drawing 5 is the system principle block diagram of embodiment two of multimedia monitoring system of the present invention;

Accompanying drawing 6 is the system principle block diagram of embodiment three of the multimedia monitoring system of the present invention;

Accompanying drawing 7 is the system block diagram of embodiment four of multimedia monitoring system of the present invention;

Figure 8 is a functional block diagram of the cross-gateway processing process in Embodiment 4.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

Embodiment one

Referring to Figure 4, the multimedia monitoring system provided in this embodiment is composed of a proxy server, a registration server and a location server set in the Internet, and clients set in multiple domains.

The client includes a monitoring terminal and a monitored terminal. The monitored terminal is used to collect images in its monitored area, and the monitored terminal is used to display the images sent by the monitored terminal for user monitoring. Each client has its account information, which can be applied by the user to the operator in advance, or defaulted by the client system.

The monitoring terminal can be any computer or handheld device that can be used by the user to access the Internet, such as a mobile phone or a handheld computer. In other words, the user can connect any computer or handheld device that is connected to the Internet to As a monitoring terminal to monitor the area you need to monitor.

The monitored terminal is a camera device that can access the Internet, such as a network camera (IPCamera), and the camera device is moved anywhere that the user needs to monitor. As long as the camera device can access the Internet through a network interface, the user can pass The monitoring terminal monitors the area monitored by the camera device. As an application example, the network camera may include a camera module, an image processing module, a control unit, a storage module and a network interface, the account information of which is stored in the storage module, and the control unit is provided with a client application program, It is used to realize that the monitored terminal automatically sends a registration and login authentication request to the registration server, and regularly sends its address information to the positioning server. In addition, the same client application program can also be installed in the computer used as the monitoring terminal.

The positioning server is set in the Internet, and a database for storing account information and address information of clients in all domains is set inside it, and can be queried by the proxy server and the positioning server.

The registration server is set in the Internet, and is used for registering, logging in and authenticating the client according to the positioning server.

In addition, the positioning server is also used to store system information of the client.

The client uses the account information to send a registration and login authentication request to the registration server through the proxy server based on the Internet, and the registration queries the positioning server, thereby authenticating the client according to the stored account information, and updating after the authentication is passed. The client's current address information, account information, and system information.

The proxy server is set in the Internet to receive requests from clients in various domains. After the proxy server receives the request from the client to call other clients, it will, according to the account information of other clients included in the request, from the The positioning server queries the corresponding address information, so as to forward the call request to the called client.

In addition, the proxy server is also used for client grouping and authority management. The positioning server stores client grouping and authority management information. After receiving the request from the client, the proxy server will first The information queries the grouping and authority management information of the client from the positioning server, so as to determine the grouping status and authority range of the client.

In order to clearly illustrate the system structure principle and work flow of this embodiment, the following will describe in detail according to application examples.

First, it is assumed that this embodiment includes two domains, as shown in FIG. 4 , where the domain name of domain A is tom.com, and the domain name of domain B is jimmy.net. Of course, in actual implementation, the number of domains can be expanded as required.

The first step is the client's account application.

If a certain client does not have legal account information in this monitoring system, the user needs to apply for a legal account to the operator of this monitoring system before using it. The process of applying for the account may be to directly apply to the operator, and then implant the account information into the client, such as the storage module of the camera device.

After the client has applied for a legal account, the location server will store the account information of the client.

The account information includes an account name and a password, and the account name may adopt a structure of user name@domain name, so that a legal domain is assigned to the client at the account application stage. In addition, the administrator can group the clients on the positioning server, such as dividing the clients in the same domain into a group according to the domain name, or grouping the monitoring terminal and the monitored terminal, or in other ways, Group clients in different domains into a group. In this way, by configuring the attributes of the group, the authority of the user in the corresponding group, the type of service that can be enjoyed, and the quality of the service can be managed. Of course, the client can also modify and update its permissions and other attributes within a certain range after logging in to the Internet.

For example, the account name of the implanted camera device A is Zhangsan@tom.com, and the password is 111111; the account name of the monitoring terminal B is Lisi@jimmy.net, and its password is 000000.

In the second step, the client logs in.

When the user connects the camera device A to the Internet through its network interface, the control unit in the camera device A will call the client application program and control the camera device to automatically send a login request to the proxy server, and the proxy The server forwards the login request to the registration server.

Specifically, the control unit of the camera device A takes out the account name and password in the storage module according to the client application program, and first forwards the account name to the registration server through the proxy server. After the registration server receives the account name, it correspondingly Generate a random code, save and return the random code; camera device A encrypts its account name, password and the random code through a pre-agreed encryption algorithm (such as MD5 algorithm) after receiving the random code The encrypted data is calculated, and then the encrypted data is forwarded to the registration server through the proxy server. The registration server takes out the account name and password of the camera device A stored in it, and uses the same encryption algorithm to encrypt and calculate the account name, password and random code to obtain encrypted data, and then encrypts the encrypted data obtained by itself. Compared with the encrypted data sent by the camera device A, when the two are completely consistent, the authentication is passed, and the camera device A login is successful.

Similarly, the user can use the account name 李四@jimmy.net and the password to use any computer or handheld device connected to the Internet as the monitoring terminal B to log in to the registration server through the proxy server.

The third step is client registration.

After the login is successful, with the support of the client program, the camera device A can register its own system information with the registration server through the proxy server.

The system information of the camera device A includes its address information, temperature, humidity, nickname, etc. Of course, the camera device A will be equipped with temperature and humidity sensors at the same time. The address information may be IP address and port.

If the camera device registers its own system information with the registration server through the proxy server for the first time, the registration server will store all the system information of the camera device A to the positioning server. If the imaging device is not registered with the registration server for the first time, the registration server will only update the part of the location server that will be different from the last registration information, especially update the address information of the imaging device A to the current address. In a particular embodiment, said registration request is sent to the registration server together with a login request. In another specific embodiment, the camera will regularly register with the registration server under the support of the client program.

The setting of the camera device provided by the present invention enables the camera device to actively register its system information, especially address information, with the registration server no matter where the camera device is moved, as long as it can be connected to the Internet, so that the monitoring terminal can pass The location server finds it.

Similarly, after the monitoring terminal B logs in, the monitoring terminal A will also register its system information, especially address information, with the registration server. This design of the monitoring terminal also enables users not only to use fixed monitoring equipment, but also to choose any equipment as the monitoring terminal.

In the fourth step, the monitoring terminal monitors the monitored terminal (that is, the camera device).

If the monitoring terminal B 李四@jimmy.net needs to monitor the camera device A 张三@tom.com, first, the monitoring terminal B sends a call request to the camera device A to the proxy server.

The call request data packet includes the account name of the requested camera A, the account name and address information of the monitoring terminal B 李四@jimmy.net itself, and may also include other information such as call content and data type.

As an implementation, after the proxy server receives the call request, it first conducts an address query to the location server according to the account name of the camera device A to obtain the IP address and port of the camera device A, and then the call request according to The queried IP address and port are forwarded to camera device A.

As another specific implementation manner, after the proxy server receives the call request, firstly, according to the account names of the monitoring terminal B and the camera device A, it inquires their grouping and rights management information in the positioning server, so that the proxy server Determine whether they have permission to communicate with each other. When the proxy server determines that the monitoring terminal B has no right to access the camera device A, the proxy server will return a message of denial of access to the monitoring terminal B. On the contrary, the proxy server inquires the obtained address information of the camera device A from the positioning server, and then continues the above-mentioned operation of forwarding the call request.

Since the account name in this embodiment adopts the structure of username@domain name, the proxy server can query according to the domain name first, and then query according to the username, thereby speeding up the query.

After the camera device A receives the call request, it can return the consent information to the monitoring terminal B through a proxy server or directly. At this time, the camera device A and the monitoring terminal B already know each other's latest address information. Next, the camera device A can transmit the multimedia data in the monitored area to the monitoring terminal B according to the call request for the user to monitor. It should be noted that, the response request or image transmission action of the above-mentioned camera device is all performed by the camera device under the control of its control unit according to the client application program.

In this embodiment, the expansion of the system can be realized conveniently through the setting of domains. If a new domain is added, only the domain name of the domain needs to be recorded in the location server. If a new client is added in the domain, it is only necessary to record the account information and address information of the client to the location server. In this way, the proxy server can easily contact the client in any domain in the system, thereby helping the client to realize data communication between the domain and the domain.

Embodiment two

Referring to FIG. 5 , the difference from Embodiment 1 is that in this embodiment, each domain has its own proxy server, registration server and location server. Therefore, the proxy server of each domain receives the request of the client in this domain and the information forwarded by the proxy servers of other domains, the location server of each domain stores the account information and address information of the client in this domain, and the registration server of each domain according to The positioning server in the local domain performs registration, login and authentication processes for the clients in the local domain. A client in a certain domain only performs login, authentication, and registration operations with the proxy server, registration server, and location server in the domain. The specific login, authentication, and registration operations are similar to those in Embodiment 1, so details are not repeated here.

The call request sent by the monitoring terminal in a certain domain to the monitored terminal (ie, the camera device in this embodiment) is first sent to the proxy server in the domain where it is located. After the proxy server receives the call request, it first inquires to the location server according to the account names of the monitoring terminal and the monitored terminal contained in the call request, thereby judging the grouping and authority status of the monitoring terminal, and inquiring if the monitoring terminal has no right to call the monitored terminal. When monitoring the terminal, return the message of rejecting the call. And when having the call authority, the call request is forwarded. Since the account names of the monitoring terminal and the monitored terminal in this embodiment all adopt the structure of user name@domain name, the proxy server analyzes the account name of the camera device to obtain the domain name of the camera device, thereby judging whether the camera device belongs to the domain, If yes, then obtain its address information, so as to forward the call request to the monitored terminal; if not, forward the call request to the proxy server in other domains.

In this way, the communication channel between the monitoring terminal and the monitored terminal in different domains is established, so that they can communicate with each other. After the monitoring terminal establishes communication with the camera device, the subsequent steps are the same as those described in Embodiment 1, and will not be repeated here.

If a new domain is added in this embodiment, it is only necessary to add a proxy server, a registration server, and a location server of the domain, and at the same time inform other domains of its domain name and address information of the proxy server, so that system expansion can be easily realized.

Embodiment Three

As can be seen from the analysis of Embodiment 2, compared with Embodiment 1, since the proxy server, registration server and positioning server are all set in each domain, the workload of the proxy server, registration server and positioning server is not only reduced, but also more beneficial to the system. expansion. However, when a default proxy server is preset in the client application program of the client itself, when the client logs in, it will send a login request to the default proxy server, and at this time, the client does not necessarily belong to the proxy server The domain you are in. When the second embodiment handles this situation, the login request will be forwarded to other proxy servers until the proxy server of the domain to which the client belongs is queried. In this way, since the client does not know that the proxy server that sends the request is not the proxy server of the domain to which it belongs, every time it logs in, or even every subsequent request will be sent to the default proxy server, thus causing the default proxy server. The workload of the proxy server is heavy, and the internal management of the system is chaotic.

Based on the above reasons, with reference to FIG. 6, a redirection server is added in the present embodiment, which is set in the Internet for determining the location information of proxy servers in all domains, and sending a redirection message to the client.

Specifically, when the client sends a login request to its default proxy server for the first time, the proxy server knows that the client does not belong to the domain it manages according to the domain name contained in the account name of the client, so it forwards the login request to the redirect server. At this time, the redirection server will query the location servers of all domains to determine the domain to which the client belongs, and send the location information (which may include IP address and port) of the proxy server of this domain in the form of a redirection message Sent to the client directly or through the default proxy server. After receiving this redirection message, the client will replace its default proxy server with the redirected proxy server, so that it will only communicate with the redirected proxy server in its subsequent operations.

Embodiment four

Referring to FIG. 7 , the difference between this embodiment and the previous three embodiments is that a cross-gateway server, a database, and a media server are added to the system. in,

The database is set in the Internet for storing account information of clients in various domains and providing it to the redirection server for query and location server for download.

The media server is set in the Internet and is used to receive and store the media stream and data from the monitored terminal, and at the same time provide the monitoring terminal with the query of the stored information.

The cross-gateway server is set in the Internet, and is used to realize the data communication across the gateway for the clients located in the gateway.

Therefore, compared with the previous three embodiments, this embodiment has stronger functionality, smoother network data interaction, and more reasonable structural configuration.

In this embodiment, the processes of the client logging in to the registration server, authentication, registration, proxy server authority management, and request forwarding are similar to those in the previous embodiment, and will not be repeated here. Differences of this embodiment from the previous embodiments will be described in detail below.

Since the database is added in the system of this embodiment, when the client applies for an account with the operator, its account information will be recorded in the database. The location server in each domain will download the updated client account information of the domain from the database. When the redirection server performs redirection, it can also directly query the database for the domain to which a certain client belongs. In this way, on the one hand, the redundancy of system data is increased to ensure that data is not lost, and on the other hand, the work efficiency of the redirection server is also accelerated.

Due to the addition of a media database in the system of this embodiment, the media stream or data of the client (especially the camera) can be stored by sending to the media server, and the authenticated user can also access the media server. Get saved information.

As shown in FIG. 7, the monitoring terminal C logs into the Internet through the gateway. In this way, the cross-gateway server is added in the system of this embodiment, which solves the problem of cross-gateway communication of the client in the monitoring system.

As a specific implementation, in this embodiment, refer to Fig. 8 (it should be noted that, for the sake of simplicity, other servers such as proxy server are omitted in Fig. Gateway server communication), the specific workflow of the cross-gateway server is:

1. Cross-gateway server initialization process:

After the cross-gateway server is started, it will initialize the mapping table inside it, specifically: if there is no mapping table inside the cross-gateway server, the cross-gateway server will establish a mapping table, which is composed of several records. And each record is one or two tuples, and the specific content of the tuples will be described below. If the mapping table already exists inside the cross-gateway server, clear the contents of the mapping table to obtain the latest mapping relationship.

When the cross-gateway server completes the setting and initialization steps of the mapping table, it will open the listening port port0 and enter the listening state. In this way, it is possible to wait for the help network camera device to communicate across the gateway.

2. Client (including camera and monitoring terminal) network status detection process:

After the client logs in to the network, it first sends a detection request to a port (such as port x) of the cross-gateway server.

After the cross-gateway server receives the detection request, it will reply a message to the client through the port receiving the detection request (ie port x) and another port (such as port y) respectively, and include the source address of the request received by the port x in the message, When the client is in the public network, the source address is the IP address and the port number assigned by the network communication interface for its access to the public network; when it is in the private network, the source address is assigned by the gateway where it is located public IP address and port number.

After the client receives the message replied by the cross-gateway server, it can compare the source address in the message with the address it actually sends the request to. If the two are consistent, it means that the client is directly connected to the public network. If the addresses are inconsistent, it means that the client connects to the public network through the gateway.

The client can store the network status detection result locally, or inform the proxy server. When the client sends a request to the outside world through the proxy server, the proxy server judges whether the request needs to be forwarded to the cross-gateway server according to the network status detection result of the client.

3. Client cross-gateway communication process:

When the client is in the private network, the client will send a connection request (TCP/IP protocol) or a distribution request (UDP protocol) to the listening port port0 of the cross-gateway server through the proxy server. The gateway performs network address translation, thereby assigning a port port1 to the client.

After the cross-gateway server monitors the request, it detects the protocol type with the gateway. When the detected protocol type is UDP, the message is an allocation request, and the cross-gateway server allocates ports, and UDP is used as the protocol type value, and is written into the mapping table together with the port mapping relationship between the gateway and the cross-gateway server.

Suppose we set the intranet IP address of one of the clients (such as a monitoring terminal) to 10.0.1.16, which connects to the gateway through port 5500. The public network IP address of the gateway is 202.1.1.1, and the port number of the port port1 allocated to the network camera device when it performs network address translation is 2222. So there is a mapping relationship:

10.0.1.16:5500 → 202.1.1.1:2222.

The cross-gateway server allocates another port port1-1 according to the request obtained by monitoring the port port0, and records the mapping relationship between the gateway port port1 and the cross-gateway server port port1-1 in the mapping table, specifically: record the gateway port port1 The IP address and port number of the server, the IP address and port number of the port port1-1 of the cross-gateway server. In this embodiment, we can set the IP address and port number of the listening port port0 to 202.0.10.168:4000. Set the IP address and port number of port port1-1 to 202.0.10.169:5000. Therefore, the mapping relationship of records in the mapping table is:

202.1.1.1:2222→202.0.10.169:5000.

In addition, the cross-gateway server also records the protocol type of the request in the mapping table.

Then, the cross-gateway server sends a reply to the gateway through the port port0, thereby forwarding to the client (monitoring terminal) through the gateway.

Once the cross-gateway server records the address mapping relationship of the network camera device in the mapping table, other clients (such as a camera device, and whether it is located in the public network or a private network under the gateway) only need to know the address mapping relationship of the network camera device on the cross-gateway server. The IP address and port number of the port port1-1 can send communication data to the client (monitoring terminal).

After the port port1-1 of the cross-gateway server receives communication data from other clients, the cross-gateway server will forward the message to the gateway through port port0 according to the mapping relationship recorded in its mapping table, thereby passing the gateway Forward to the client (monitoring terminal). Similarly, the communication data from the client may also be forwarded.

In this way, the data interaction process of the client under the private network for cross-gateway communication based on the UDP protocol is completed.

When the protocol type detected by the cross-gateway server is TCP/IP, the message is a connection request. After the monitoring port port0 of the cross-gateway server receives the connection request from the client behind the gateway (such as a monitoring terminal), it will allocate a communication port port1-0, thereby returning a reply to the client through the communication port port1-0. And maintain the communication state between the port and the gateway.

After the client receives this reply, it will send another allocation request to the port port1-0 of the cross-gateway server, and at this time, the cross-gateway server will open another listening port port0-1. When the listening port port0-1 receives a connection request from other clients (such as a camera), it will close the listening port port0-1, and allocate another port port1-1 for communicating with the other clients. Then cross-gateway server records the IP address and port number of port port1-1, the IP address and port number of the client (monitoring terminal), the IP address and port number of the communication port port1-0 in the mapping table, all Describe the IP address and port number of the gateway port port1, the protocol type of communication between the gateway and the cross-gateway server, and the protocol type of communication between other clients (camera devices) and the cross-gateway server.

In this embodiment, the intranet IP address of the monitoring terminal can be set to 10.0.1.16, which is connected to the gateway through port 5500. The public network IP address of the gateway is 202.1.1.1, and the port number of the port port1 assigned to the monitoring terminal when it performs network address translation is 2222. So there is a mapping relationship:

10.0.1.16:5500 → 202.1.1.1:2222.

And set the IP address and port number of the listening port port0 to 202.0.10.168:4000. Set the IP address and port number of the communication port port1-0 as: 202.0.10.168:4001. Set the IP address and port number of port port0-1 to 202.0.10.169:5001. Set the IP address and port number of port port1-1 to 202.0.10.169:5000. In addition, the IP address and port number of the camera device are also set as: 202.10.1.111:4000. Therefore, the two quintuples contained in a record in the mapping table are:

202.1.1.1:2222, 202.0.10.168:4001, TCP;

202.0.10.169:5000, 202.10.1.111:4000, TCP.

Therefore, when the camera device sends a message to the port port0-1, the cross-gateway server can send the message to the port1 port of the gateway through the port port1-0 according to the mapping relationship in the mapping table, thereby forwarding to the monitoring terminal. Similarly, the monitoring terminal also sends the communication data to the cross-gateway server, so as to forward it to the camera device.

In this way, the data interaction process of the client in the system for cross-gateway communication based on the TCP/IP protocol is completed.

As can be seen from the above, the present invention adopts the concept of domains, performs authority management and grouping on users through domains, uses proxy servers and registration and location servers to manage domain users, and routes messages between users in different domains through proxy servers. For the transfer of orders and messages, the cross-gateway communication of the client is realized through the cross-gateway server, and the capacity of the system can be expanded by expanding the proxy server and the registration and positioning server. In addition, the client terminal in the system of the present invention can realize mobile monitoring and being monitored, and is more convenient to use. Moreover, the system framework of the present invention adopts a distributed structure, which is convenient for expansion and large-area use, and provides an effective, safe and reliable operation platform for commercial operations.

Claims (12)

1. A multimedia monitoring system comprises a client, a registration server, a positioning server and a proxy server, wherein:

the client sides are arranged in a plurality of domains and comprise monitored terminals and monitoring terminals, wherein the monitored terminals are used for collecting images in monitoring areas of the monitored terminals, the monitoring terminals are used for displaying the images sent by the monitored terminals for monitoring by users, and each client side is provided with account information of the client side; the positioning server is arranged in the Internet and used for receiving and storing account information and address information sent by the client;

the registration server is arranged in the Internet and used for performing the registration, login and authentication processes of the client according to the positioning server;

the proxy server is arranged in the Internet and receives the request sent by the client, so that the communication among the clients and among the client, the registration server and the positioning server is realized according to the request content and the address information provided by the registration server and the positioning server.

2. The multimedia monitoring system of claim 1, wherein: the proxy server receives requests from clients of all domains, the positioning server stores account information and address information of the clients of all domains, and after receiving requests for calling other clients sent by the clients, the proxy server inquires corresponding address information from the positioning server according to the account information of other clients included in the requests, so as to forward the calling requests to called clients.

3. The multimedia monitoring system of claim 1, wherein: the proxy server of each domain receives the request of the client in the domain and the information forwarded by the proxy servers of other domains; the positioning server of each domain stores account information and address information of the client in the domain; the registration server of each domain performs registration, login and authentication processes on the client in the domain according to the positioning server in the domain; after receiving a call request from a client to another client, the proxy server inquires whether the called client belongs to the local domain from the positioning server according to account information of the called client contained in the request, and if so, obtains address information of the called client, thereby forwarding the call request to the called client; if not, the call request is forwarded to the proxy servers of the other domains.

4. The multimedia monitoring system of claim 1, wherein: the positioning server is also used for storing the system information of the client, the client sends registration and login authentication requests to the registration server through the proxy server based on the Internet by using the account information, and the registration server inquires the positioning server so as to authenticate the positioning server according to the stored account information and update the current address information, the account information and the system information of the client after the authentication is passed.

5. The multimedia monitoring system of claim 1, wherein: the proxy server is also used for grouping clients and managing the authority, the positioning server stores the client grouping and the authority management information, and after receiving a request sent by the client, the proxy server inquires the grouping and the authority management information of the client from the positioning server according to the client account information so as to judge the grouping condition and the authority range of the client.

6. The multimedia monitoring system of claim 1, 3, 4 or 5, wherein: the system also comprises a redirection server which is arranged in the Internet and used for determining the position information of the proxy servers of all domains and sending out a redirection message to the client.

7. The multimedia monitoring system of claim 6, wherein: the system also comprises a database which is arranged in the Internet and used for storing account information of the client of each domain and providing the account information for the redirection server to inquire.

8. The multimedia monitoring system of claim 1, wherein: the monitored terminal comprises a camera module, an image processing module, a control unit, a storage module and a network interface, the account information of the monitored terminal is stored in the storage module, and a client application program is arranged in the control unit and used for automatically sending registration and login authentication requests to the registration server and sending address information to the positioning server at regular time.

9. The multimedia monitoring system of claim 7, wherein: the media server is used for receiving and storing the media stream and data from the monitored terminal and providing the query of the monitoring terminal for the stored information.

10. The multimedia monitoring system of claim 7, wherein: the system also comprises a cross-gateway server which is used for realizing the data communication of the client end positioned in the gateway and crossing the gateway.

11. The multimedia monitoring system of claim 1, wherein: the monitored terminal is a network camera.

12. The multimedia monitoring system of claim 1, wherein: the account information comprises an account name and a password, the account name adopts a structure of a user name @ domain name, and the address information comprises an IP address and a port.

Images