CN114845150A - A multi-video display synchronization system, method, device and storage medium for a display screen - Google Patents

Abstract

The invention belongs to the technical field of LED display, and discloses a multi-video display synchronization method on a display screen. First, a bus communication is added between players as a supplement for GPS/Beidou satellite time synchronization, so as to ensure that the multi-player can play the divided video. Simultaneity; second, in the process of processing the video file of the player, the decoded image frame is added to the frame identification and the HDMI data stream is formed. After receiving the HDMI data stream output by the player, the sending card parses to obtain the frame ID, and restores the original image frame. Then the sending card selects the image frame according to the predetermined frame configuration parameters through the frame identification and sends it to the receiving card through the network, thus ensuring that the spliced image comes from the same image frame generated by the original video. Through the above improvements, the video playback frame synchronization of the display screen is ensured, and the display effect of the large display screen is improved.

Description

A multi-video display synchronization system, method, device and storage medium for a display screen

technical field

The invention belongs to the technical field of LED display screens, and relates to a multi-video display synchronization system, method, device and storage medium on a display screen.

Background technique

The distributed three-layer architecture design of the large LED display is shown in Figure 1.

The first layer consists of a player and a sending card. The player decodes the video file to generate video data frames and sends them to the video sending card through the HDMI interface. The video sending card receives the HDMI signal, and performs a series of operations on the video image, such as scaling, YUV to RGB conversion, Gamma inverse conversion, etc., and then outputs it through the Gigabit Ethernet (10 Gigabit Ethernet) port in the form of data packets.

The second layer is composed of n receiving cards. The receiving card receives the data packets of the receiving card through address identification, and sends it to the LED module in the sequence and data format of the LED light points. (Note that each receiving card only receives the data of the LED light point corresponding to the LED module)

The third layer is composed of n LED modules, each LED module is composed of i*k LED light point arrays, and the LED module contains LED light points and drivers. The LED module receives the module image data sent by the receiving card, and realizes the display of the module pattern.

With the continuous development of LED display technology, outdoor LED display shows the following characteristics:

(1) The display resolution has been continuously improved, and the maturity of mini LED has enabled the LED display to achieve high-definition and ultra-high-definition video display capabilities. This means that the number of LED light points contained in the LED display screen increases, and the number of light points for a large LED display screen can reach more than 2 million points.

(2) For high-definition video playback, a refresh rate of up to 60Hz is required.

(3) The size of the LED display screen is constantly increasing, and it presents the characteristics of various special-shaped structures. The LED screen is large enough to install the entire facade of the building into a display screen, and the display screen of thousands of square meters is already very common. At the same time, large-scale LED outdoor screens are generally designed and integrated with the building according to the facade structure of the building. Therefore, the aspect ratio of the screen is not constrained by the aspect ratio of the general video, and it is expressed in the form of a special-shaped screen. Played video source.

It can be seen from the above characteristics that in large outdoor LED displays, special-shaped screens with aspect ratios that do not conform to general video aspect ratios often appear. These special-shaped screens use dedicated video sources, which exceed the decoding capability of general-purpose players. , normal playback cannot be performed. Another situation is that when the video is played, the image data decoded by the player is transmitted through the network, and the bandwidth of the transmitted network cannot meet the high refresh rate transmission requirements of the video image data. In order to solve the above problems, the video cutting method is generally adopted in engineering, and the video is divided into multiple video files according to the screen, and multi-player is used to cooperate to complete the large-screen display, so as to meet the decoding capability requirements of the universal player and reduce the network bandwidth requirements.

The prerequisite for the above multi-player to perform video image splicing and display of large LED screen is the synchronous playback of video, that is, the image frame spliced on the LED display screen at any time of the video image output by the multi-player is the original video image frame segmentation come. The current video synchronization method is as follows:

(1) Time synchronization

Time synchronization means that multiple players play the split video at the same time, so as to achieve the purpose of frame synchronization. At present, GPS/Beidou satellite signals are used for time synchronization, and the synchronization method of playing at the same time point is used. The disadvantage is that when the geographical location of the device is weak or the signal is weak due to obstacles, time synchronization is difficult to guarantee.

(2) Use decoded I frame synchronization

For H264-encoded video, I, P, and B frames are formed after decoding (I frame: key frame; P frame: forward reference frame; B frame: bidirectional reference frame), and the decoded I frame is usually used to achieve video synchronization. However, Because the appearance and frequency of the I frame are associated with the video content. When the video content does not change much for a long time, the number of I frame occurrences is significantly reduced, which will cause the video image frame to be out of sync.

However, when the number of pixels in the video image frame reaches a certain number, due to the limitation of bandwidth, the network transmission cannot reach the refresh rate of 60 frames/s, and the HDMI output of the player is a fixed refresh rate of 60 frames/s. In this case, the sending card should discard a part of the image frames, and adopt the frame extraction method to reduce the high bandwidth requirement of data transmission. It can be seen from the sending process of the image data of the sending card that the frame drawing process is random for each sending card. Even if the time synchronization of the player is accurate, the image frames received by the receiving cards of different systems cannot be guaranteed to be synchronized.

Taking into account the above situations, to achieve synchronous display, we must find a more effective method.

SUMMARY OF THE INVENTION

In order to overcome the defects of the above technologies, the purpose of the present invention is to provide a multi-video display synchronization system, method, device and storage medium on a display screen, which solves the problem of asynchrony of image frames in current video playback.

The present invention is achieved through the following technical solutions:

A multi-video display synchronization system on a display screen, comprising a plurality of players and a plurality of sending cards, and the players and the sending cards are connected;

Add RS232/RS485 bus communication module and RS232/RS485 communication interface on the player, for the player to choose between RS232/RS485 communication synchronization and satellite synchronization;

The player is provided with a video decoding module, an image frame generation module and an HDMI controller that are connected in sequence. The video decoding module is used to generate continuous image frames, and the image frame generation module is used to add frames to the continuous image frames generated after decoding. ID, and write to HDMI send buffer; HDMI controller is used to output HDMI data stream with frame ID;

The sending card is provided with a storage module and a receiving module, an image frame generation module, an image frame extraction module, a scaling module, a Gamma correction module, a comparison module and an image subcontracting module which are connected in sequence;

The receiving module is used to receive the HDMI data stream with the frame identification; the image frame generation module is used to generate the image frame; the image frame extraction module is used to extract the image frame through the frame identification according to the set frame extraction rules, and restore the original image frame; scaling module, used to scale and compress the restored image frame; Gamma correction module, used to perform Gamma inverse transform processing on the image frame; storage module, used to store the frame sampling table; comparison module, used to check the sampling The frame table is used to judge whether the image frame with the frame identifier is extracted; the image packetization module is used for packetization and transmission in the network.

Further, in the RS232/RS485 communication mode, one of the players is used as the master, and the other players are used as the slaves.

Further, the player is connected with the sending card through the HDMI interface.

The invention also discloses a multi-video display synchronization method on a display screen, including clock synchronization and frame extraction synchronization. The specific process is as follows:

The first step: clock synchronization, including two ways:

Mode 1: In the case of good GPS/Beidou satellite signals, all players work in GPS/Beidou satellite synchronization mode to achieve time synchronization;

Mode 2: In the case of weak GPS/Beidou satellite signals, configure one of the players as the RS232/RS485 host mode, and the other players are configured as slaves. host synchronization;

The second step: frame synchronization, which includes the following steps:

Step 1: Start the player, decode to generate continuous image frames and image frame numbers;

Step 2: Add frame identification to the continuous image frames generated after decoding;

Step 3: Generate image frames with frame IDs;

Step 4: Write the image frame with frame identification into the HDMI send buffer;

Step 5: The HDMI controller reads the HDMI send buffer, and the controller image frame is converted into an HDMI data stream and output through the HDMI interface;

Step 6: The sending card receives the HDMI data stream and parses it into image frames with frame identification;

Step 7: Extract the frame identification from the image frame with the frame identification, and restore the original image frame;

Step 8: Check the frame drawing data table, obtain the frame ID of the sending frame, read the corresponding image frame, and package it in the sending network.

Further, in the second step, adding frame identification to the continuous image frames generated after decoding is specifically:

The lower 8 bits of the image frame number are taken as the frame identification, and the frame identification is embedded in the image frame.

Further, the third step is specifically: adding a row/column of pixels in the fixed position of each frame of image, and replacing the Y value in the YUV of the pixel with the image frame number to generate an image frame with a frame identification.

Further, in step 8:

The frame identification range is 0-255, and the frame data table is an array composed of 256 0s or 1s;

The frame extraction synchronization process is to find the elements in the array through the frame identifier. If the extracted element is 1, the restored original image will be written into the image frame sending buffer; if the extracted element is 0, the frame image will be discarded.

The invention also discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the display screen when the processor executes the computer program Steps of a multi-video display synchronization method.

The invention also discloses a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, realizes the steps of the multi-video display synchronization method on a display screen.

Compared with the prior art, the present invention has the following beneficial technical effects:

The invention discloses a multi-video display synchronization method on a display screen. By adding an RS485 communication channel to a player in a multi-video playback and splicing display system, communication between the multi-players is established to achieve time synchronization, thereby ensuring that the divided video is in the Simultaneity of playback on different players; image frame identifiers are added to consecutive image frames after decoding in the player to form HDMI data stream output with frame identifiers. This image frame with frame identification is transmitted to the sending card, the sending card decodes and restores the original image frame and extracts the frame identification; the sending card performs frame extraction processing according to the set rule table, and writes the extracted frame into the sending buffer , and the rest of the image frames will be discarded, and the image frames with the definite frame ID will be sent to the receiving card through the network. Through the above improvements, in any case, the simultaneity of multi-player video playback and the certainty of frame drawing by the sending card are ensured, and the synchronization of video display splicing image frames is realized.

Description of drawings

Figure 1 shows the structure of the large LED display control system before the improvement;

Figure 2 shows the structure of the improved large-scale LED display control system;

Fig. 3 is the processing procedure before and after the improvement of the player for video files; (a) is the processing flow before the player is improved; (b) is the processing flow after the player is improved;

Fig. 4 shows the processing procedure of the video file before and after the improvement of the sending card; (a) is the processing flow before the improvement of the sending card; (b) is the processing flow after the improvement of the sending card.

Detailed ways

Below in conjunction with accompanying drawing, the present invention is described in further detail:

As shown in Figure 1, the distributed three-layer architecture design of the large LED display.

The first layer consists of a player and a sending card. The player decodes the video to generate video data frames and sends them to the video sending card through the HDMI interface. The video sending card receives the HDMI signal, and performs a series of operations on the video image, such as scaling, YUV to RGB conversion, Gamma inverse conversion, etc., and then outputs it through the Gigabit Ethernet (10 Gigabit Ethernet) port in the form of data packets.

The second layer is composed of n receiving cards. The receiving card receives the data packets of the receiving card through address identification, and sends it to the LED module in the sequence and data format of the LED light points. (Note that each receiving card only receives the data of the LED light point corresponding to the LED module)

The third layer is composed of n LED modules, each LED module is composed of i*k LED light point arrays, and the LED module contains LED light points and drivers. The receiving card sends the image data of the module to the module in the timing format required by the LED driver, so as to realize the display of the module pattern.

For the special-shaped screen, it is often necessary to divide the video and realize the video display through the video playback and display system. Since multiple video playback and display systems are independent of each other, to achieve synchronous playback, the consistency of the clocks of the two systems must be ensured. In practice, the satellite navigation synchronization method is used, and the synchronization effect is poor due to various reasons such as the installation position of the player and the occlusion of obstacles.

In practice, there is also a situation in which frame synchronization cannot be realized by using time synchronization. When the number of video image pixels reaches a certain number, due to the limitation of bandwidth, the sending card or receiving card cannot reach the refresh rate of 60 frames/s, while the HDMI output of the player is a fixed refresh rate of 60 frames/s. In this case, the sending card should discard a part of the image frames, and adopt the frame extraction method to reduce the high bandwidth requirement of data transmission. It can be seen from the sending process of the image data of the sending card that the frame drawing process is random for each sending card. Even if the time synchronization of the player is accurate, the image frames received by the receiving cards of different systems cannot be guaranteed to be synchronized.

The improved multi-video display synchronization system structure of the display screen is shown in Figure 2.

First, the player increases the RS232/RS485 communication channel to ensure the time synchronization between multiple machines when there is no GPS/Beidou satellite signal. The RS232/RS485 interface configuration is optional as the master/slave mode, and the player's clock synchronization mode is optional. When multiple players work, you can choose GPS/Beidou satellite synchronization or RS232/RS485 communication synchronization. When selecting the RS232/RS485 communication mode for synchronization, configure the synchronization mode as RS232/RS485 synchronization mode, and select one of the players as the RS232/RS485 communication master, and configure the rest as slaves. As shown in the figure, the large-screen display system composed of two sets of video playback and control systems uses the RS232/RS485 communication method to synchronize the structure diagram.

When the satellite synchronization signal strength is good and the number of satellites that can be captured at the same time exceeds 3, the satellite is used to achieve time synchronization; when the satellite synchronization signal is poor, the RS232/RS485 communication method is used for synchronization. In the RS232/RS485 communication mode, through the parameter configuration of the players, one of the players works in the master mode, and the other players work in the slave mode. The master sends the clock signal regularly, and the slave receives the signal and processes it to achieve clock synchronization .

Second, the player decodes the image frame and adds the image frame identifier

The playback process of the original player is shown in Figure 3(a); the improved playback process is shown in Figure 3(b).

The player is provided with a video decoding module, an image frame generation module and an HDMI controller that are connected in sequence. The video decoding module is used to generate continuous image frames, and the image frame generation module is used to add frames to the continuous image frames generated after decoding. Flag, and write to HDMI send buffer; HDMI controller is used to output HDMI data stream with frame flag.

For the continuous image frames and image frame numbers generated after video decoding, the lower 8 bits of the image frame number are taken as the frame identifier, and the frame identifier is embedded in the image frame by the following method.

A row (column) of pixels is added to the fixed position of each frame of image, the Y value in the YUV of the pixel is replaced by the image frame identifier, an image frame with a frame identifier is generated, and the image frame with the frame identifier is converted by the controller. Output for HDMI data stream.

Third, the improvement of the sending card

The workflow of the original sending card is shown in Figure 4(a); the improved workflow is shown in Figure 4(b). In the image processing flow of the original sending card, the process of image frame identification extraction, original image frame restoration and frame extraction processing has been added.

Specifically, the sending card is provided with a storage module and a receiving module, an image frame generation module, an image frame extraction module, a scaling module, a Gamma correction module, a comparison module and an image sub-packet module connected in sequence;

The receiving module is used to receive the HDMI data stream with frame identification;

Image frame generation module for generating image frames;

The image frame extraction module is used to extract the image frame through the frame identification according to the set frame extraction rule, and restore the original image frame;

The scaling module is used for scaling and compressing the restored image frame;

The Gamma correction module is used to perform inverse Gamma transform processing on the image frame, and perform image enhancement processing and color attenuation correction on the image frame to make it more in line with the visual characteristics of the human eye;

The storage module is used to store the frame drawing table;

The comparison module is used to check the frame extraction table to determine whether the image frame with the frame identification is extracted;

The image packetization module is used for packetization and transmission in the network.

The frame extraction process adopts the look-up table method. First, the frame extraction data table is established, as shown in the following table. Since the frame identification range is 0-255, the data table is an array composed of 256 0s or 1s. The frame extraction process is to find the elements in the array through the frame identifier. If it is 1, the restored original image will be written into the image frame sending buffer, and if it is 0, the image of this frame will be discarded. The purpose of accurate frame extraction is achieved through the above method.

Through the above improvements, the playback synchronization of the player and the accurate frame drawing of the sending card can be ensured, thereby ensuring the consistency of image frames received by the receiving card at the same time.

After the improvement, the synchronization process of the large-screen display system of multiple players is as follows:

(1) Clock synchronization

GPS/Beidou satellite mode: When GPS/Beidou satellites are unobstructed and satellite signals are excellent, all players work in GPS/Beidou satellite synchronization mode to achieve time synchronization.

Master-slave mode: When GPS/Beidou satellites are blocked and the satellite signal is weak, connect the RS485 interface of the player through the bus, and configure one of the players as the RS232/RS485 host mode, and the rest of the players are configured as slaves. machine. The master sends a clock synchronization signal regularly, and the slave receives the synchronization signal and synchronizes with the master.

(2) frame extraction synchronization

Step 1: Start the player, decode to generate continuous image frames and image frame numbers;

Step 2: take the lower 8 bits of the image frame number generated after decoding as the frame identification (0-255), and embed the frame identification in the continuous image frames generated after decoding;

Step 3: Generate image frames with frame IDs.

The specific method is to add a row (column) of pixels in the fixed position of the image, and replace the Y value of the brightness information in the YUV of the pixel with an 8-bit image frame number;

Step 4: Write the image frame with frame identification into the HDMI send buffer;

Step 5: The HDMI controller reads the HDMI transmit buffer, and the controller converts the image frame to HDMI data stream and outputs it through the HDMI interface;

Step 6: The sending card receives the HDMI data stream and parses it into image frames with frame identification;

Step 7: Extract the frame identification from the image frame with the frame identification, and restore the original image frame;

Step 8: Check the frame drawing table, obtain the frame ID of the sent frame, read the corresponding image frame, package it and send it through the network.

The method for synchronizing multi-video display on a display screen of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

If the multi-video display synchronization method on the display screen of the present invention is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like. Computer-readable storage media includes both persistent and non-permanent, removable and non-removable media, and storage of information can be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. It should be noted that the content contained in the computer-readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable media Electric carrier signals and telecommunication signals are not included. Wherein, the computer storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor memory (eg, ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state disk (SSD)), and the like.

In an exemplary embodiment, there is also provided a computer apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the computer program when executed The steps of the display multi-video display synchronization method. The processor may be a central processing unit (CentralProcessingUnit, CPU), or other general-purpose processors, digital signal processors (DigitalSignalProcessor, DSP), application specific integrated circuit (ApplicationSpecificIntegratedCircuit, ASIC), ready-made programmable gate array (Field-ProgrammableGateArray, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

The content of the present invention is not limited to those listed in the above embodiments, and various modifications or deformations that can be made by those skilled in the art without creative efforts are covered by the claims of the present invention.

Claims (9)

1. a multi-video display synchronization system on a display screen, is characterized in that, comprises a plurality of players and a plurality of sending cards, and the player is connected with the sending card; Add RS232/RS485 bus communication module and RS232/RS485 communication interface on the player, for the player to choose between RS232/RS485 communication synchronization and satellite synchronization; The player is provided with a video decoding module, an image frame generation module and an HDMI controller that are connected in sequence. The video decoding module is used to generate continuous image frames, and the image frame generation module is used to add frames to the continuous image frames generated after decoding. ID, and write to HDMI send buffer; HDMI controller is used to output HDMI data stream with frame ID; The sending card is provided with a storage module and a receiving module, an image frame generation module, an image frame extraction module, a scaling module, a Gamma correction module, a comparison module and an image subcontracting module which are connected in sequence; The receiving module is used to receive the HDMI data stream with the frame identification; the image frame generation module is used to generate the image frame; the image frame extraction module is used to extract the image frame through the frame identification according to the set frame extraction rules, and restore the original image frame; scaling module, used to scale the restored image frame; Gamma correction module, used to perform inverse Gamma transform processing on the image frame; storage module, used to store the frame selection table; comparison module, used to check the frame selection The table is used to judge whether the image frame with the frame identifier is extracted; the image packetization module is used for packetization and transmission in the network. 2 . The multi-video display synchronization system on a display screen according to claim 1 , wherein, under the RS232/RS485 communication mode, one of the players is used as the host, and the other players are used as the slaves. 3 . 3 . The multi-video display synchronization system on a display screen according to claim 1 , wherein the player is connected to the sending card through an HDMI interface. 4 . 4. a multi-video display synchronization method on a display screen, is characterized in that, comprises clock synchronization and frame extraction synchronization, and concrete process is as follows: The first step: clock synchronization, including two ways: Mode 1: In the case of good GPS/Beidou satellite signals, all players work in GPS/Beidou satellite synchronization mode to achieve time synchronization; Mode 2: In the case of weak GPS/Beidou satellite signals, configure one of the players as the RS232/RS485 host mode, and the other players are configured as slaves. host synchronization; The second step: frame synchronization, which includes the following steps: Step 1: Start the player, decode to generate continuous image frames and image frame numbers; Step 2: Add frame identification to the continuous image frames generated after decoding; Step 3: Generate image frames with frame IDs; Step 4: Write the image frame with frame identification into the HDMI send buffer; Step 5: The HDMI controller reads the HDMI send buffer, and the controller image frame is converted into an HDMI data stream and output through the HDMI interface; Step 6: The sending card receives the HDMI data stream and parses it into image frames with frame identification; Step 7: Extract the frame identification from the image frame with the frame identification, and restore the original image frame; Step 8: Check the frame drawing data table, obtain the frame ID of the sending frame, read the corresponding image frame, and package it in the sending network. 5. a kind of display screen multi-video display synchronization method according to claim 4 is characterized in that, in the 2nd step, to the continuous image frame that is generated after decoding, adding frame identification is specifically: The lower 8 bits of the image frame number are taken as the frame identification, and the frame identification is embedded in the image frame. 6. a kind of display screen multi-video display synchronization method according to claim 4, is characterized in that, the 3rd step is specifically: in each frame image fixed position, increase row/column pixel, the Y value in the YUV of pixel point Use the image frame number instead to generate an image frame with a frame ID. 7. a kind of display screen multi-video display synchronization method according to claim 4 is characterized in that, in the 8th step: The frame identification range is 0-255, and the frame data table is an array composed of 256 0s or 1s; The frame extraction synchronization process is to find the elements in the array through the frame identifier. If the extracted element is 1, the restored original image will be written into the image frame sending buffer; if the extracted element is 0, the frame image will be discarded. 8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the computer program as claimed in the claims Steps of the method for synchronizing multi-video display on a display screen according to any one of 4 to 7. 9. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the display screen as claimed in any one of claims 4 to 7 is implemented. The video shows the steps of the synchronization method.

Images