CN118055243B - Audio and video coding processing method, device and equipment for digital television - Google Patents
Audio and video coding processing method, device and equipment for digital television Download PDFInfo
- Publication number
- CN118055243B CN118055243B CN202410449268.7A CN202410449268A CN118055243B CN 118055243 B CN118055243 B CN 118055243B CN 202410449268 A CN202410449268 A CN 202410449268A CN 118055243 B CN118055243 B CN 118055243B
- Authority
- CN
- China
- Prior art keywords
- audio
- video
- video data
- transmission
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 23
- 230000005540 biological transmission Effects 0.000 claims abstract description 194
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 48
- 238000012545 processing Methods 0.000 claims abstract description 28
- 230000004927 fusion Effects 0.000 claims abstract description 19
- 238000012544 monitoring process Methods 0.000 claims abstract description 17
- 238000005538 encapsulation Methods 0.000 claims abstract description 11
- 238000012937 correction Methods 0.000 claims description 21
- 238000005457 optimization Methods 0.000 claims description 17
- 238000004458 analytical method Methods 0.000 claims description 16
- 239000013598 vector Substances 0.000 claims description 14
- 230000006835 compression Effects 0.000 claims description 11
- 238000007906 compression Methods 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 10
- 230000009466 transformation Effects 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000004806 packaging method and process Methods 0.000 claims description 9
- 230000001360 synchronised effect Effects 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 238000003066 decision tree Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 238000001228 spectrum Methods 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 238000004140 cleaning Methods 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 2
- 238000000034 method Methods 0.000 abstract description 40
- 230000000694 effects Effects 0.000 abstract description 11
- 230000008569 process Effects 0.000 description 30
- 230000008859 change Effects 0.000 description 16
- 230000005236 sound signal Effects 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000008713 feedback mechanism Effects 0.000 description 3
- 230000008014 freezing Effects 0.000 description 3
- 238000007710 freezing Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000000547 structure data Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
- H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64784—Data processing by the network
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application relates to the technical field of audio and video coding, and discloses an audio and video coding processing method, device and equipment of a digital television. The method comprises the following steps: performing signal processing on the original audio and video signals to obtain first audio and video signals, extracting content and performing self-adaptive hierarchical coding to obtain first audio and video data streams; performing feature decomposition and data encapsulation to obtain a plurality of second audio and video data streams; creating an initial transmission parameter set through a multi-path transmission model and transmitting data; receiving a plurality of second audio and video data streams through the digital television, and carrying out data stream decoding and depth signal fusion to obtain second audio and video signals; playing the second audio and video signals through the digital television and performing state monitoring to obtain playing state data; the application realizes the audio and video dynamic coding of the digital television, improves the transmission stability of audio and video data, and further improves the playing effect of the digital television.
Description
Technical Field
The present application relates to the field of audio and video coding technologies, and in particular, to an audio and video coding processing method, apparatus, and device for a digital television.
Background
In the field of digital televisions today, with continuous progress of technology and increasing demands of consumers for high-quality audio and video contents, an audio and video coding processing method becomes a key for improving viewing experience of users. Conventional audio and video coding methods face various challenges, such as noise problem in the original signal, data loss in the signal transmission process, and asynchronous audio and video during playing, which seriously affect the viewing experience.
The original audio and video signals often contain various background noises and interferences, which not only reduce the signal quality, but also increase the coding difficulty, thereby influencing the transmission efficiency and the final playing quality. Therefore, it becomes important to perform efficient denoising and signal enhancement on the original signal. Secondly, with the diversity and instability of network conditions, how to ensure efficient, reliable transmission of audio-video data, especially in the case of limited broadband or network congestion, is another challenge. The traditional single-path transmission model is difficult to cope with network fluctuation, is easy to cause data loss or play and is blocked, and user experience is seriously affected. Audio-video dyssynchrony is a common problem affecting the viewing experience of digital television, which not only confuses the viewer, but also reduces the viewing comfort.
Disclosure of Invention
The application provides an audio and video coding processing method, device and equipment for a digital television, which realize audio and video dynamic coding of the digital television, improve the transmission stability of audio and video data and further improve the playing effect of the digital television.
In a first aspect, the present application provides an audio/video coding processing method of a digital television, where the audio/video coding processing method of the digital television includes:
denoising and enhancing the original audio and video signals to obtain first audio and video signals, and extracting content and adaptively layering coding the first audio and video signals to obtain a first audio and video data stream;
performing feature decomposition and data encapsulation on the first audio/video data stream to obtain a plurality of second audio/video data streams;
Creating an initial transmission parameter set of the plurality of second audio/video data streams through a multi-path transmission model, and carrying out data transmission on the plurality of second audio/video data streams according to the initial transmission parameter set;
Receiving the plurality of second audio and video data streams through a digital television, and carrying out data stream decoding and depth signal fusion on the plurality of second audio and video data streams to obtain second audio and video signals;
playing the second audio and video signals through the digital television, and carrying out audio and video synchronous correction and state monitoring on the second audio and video signals to obtain playing state data;
And carrying out state feedback optimization on the initial transmission parameter set according to the play state data to generate a target transmission parameter set.
In a second aspect, the present application provides an audio/video coding processing apparatus of a digital television, where the audio/video coding processing apparatus of the digital television includes:
The coding module is used for carrying out denoising processing and signal enhancement on the original audio and video signals to obtain first audio and video signals, and carrying out content extraction and self-adaptive hierarchical coding on the first audio and video signals to obtain a first audio and video data stream;
The packaging module is used for carrying out feature decomposition and data packaging on the first audio/video data stream to obtain a plurality of second audio/video data streams;
the transmission module is used for creating an initial transmission parameter set of the plurality of second audio/video data streams through a multi-path transmission model and carrying out data transmission on the plurality of second audio/video data streams according to the initial transmission parameter set;
the decoding module is used for receiving the plurality of second audio and video data streams through the digital television, and carrying out data stream decoding and depth signal fusion on the plurality of second audio and video data streams to obtain second audio and video signals;
The correction module is used for playing the second audio and video signals through the digital television, and carrying out audio and video synchronous correction and state monitoring on the second audio and video signals to obtain playing state data;
and the optimizing module is used for carrying out state feedback optimization on the initial transmission parameter set according to the play state data to generate a target transmission parameter set.
A third aspect of the present application provides a computer apparatus comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the computer device to execute the audio and video coding processing method of the digital television.
A fourth aspect of the present application provides a computer-readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform the above-described audio/video encoding processing method of a digital television.
According to the technical scheme provided by the application, the wavelet transformation algorithm is adopted to carry out denoising treatment on the original audio and video signals, and the denoised signals are subjected to dynamic range compression and local contrast enhancement, so that the definition and the ornamental value of the audio and video signals are obviously improved. The wavelet transformation algorithm can effectively remove noise components in the signals, and the dynamic range compression and the local contrast enhancement further ensure that details and contrast of the signals are reserved and highlighted, so that the final audio and video content is more vivid and lifelike. The transmission process of the content is optimized through a multipath transmission model and self-adaptive hierarchical coding based on the characteristics of the content. The multipath transmission model allows the data stream to be transmitted through different network paths, so that the transmission interruption risk caused by single path faults is effectively reduced, and the reliability of data transmission is improved. The self-adaptive hierarchical coding dynamically adjusts the coding strategy according to different characteristics of the content, such as movement speed, color change and the like, so that high-quality audio and video content can be transmitted under the condition of limited bandwidth. By carrying out depth signal fusion and audio-video synchronization correction on the received audio-video data stream, the playing stability and the playing synchronism are obviously improved. The depth signal fusion utilizes advanced algorithm to combine signals on different transmission paths, compensates possible data loss, and ensures signal integrity. The audio and video synchronous correction ensures that the playing time of the audio and video is strictly consistent, and the problem of audio-visual mismatch in the watching process is avoided. The overall viewing experience is further improved by monitoring the playing state in real time and performing dynamic feedback optimization on playing and transmission parameters. The state monitoring can timely find out problems in the playing process, such as buffer delay, picture freezing and the like, and the dynamic feedback mechanism can adjust parameters of transmission rate, coding quality and the like according to the information, so that the playing effect is optimized in real time, and the user can enjoy smooth and high-quality audio and video content.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained based on these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an embodiment of an audio/video coding method of a digital television according to an embodiment of the present application;
fig. 2 is a schematic diagram of an embodiment of an audio/video encoding processing device of a digital television according to an embodiment of the present application.
Detailed Description
The embodiment of the application provides an audio and video coding processing method, device and equipment of a digital television. The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
For easy understanding, the following describes a specific flow of an embodiment of the present application, referring to fig. 1, and an embodiment of an audio/video encoding processing method of a digital television in the embodiment of the present application includes:
Step 101, denoising and signal enhancement are carried out on an original audio/video signal to obtain a first audio/video signal, and content extraction and self-adaptive layered coding are carried out on the first audio/video signal to obtain a first audio/video data stream;
It can be understood that the execution subject of the present application may be an audio/video encoding processing device of a digital television, and may also be a terminal or a server, which is not limited herein. The embodiment of the application is described by taking a server as an execution main body as an example.
Specifically, the wavelet transformation algorithm is adopted to carry out denoising treatment on the original audio and video signals, and the algorithm can effectively separate noise and useful information in the signals to obtain denoised audio and video signals. And the dynamic range compression is carried out on the denoising video signal, so that the dynamic range of the signal is reduced, the compressed audio and video signal is obtained, the complexity of the signal is reduced, and the processing efficiency is improved. And carrying out local contrast enhancement and formatting processing on the compressed audio and video signals, improving the visual effect of the signals, and obtaining a clearer first audio and video signal. The first audio and video signals are subjected to content extraction through a Convolutional Neural Network (CNN) model, and important content information can be effectively extracted from the first audio and video signals through the model. And analyzing the content characteristics of the audio and video signal content information to obtain a set reflecting the characteristics of the video content, thereby being beneficial to understanding the structure and characteristics of the audio and video content. The first audio-video signal is adaptively layered based on the content characteristic set, the signal is decomposed into a plurality of target coding layers according to different characteristics of the content, each layer aims at a specific part of the content, and the layering mode enables the coding process to be optimized according to the importance and the characteristics of the content. Each target coding hierarchy will have its corresponding coding policy settings, which are formulated according to the characteristics of the content to ensure coding efficiency and quality. And according to the set target coding strategy, carrying out signal coding on each target coding layer, and converting the first audio and video signals into a first audio and video data stream.
102, Performing feature decomposition and data encapsulation on the first audio/video data stream to obtain a plurality of second audio/video data streams;
Specifically, the inter-frame difference calculation is performed on the video data in the first audio/video data stream, and the inter-frame difference value is obtained by comparing the differences between adjacent frames, so as to help identify the time when the video content changes, which may be due to scene transition or important actions. And performing color histogram difference calculation on the video data, and calculating to provide detailed information about color change in the video so as to obtain a color histogram difference value. By means of the two difference values, a plurality of first characteristic decomposition points of the first audio-video data stream can be accurately determined, and the decomposition points mark the positions of important changes in video content. And carrying out audio frequency spectrum characteristic decomposition on the audio data in the first audio-video data stream, and generating a plurality of second characteristic decomposition points by analyzing the frequency spectrum characteristics of the audio signal. The audio spectral decomposition can reveal main components and characteristics in the audio signal, such as changes in high and low frequency components, thereby providing basis for changes in audio content. And comprehensively analyzing the first characteristic decomposition point of the video and the second characteristic decomposition point of the audio, and determining a plurality of target characteristic decomposition points by considering the overall characteristics of the audio and video data. And based on the determined multiple target feature decomposition points, carrying out data stream decomposition on the first audio/video data stream, and decomposing the original audio/video data stream into multiple decomposition data streams. Each of the decomposed data streams represents a segment of the original data stream that is partitioned according to varying characteristics of the content, ensuring that the decomposed data streams better reflect the structure and characteristics of the original content. And data packaging is carried out on the decomposed data streams, and each decomposed data stream is packaged into an independent second audio/video data stream. The encapsulation process not only includes packaging of the data, but also involves encoding and formatting processes to ensure that each second audio-video data stream can be efficiently transmitted and played in the digital television system.
Step 103, creating an initial transmission parameter set of a plurality of second audio/video data streams through a multi-path transmission model, and carrying out data transmission on the plurality of second audio/video data streams according to the initial transmission parameter set;
Specifically, a plurality of candidate transmission paths are initialized, flexibility and high efficiency of data transmission are ensured, and the paths are selected by analyzing the topology structure and bandwidth resources of the network. And constructing a transmission model for a plurality of candidate transmission paths through a preset graph neural network. The graph neural network can predict the transmission characteristics of each path based on the real-time state and the historical data of the network, and a multipath transmission model which reflects the current network state and considers the historical data is constructed. And carrying out transmission path matching on a plurality of second audio/video data streams through the multi-path transmission model, so as to ensure that each data stream is distributed on a path which is most suitable for the characteristics of the data stream. The matching process considers the size of the data streams, the transmission requirements, and the current state of the network, and finds a path with highest transmission efficiency and lowest delay for each data stream. Network condition parameters including network congestion conditions, bandwidth variations, delays, etc. are acquired, and based on these parameters, an optimal transmission rate and retransmission policy are calculated for each target transmission path, respectively. And creating a plurality of initial transmission parameter sets of the second audio-video data streams according to the transmission rate and the retransmission strategy of each target transmission path. All necessary parameter settings, such as transmission rate, number of retransmissions, retransmission interval, etc., are included in this set to provide an initial, optimized parameter configuration for each data stream transmission. And distributing the second audio and video data streams to the corresponding target transmission paths, and carrying out multipath data transmission according to the created initial transmission parameter set. Each data stream will be transmitted on a selected path according to its specific parameter settings, which not only improves the efficiency of the transmission, but also enhances the stability and reliability of the data transmission process.
104, Receiving a plurality of second audio/video data streams through the digital television, and carrying out data stream decoding and depth signal fusion on the plurality of second audio/video data streams to obtain second audio/video signals;
Specifically, a plurality of second audio-video data streams are received through the digital television, each data stream containing a portion of the audio-video content or information of a specific aspect. And carrying out data stream decoding on the plurality of second audio/video data streams, and converting the coded signals in the data streams into playable audio/video signals. And respectively extracting depth signal characteristics of the plurality of decoded audio and video signals through a preset RNN model. The RNN model is adapted to process sequence data, such as audio-visual signals, and the model is capable of capturing time-varying features of such data. A set of depth signal features is extracted from each decoded audio-visual signal using the RNN model, the feature sets reflecting key content and attributes of the signal. And carrying out signal fusion on the plurality of decoded audio and video signals according to the depth signal characteristic set, and synthesizing the plurality of decoded audio and video signals into a single high-quality second audio and video signal. The signal fusion process considers the relation and complementarity between different signals, and realizes the seamless integration of the content by adjusting and combining the characteristics of the signals. The fusion process dynamically adjusts the fusion strategy according to the extracted depth signal features to ensure that the finally obtained second audio-video signal has the best quality in terms of vision and hearing.
Step 105, playing a second audio and video signal through the digital television, and performing audio and video synchronous correction and state monitoring on the second audio and video signal to obtain playing state data;
Specifically, parameter configuration is carried out on a player in the digital television according to the depth signal characteristic set. The depth signal feature set contains key information that affects the playback quality and viewing experience, such as the dynamic range, contrast, color depth, etc. of the signal. By adjusting the parameters of the player to match these features, it is ensured that the second audio-visual signal can achieve optimal visual and audible effects when played. And when the second audio and video signals are played through the player, performing audio and video synchronization correction. The synchronous rectification effort relies on algorithms to analyze and adjust the playback rate of the audio and video data streams to ensure that they are perfectly aligned at the correct point in time. Meanwhile, the state of the player is monitored, including the operation state, the data buffering condition, the network connection quality and the like of the player, so that any problem which possibly affects the playing experience is captured in real time. And analyzing the performance of the player and the stability of the network environment according to the initial state data. These initial state data may contain noise or non-standardized information, and thus require data cleansing and data normalization processing. Data cleansing includes removing erroneous, duplicate, or extraneous data records to ensure accuracy and correlation of the data. The data normalization process converts the data into a format that is easier to analyze and understand, and the resulting play status data reflects the performance of the player and various states during the play process.
And 106, performing state feedback optimization on the initial transmission parameter set according to the play state data to generate a target transmission parameter set.
Specifically, status feature encoding is performed on the play status data, where the status data includes information about performance of the player, network conditions, user experience, and the like. The data is converted into state feature code vectors by state feature coding, and the vectors reflect key state features in the playing process in a structured form. And inputting the state characteristic coding vector into a preset decision tree model. The decision tree model is a machine learning model applied to classification and regression tasks that generates a corresponding strategy by analyzing the input feature vectors for generating an initial parameter adjustment execution strategy for each target transmission path. And acquiring rewarding feedback parameters corresponding to each target transmission path, wherein the parameters reflect the effects of the previous transmission strategy, including transmission delay, data packet loss rate, user satisfaction and the like. And (3) carrying out strategy gradient analysis on each target transmission path by using the reward feedback parameters, wherein the strategy gradient analysis is an optimization method based on gradient descent and indicates how to adjust the transmission strategy to improve the reward feedback parameters, namely, improve the transmission effect. And carrying out strategy parameter feedback analysis on the reward feedback parameters according to the strategy gradient, and calculating to obtain strategy parameter feedback values of each target transmission path. These feedback values provide a well-defined direction and magnitude for parameter adjustment, guiding how to optimize the initial parameter adjustment execution strategy for better transmission performance. And optimizing the initial parameter adjustment execution strategy according to the strategy parameter feedback value to generate a target parameter adjustment execution strategy of each target transmission path. The target strategy considers the latest network condition and user experience feedback, and can more accurately meet the transmission requirement. And performing comprehensive parameter optimization on the initial transmission parameter set according to the target parameter adjustment execution strategy to generate a target transmission parameter set. The set contains key parameters such as optimized transmission rate, coding quality, retransmission strategy and the like, ensures that the transmission process of audio and video data is more efficient and stable, can adapt to the change of network environment, and finally improves the watching experience of users.
In the embodiment of the application, the noise of the original audio and video signals is removed by adopting the wavelet transformation algorithm, and the dynamic range compression and the local contrast enhancement are carried out on the signals after the noise removal, so that the definition and the ornamental value of the audio and video signals are obviously improved. The wavelet transformation algorithm can effectively remove noise components in the signals, and the dynamic range compression and the local contrast enhancement further ensure that details and contrast of the signals are reserved and highlighted, so that the final audio and video content is more vivid and lifelike. The transmission process of the content is optimized through a multipath transmission model and self-adaptive hierarchical coding based on the characteristics of the content. The multipath transmission model allows the data stream to be transmitted through different network paths, so that the transmission interruption risk caused by single path faults is effectively reduced, and the reliability of data transmission is improved. The self-adaptive hierarchical coding dynamically adjusts the coding strategy according to different characteristics of the content, such as movement speed, color change and the like, so that high-quality audio and video content can be transmitted under the condition of limited bandwidth. By carrying out depth signal fusion and audio-video synchronization correction on the received audio-video data stream, the playing stability and the playing synchronism are obviously improved. The depth signal fusion utilizes advanced algorithm to combine signals on different transmission paths, compensates possible data loss, and ensures signal integrity. The audio and video synchronous correction ensures that the playing time of the audio and video is strictly consistent, and the problem of audio-visual mismatch in the watching process is avoided. The overall viewing experience is further improved by monitoring the playing state in real time and performing dynamic feedback optimization on playing and transmission parameters. The state monitoring can timely find out problems in the playing process, such as buffer delay, picture freezing and the like, and the dynamic feedback mechanism can adjust parameters of transmission rate, coding quality and the like according to the information, so that the playing effect is optimized in real time, and the user can enjoy smooth and high-quality audio and video content.
In a specific embodiment, the process of executing step 101 may specifically include the following steps:
(1) Denoising the original audio and video signals by adopting a wavelet transformation algorithm to obtain denoised video signals;
(2) Dynamic range compression is carried out on the denoising video signal to obtain a compressed audio/video signal;
(3) Local contrast enhancement and formatting processing are carried out on the compressed audio and video signals, so that first audio and video signals are obtained;
(4) Extracting the content of the first audio and video signal through the CNN model to obtain audio and video signal content information;
(5) Analyzing the content characteristics of the content information of the audio and video signals to obtain a content characteristic set;
(6) Performing self-adaptive layering on the first audio and video signals according to the content characteristic set to obtain a plurality of target coding layers;
(7) Setting a coding strategy for a plurality of target coding levels to obtain a target coding strategy of each target coding level;
(8) And carrying out signal coding on the first audio and video signals according to the target coding strategy to obtain a first audio and video data stream.
Specifically, a wavelet transformation algorithm is adopted to carry out denoising treatment on an original audio and video signal, and useful information and noise in the signal are separated. The wavelet transform algorithm effectively distinguishes detailed portions (e.g., edges) and smooth portions in the signal by analyzing the signal over multiple scales so that the denoised audio-video signal retains more valuable information. For example, when processing an audio signal containing background noise and dialog, the wavelet transform can distinguish the waveform difference between the waveform of the dialog and the waveform of the background noise, thereby effectively removing the background noise and preserving clear dialog content. And the dynamic range compression is carried out on the denoised audio and video signals, so that the dynamic range of the signals is reduced, and the signals can keep good audio-visual effects in different playing environments. Dynamic range compression reduces the difference between the extremely high and low amplitude parts of the signal and the average level of the signal by adjusting them so that the details are more apparent while avoiding the problem of excessive or insufficient volume. And carrying out local contrast enhancement and formatting processing on the compressed audio and video signals. Local contrast enhancement focuses on improving the visual effect of images, particularly in areas with rich details, such as scenes with staggered light and shadow, by increasing local contrast, details are clearer, and viewing experience is improved. The formatting process ensures compatibility of the signal format with subsequent processing and playback devices. For example, local contrast enhancement may make dark detail more visible when processing a night scene video, while formatting ensures that the video can be played on a different digital television. The first audio-video signal is content extracted using a Convolutional Neural Network (CNN) model that is capable of identifying important content information, such as scene features, objects, and voice commands, from the audio-video signal. For example, by processing a video of a city street view through a CNN model, key elements such as vehicles, pedestrians, etc., as well as noise of the background and dialogue content can be identified. And (3) carrying out content characteristic analysis on the extracted audio and video signal content information to obtain a characteristic set of the content, such as moving speed, color change, rhythm of sound and the like. Based on the content characteristic set, the first audio-video signal is adaptively layered, and the signal is decomposed into a plurality of target coding layers according to the importance and characteristics of different contents. The layering strategy allows different encoding strategies to be adopted for different levels of content, and storage and transmission efficiency is optimized. For example, in a scene containing multiple layers of dialogue and background music, the dialogue and background music can be processed separately by adaptive layering to ensure the clarity of the dialogue and the richness of the background music. Coding strategies are set for each target coding level, and signal coding is carried out on the first audio/video signals according to the strategies, wherein the strategies consider the characteristics and transmission requirements of contents of each level, such as high bit rate coding for high dynamic scenes and low bit rate coding for static scenes. Finally, a first audio and video data stream is obtained.
In a specific embodiment, the process of executing step 102 may specifically include the following steps:
(1) Carrying out inter-frame difference calculation on video data in the first audio/video data stream to obtain an inter-frame difference value;
(2) Performing color histogram difference calculation on video data in the first audio and video data stream to obtain a color histogram difference value;
(3) Determining a plurality of first feature decomposition points of the first audio/video data stream according to the inter-frame difference value and the color histogram difference value;
(4) Performing audio frequency spectrum feature decomposition on the audio data in the first audio-video data stream to generate a plurality of second feature decomposition points;
(5) Comprehensively analyzing the feature demarcation points of the plurality of first feature decomposition points and the plurality of second feature decomposition points to obtain a plurality of target feature decomposition points;
(6) Carrying out data stream decomposition on the first audio and video data stream according to the target feature decomposition points to obtain a plurality of decomposition data streams;
(7) And carrying out data encapsulation on the plurality of decomposed data streams to obtain a plurality of second audio/video data streams.
Specifically, the inter-frame difference calculation is performed on the video data in the first audio/video data stream, the difference between two adjacent frames is compared, and the motion or scene change degree in the video is evaluated. By calculating these differences, inter-frame difference values are obtained that are highly reflective of the dynamic changes in the video content. And performing color histogram difference calculation, and evaluating the change of color distribution. The color histogram graphically represents the frequencies of different colors in the image, and the color histogram of the adjacent frames is compared to obtain a color histogram difference value. The value can reveal details of scene color change, such as gradual change of sky color in sunset or color difference caused by abrupt change of light, and provide quantized measurement for the color change of video content. And combining the inter-frame difference value and the color histogram difference value to determine a plurality of first characteristic decomposition points of the first audio-video data stream. The feature decomposition point marks an important change in the video content, such as the moment of a jump from one scene to another, or a transition from static to dynamic. And simultaneously, carrying out audio frequency spectrum characteristic decomposition on the audio data in the first audio-video data stream, and generating a plurality of second characteristic decomposition points by analyzing the frequency spectrum of the audio signal. The audio spectral feature decomposition can reveal the dominant frequency components in the audio, forming second feature decomposition points. And carrying out characteristic demarcation point comprehensive analysis on the plurality of first characteristic decomposition points and the plurality of second characteristic decomposition points. And fusing key feature points of the video and the audio, and determining a plurality of target feature decomposition points which represent key change nodes of the audio and video content and are the basis of data stream decomposition. The first audio-video data stream is decomposed into a plurality of decomposed data streams based on the target feature decomposition points. The data stream is split into smaller paragraphs according to the content variations, each paragraph focusing on a particular content or scene. And carrying out data encapsulation on the decomposed data streams to generate a plurality of second audio/video data streams. The data encapsulation refers to packaging the decomposed data stream according to a certain format and protocol, so that the data stream is suitable for different playing and transmission requirements. Through data encapsulation, each decomposed data stream is given an independent identification and necessary metadata, so that management and calling in a digital television system are facilitated.
In a specific embodiment, the process of executing step 103 may specifically include the following steps:
(1) Initializing a plurality of candidate transmission paths, and constructing a transmission model of the plurality of candidate transmission paths through a preset graph neural network to obtain a multipath transmission model;
(2) Carrying out transmission path matching on a plurality of second audio and video data streams through a multipath transmission model to obtain a target transmission path of each second audio and video data stream;
(3) Acquiring network condition parameters, and respectively calculating the transmission rate and retransmission strategy of each target transmission path according to the network condition parameters;
(4) Creating a plurality of initial transmission parameter sets of the second audio and video data streams according to the transmission rate and the retransmission strategy of each target transmission path;
(5) And distributing the second audio and video data streams to corresponding target transmission paths, and carrying out multipath data transmission on the second audio and video data streams according to the initial transmission parameter set.
Specifically, a plurality of candidate transmission paths are initialized. These paths are selected based on the physical topology of the network, historical performance data, or expected network loads. For example, in a trans-regional digital television transmission system, candidate paths may include network backbones through different regions, each path having its own unique delay, bandwidth, and packet loss characteristics. And constructing a transmission model of the candidate transmission path through a preset graph neural network. A Graph Neural Network (GNN) is a machine learning model that is adapted to process graph structure data, such as network topology. GNNs are able to learn the characteristics of nodes (e.g., routers, switches) and edges (i.e., connections) in the network and how they affect overall transmission performance. Through training, the GNN model can predict the performance of each path under the specific network condition, thereby constructing a multipath transmission model reflecting the real network state. And carrying out transmission path matching on the plurality of second audio/video data streams. The target transmission path most suitable for each data stream is selected by a model taking into account the characteristics (e.g., size, real-time requirements) of each data stream and the performance (e.g., expected delay, bandwidth) of each path. Network condition parameters such as current network congestion level, real-time bandwidth and delay of each path, etc. are acquired. Based on these parameters, a transmission rate and a retransmission policy are calculated for each target transmission path. According to the network performance model, factors such as the size of a data packet, a transmission window, a congestion control algorithm and the like are considered. By calculation, an initial set of transmission parameters is set for each data stream, which set of parameters is aimed at achieving optimal transmission efficiency and reliability under the current network conditions. And distributing the second audio and video data streams to respective target transmission paths, and carrying out multipath data transmission according to respective initial transmission parameter sets. Each data flow is transmitted on a selected path according to its particular parameters to accommodate possible network variations and congestion. For example, if a path suddenly becomes congested, the associated data stream may adapt to this change by dynamically adjusting its transmission parameters (e.g., reducing the sending rate or switching to a backup path), thereby ensuring transmission continuity and quality of the data stream.
In a specific embodiment, the process of executing step 104 may specifically include the following steps:
(1) Receiving a plurality of second audio/video data streams through a digital television;
(2) Performing data stream decoding on the plurality of second audio/video data streams to obtain a plurality of decoded audio/video signals;
(3) Respectively extracting depth signal characteristics of a plurality of decoded audio and video signals through a preset RNN model to obtain a depth signal characteristic set of each decoded audio and video signal;
(4) And carrying out signal fusion on the plurality of decoded audio and video signals according to the depth signal characteristic set to obtain a second audio and video signal.
Specifically, the digital television system obtains a plurality of second audio/video data streams through a receiving module thereof. These data streams may be transmitted via different transmission paths, each containing a portion of the content of a television program, e.g., one data stream may carry a main video signal, while the other data stream may contain different angles of shots, additional audio tracks, or special effects. And decoding the received second audio-video data stream. Decoding is the process of restoring a compressed data stream to a playable audio-video signal, requiring the system to be able to handle various encoding formats. After decoding, a plurality of decoded audio-video signals are obtained, each representing a different aspect of the original program. And extracting depth signal characteristics of the decoded audio and video signals through a preset cyclic neural network (RNN) model. The RNN model is suitable for processing sequence data, such as a sequence of video frames or audio samples, and can take into account the time dependence in the data. A set of depth signal features is extracted from each decoded signal by the RNN model, which features reflect key properties of the signal such as motion pattern, scene change, pitch and cadence. And performing signal fusion on the plurality of decoded audio and video signals based on the extracted depth signal feature set. How to optimally combine the different signals is dynamically determined by a deep learning model. And finally generating a fused second audio/video signal.
In a specific embodiment, the process of executing step 105 may specifically include the following steps:
(1) Performing parameter configuration on a player in the digital television according to the depth signal feature set, and playing a second audio/video signal through the player;
(2) Performing audio and video synchronization correction on the second audio and video signal, and performing state monitoring on the player to obtain initial state data;
(3) And performing data cleaning and data standardization processing on the initial state data to obtain play state data.
Specifically, parameter configuration is carried out on a player in the digital television according to the depth signal characteristic set. The depth signal feature set includes information such as resolution, frame rate, dynamic range of the video, and sampling rate and channel number of the audio. These features provide the player with the necessary information to configure it accordingly, ensuring that the player can take full advantage of these signal features to provide optimal playback quality. For example, if the feature set indicates that the video signal has High Dynamic Range (HDR) content, the player will be configured in HDR mode to exhibit richer colors and deeper contrast. Also, if the audio signal contains multiple channels, the player will be configured as a multi-channel output to provide a more immersive listening experience. And when the second audio and video signal is played, performing audio and video synchronization correction. Audio-video dyssynchrony may be caused by delay differences in the encoding, transmission, or decoding process. The player monitors the time offset between audio and video by a built-in synchronization correction mechanism and dynamically adjusts to eliminate these offsets. For example, if the video signal is advanced relative to the audio signal, the player may temporarily delay the playing of the video, or speed up the playing of the audio, to achieve synchronization. Meanwhile, the player is monitored for status, such as buffer status, play error and network connection status. And obtaining initial state data of the playing process through the monitoring data. And performing data cleaning and data standardization processing on the initial state data. Data cleansing includes removing extraneous or erroneous data records, identifying and correcting outliers, while data normalization is the conversion of data into a more consistent format for ease of analysis and understanding, and ultimately to obtain standardized play status data that can be used to analyze play performance, identify and solve problems during play, such as finding that an increase in play delay may indicate a network congestion problem, and thus, actions such as reducing play quality or switching to a better network connection may be taken to improve play experience.
In a specific embodiment, the process of executing step 106 may specifically include the following steps:
(1) Performing state feature coding on the play state data to obtain a state feature coding vector;
(2) Inputting the state feature coding vector into a preset decision tree model for strategy generation to obtain an initial parameter adjustment execution strategy of each target transmission path;
(3) Acquiring rewarding feedback parameters corresponding to each target transmission path, and carrying out strategy gradient analysis on each target transmission path to obtain strategy gradient of each target transmission path;
(4) Performing strategy parameter feedback analysis on the reward feedback parameters according to the strategy gradient to obtain strategy parameter feedback values of each target transmission path;
(5) Performing parameter adjustment execution policy optimization on the initial parameter adjustment execution policy according to the policy parameter feedback value, and generating a target parameter adjustment execution policy of each target transmission path;
(6) And carrying out parameter optimization on the initial transmission parameter set according to the target parameter adjustment execution strategy to generate a target transmission parameter set.
Specifically, the play status data is status feature encoded to convert the data into a form more suitable for machine processing. The play status data includes play delay, buffer count, data packet loss rate, etc., which are encoded into a status feature encoding vector. For example, if frequent buffering is experienced during a video playback, this information will be encoded into the vector reflecting a degradation in playback quality. And inputting the state characteristic coding vector into a preset decision tree model. The decision tree model is a machine learning method capable of generating decision rules based on input feature vectors, thereby generating policies. The model generates an initial parameter adjustment execution strategy for each target transmission path according to the state feature encoding vector. These strategies include preliminary adjustment suggestions for parameters such as transmission rate, retransmission strategy, etc. For example, if the feature encoding vector indicates that the data packet loss rate of a path is high, the decision tree model may recommend reducing the transmission rate of the path. And acquiring the reward feedback parameters corresponding to each target transmission path. These parameters reflect the effects of previous strategies such as success rate and efficiency of transmission. And carrying out strategy gradient analysis on each target transmission path based on feedback, wherein the strategy gradient analysis is an optimization technology based on a gradient descent method and is used for calculating the optimal estimation of each strategy adjustment direction and amplitude. The result of the policy gradient analysis is a policy gradient for each target transmission path, indicating how to adjust the policy to maximize rewards. And carrying out strategy parameter feedback analysis on the reward feedback parameters according to the strategy gradient to obtain strategy parameter feedback values of each target transmission path, wherein the values specifically indicate adjustment amounts of various parameters (such as transmission rate and retransmission times). And optimizing the initial parameter adjustment execution strategy according to the strategy parameter feedback value to generate a target parameter adjustment execution strategy of each target transmission path. For example, if the reward feedback for a path indicates that data loss can be significantly reduced by increasing the number of retransmissions, then the execution strategy for that path will be adjusted accordingly. And comprehensively optimizing the initial transmission parameter set according to the target parameter adjustment execution strategy to generate a target transmission parameter set.
The method for processing audio and video codes of the digital television in the embodiment of the present application is described above, and the following describes an audio and video coding processing device of the digital television in the embodiment of the present application, please refer to fig. 2, and one embodiment of the audio and video coding processing device of the digital television in the embodiment of the present application includes:
the encoding module 201 is configured to perform denoising processing and signal enhancement on an original audio/video signal to obtain a first audio/video signal, and perform content extraction and adaptive hierarchical encoding on the first audio/video signal to obtain a first audio/video data stream;
The packaging module 202 is configured to perform feature decomposition and data packaging on the first audio/video data stream to obtain a plurality of second audio/video data streams;
The transmission module 203 is configured to create an initial transmission parameter set of the plurality of second audio/video data streams through a multi-path transmission model, and perform data transmission on the plurality of second audio/video data streams according to the initial transmission parameter set;
The decoding module 204 is configured to receive a plurality of second audio/video data streams through the digital television, and perform data stream decoding and depth signal fusion on the plurality of second audio/video data streams to obtain a second audio/video signal;
the correction module 205 is configured to play the second audio and video signal through the digital television, and perform audio and video synchronization correction and status monitoring on the second audio and video signal to obtain play status data;
The optimizing module 206 is configured to perform state feedback optimization on the initial transmission parameter set according to the play state data, and generate a target transmission parameter set.
Through the cooperation of the components, the method and the device have the advantages that the wavelet transformation algorithm is adopted to carry out denoising processing on the original audio and video signals, and the dynamic range compression and the local contrast enhancement are carried out on the denoised signals, so that the definition and the ornamental value of the audio and video signals are obviously improved. The wavelet transformation algorithm can effectively remove noise components in the signals, and the dynamic range compression and the local contrast enhancement further ensure that details and contrast of the signals are reserved and highlighted, so that the final audio and video content is more vivid and lifelike. The transmission process of the content is optimized through a multipath transmission model and self-adaptive hierarchical coding based on the characteristics of the content. The multipath transmission model allows the data stream to be transmitted through different network paths, so that the transmission interruption risk caused by single path faults is effectively reduced, and the reliability of data transmission is improved. The self-adaptive hierarchical coding dynamically adjusts the coding strategy according to different characteristics of the content, such as movement speed, color change and the like, so that high-quality audio and video content can be transmitted under the condition of limited bandwidth. By carrying out depth signal fusion and audio-video synchronization correction on the received audio-video data stream, the playing stability and the playing synchronism are obviously improved. The depth signal fusion utilizes advanced algorithm to combine signals on different transmission paths, compensates possible data loss, and ensures signal integrity. The audio and video synchronous correction ensures that the playing time of the audio and video is strictly consistent, and the problem of audio-visual mismatch in the watching process is avoided. The overall viewing experience is further improved by monitoring the playing state in real time and performing dynamic feedback optimization on playing and transmission parameters. The state monitoring can timely find out problems in the playing process, such as buffer delay, picture freezing and the like, and the dynamic feedback mechanism can adjust parameters of transmission rate, coding quality and the like according to the information, so that the playing effect is optimized in real time, and the user can enjoy smooth and high-quality audio and video content.
The application also provides a computer device, which comprises a memory and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, cause the processor to execute the steps of the audio/video coding processing method of the digital television in the above embodiments.
The application also provides a computer readable storage medium, which can be a nonvolatile computer readable storage medium, and can also be a volatile computer readable storage medium, wherein the computer readable storage medium stores instructions, and when the instructions run on a computer, the instructions cause the computer to execute the steps of the audio/video coding processing method of the digital television.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, systems and units may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. The audio and video coding processing method of the digital television is characterized by comprising the following steps of:
denoising and enhancing the original audio and video signals to obtain first audio and video signals, and extracting content and adaptively layering coding the first audio and video signals to obtain a first audio and video data stream;
performing feature decomposition and data encapsulation on the first audio/video data stream to obtain a plurality of second audio/video data streams;
Creating an initial transmission parameter set of the plurality of second audio/video data streams through a multi-path transmission model, and carrying out data transmission on the plurality of second audio/video data streams according to the initial transmission parameter set;
Receiving the plurality of second audio and video data streams through a digital television, and carrying out data stream decoding and depth signal fusion on the plurality of second audio and video data streams to obtain second audio and video signals;
playing the second audio and video signals through the digital television, and carrying out audio and video synchronous correction and state monitoring on the second audio and video signals to obtain playing state data;
And carrying out state feedback optimization on the initial transmission parameter set according to the play state data to generate a target transmission parameter set.
2. The audio/video coding processing method of claim 1, wherein the denoising and signal enhancement are performed on the original audio/video signal to obtain a first audio/video signal, and the content extraction and adaptive layered coding are performed on the first audio/video signal to obtain a first audio/video data stream, comprising:
Denoising the original audio and video signals by adopting a wavelet transformation algorithm to obtain denoised video signals;
Performing dynamic range compression on the denoising video signal to obtain a compressed audio/video signal;
Carrying out local contrast enhancement and formatting on the compressed audio and video signals to obtain first audio and video signals;
extracting the content of the first audio and video signal through a CNN model to obtain audio and video signal content information;
content characteristic analysis is carried out on the audio and video signal content information to obtain a content characteristic set;
Performing self-adaptive layering on the first audio and video signals according to the content characteristic set to obtain a plurality of target coding layers;
Setting the coding strategy of the plurality of target coding levels to obtain a target coding strategy of each target coding level;
And carrying out signal coding on the first audio and video signals according to the target coding strategy to obtain a first audio and video data stream.
3. The audio/video coding processing method of claim 2, wherein the performing feature decomposition and data encapsulation on the first audio/video data stream to obtain a plurality of second audio/video data streams includes:
Performing inter-frame difference calculation on the video data in the first audio/video data stream to obtain an inter-frame difference value;
performing color histogram difference calculation on video data in the first audio and video data stream to obtain a color histogram difference value;
Determining a plurality of first feature decomposition points of the first audio/video data stream according to the inter-frame difference value and the color histogram difference value;
Performing audio frequency spectrum characteristic decomposition on the audio data in the first audio-video data stream to generate a plurality of second characteristic decomposition points;
Performing feature demarcation point comprehensive analysis on the plurality of first feature decomposition points and the plurality of second feature decomposition points to obtain a plurality of target feature decomposition points;
Carrying out data stream decomposition on the first audio/video data stream according to the target feature decomposition points to obtain a plurality of decomposition data streams;
and carrying out data encapsulation on the plurality of decomposed data streams to obtain a plurality of second audio/video data streams.
4. The audio/video coding processing method of claim 1, wherein creating an initial transmission parameter set of the plurality of second audio/video data streams by a multi-path transmission model, and performing data transmission on the plurality of second audio/video data streams according to the initial transmission parameter set, comprises:
Initializing a plurality of candidate transmission paths, and constructing a transmission model of the plurality of candidate transmission paths through a preset graph neural network to obtain a multipath transmission model;
carrying out transmission path matching on the plurality of second audio and video data streams through the multi-path transmission model to obtain a target transmission path of each second audio and video data stream;
Acquiring network condition parameters, and respectively calculating the transmission rate and retransmission strategy of each target transmission path according to the network condition parameters;
Creating an initial transmission parameter set of the plurality of second audio/video data streams according to the transmission rate and the retransmission strategy of each target transmission path;
and distributing the second audio and video data streams to corresponding target transmission paths, and carrying out multipath data transmission on the second audio and video data streams according to the initial transmission parameter set.
5. The audio/video coding processing method of claim 1, wherein the receiving the plurality of second audio/video data streams by the digital television, and performing data stream decoding and depth signal fusion on the plurality of second audio/video data streams, to obtain a second audio/video signal, includes:
receiving the plurality of second audio and video data streams through a digital television;
Performing data stream decoding on the plurality of second audio/video data streams to obtain a plurality of decoded audio/video signals;
respectively extracting depth signal characteristics of the plurality of decoded audio and video signals through a preset RNN model to obtain a depth signal characteristic set of each decoded audio and video signal;
And carrying out signal fusion on the plurality of decoding audio and video signals according to the depth signal characteristic set to obtain a second audio and video signal.
6. The audio/video coding processing method of claim 5, wherein playing the second audio/video signal by the digital television, and performing audio/video synchronization correction and status monitoring on the second audio/video signal to obtain playing status data, comprises:
performing parameter configuration on a player in the digital television according to the depth signal feature set, and playing the second audio and video signals through the player;
performing audio and video synchronization correction on the second audio and video signals, and performing state monitoring on the player to obtain initial state data;
and performing data cleaning and data standardization processing on the initial state data to obtain play state data.
7. The audio/video coding processing method of claim 1, wherein the performing state feedback optimization on the initial transmission parameter set according to the play state data to generate a target transmission parameter set includes:
Performing state feature coding on the play state data to obtain a state feature coding vector;
inputting the state feature coding vector into a preset decision tree model for strategy generation to obtain an initial parameter adjustment execution strategy of each target transmission path;
Acquiring rewarding feedback parameters corresponding to each target transmission path, and carrying out strategy gradient analysis on each target transmission path to obtain strategy gradient of each target transmission path;
Performing strategy parameter feedback analysis on the reward feedback parameters according to the strategy gradient to obtain strategy parameter feedback values of each target transmission path;
performing parameter adjustment execution policy optimization on the initial parameter adjustment execution policy according to the policy parameter feedback value, and generating a target parameter adjustment execution policy of each target transmission path;
And carrying out parameter optimization on the initial transmission parameter set according to the target parameter adjustment execution strategy to generate a target transmission parameter set.
8. An audio/video coding processing device of a digital television, which is characterized in that the audio/video coding processing device of the digital television comprises:
The coding module is used for carrying out denoising processing and signal enhancement on the original audio and video signals to obtain first audio and video signals, and carrying out content extraction and self-adaptive hierarchical coding on the first audio and video signals to obtain a first audio and video data stream;
The packaging module is used for carrying out feature decomposition and data packaging on the first audio/video data stream to obtain a plurality of second audio/video data streams;
the transmission module is used for creating an initial transmission parameter set of the plurality of second audio/video data streams through a multi-path transmission model and carrying out data transmission on the plurality of second audio/video data streams according to the initial transmission parameter set;
the decoding module is used for receiving the plurality of second audio and video data streams through the digital television, and carrying out data stream decoding and depth signal fusion on the plurality of second audio and video data streams to obtain second audio and video signals;
The correction module is used for playing the second audio and video signals through the digital television, and carrying out audio and video synchronous correction and state monitoring on the second audio and video signals to obtain playing state data;
and the optimizing module is used for carrying out state feedback optimization on the initial transmission parameter set according to the play state data to generate a target transmission parameter set.
9. A computer device, the computer device comprising: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the computer device to perform the audio video coding processing method of the digital television of any of claims 1-7.
10. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the audio video coding processing method of a digital television according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410449268.7A CN118055243B (en) | 2024-04-15 | 2024-04-15 | Audio and video coding processing method, device and equipment for digital television |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410449268.7A CN118055243B (en) | 2024-04-15 | 2024-04-15 | Audio and video coding processing method, device and equipment for digital television |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118055243A CN118055243A (en) | 2024-05-17 |
| CN118055243B true CN118055243B (en) | 2024-06-11 |
Family
ID=91046924
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410449268.7A Active CN118055243B (en) | 2024-04-15 | 2024-04-15 | Audio and video coding processing method, device and equipment for digital television |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118055243B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118573959B (en) * | 2024-05-28 | 2025-06-17 | 重庆平可杰信息技术有限公司 | A method and system for collecting audio and video data based on 5G terminal equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103220543A (en) * | 2013-04-25 | 2013-07-24 | 同济大学 | Real-time 3D video communication system and its realization method based on Kinect |
| CN115757037A (en) * | 2022-11-23 | 2023-03-07 | 中铁工程服务有限公司 | Application management system based on data shelter |
| CN116489473A (en) * | 2023-05-05 | 2023-07-25 | 深圳市云屋科技有限公司 | Dynamic compensation system and method for audio and video transmission optimization |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013086027A1 (en) * | 2011-12-06 | 2013-06-13 | Doug Carson & Associates, Inc. | Audio-video frame synchronization in a multimedia stream |
| CN102821323B (en) * | 2012-08-01 | 2014-12-17 | 成都理想境界科技有限公司 | Video playing method, video playing system and mobile terminal based on augmented reality technique |
-
2024
- 2024-04-15 CN CN202410449268.7A patent/CN118055243B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103220543A (en) * | 2013-04-25 | 2013-07-24 | 同济大学 | Real-time 3D video communication system and its realization method based on Kinect |
| CN115757037A (en) * | 2022-11-23 | 2023-03-07 | 中铁工程服务有限公司 | Application management system based on data shelter |
| CN116489473A (en) * | 2023-05-05 | 2023-07-25 | 深圳市云屋科技有限公司 | Dynamic compensation system and method for audio and video transmission optimization |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118055243A (en) | 2024-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI805784B (en) | A method for enhancing quality of media | |
| Barman et al. | No-reference video quality estimation based on machine learning for passive gaming video streaming applications | |
| US12149699B2 (en) | Content adaptation for streaming | |
| CN110324621B (en) | Video encoding method, video encoding device, electronic equipment and storage medium | |
| JP5778146B2 (en) | An adaptive gain control method for digital audio samples in a media stream carrying video content | |
| CN118055243B (en) | Audio and video coding processing method, device and equipment for digital television | |
| CN118694945B (en) | Data video stream self-adaptive processing system and method based on streaming processing technology | |
| CN111277826A (en) | Video data processing method and device and storage medium | |
| WO2012103332A2 (en) | Encoding of video stream based on scene type | |
| KR20040105869A (en) | Apparatus and method for synchronization of audio and video streams | |
| Mäki et al. | A reduced-reference parametric model for audiovisual quality of IPTV services | |
| Göring et al. | Modular framework and instances of pixel-based video quality models for UHD-1/4K | |
| Huang et al. | DAVE: Dynamic adaptive video encoding for real-time video streaming applications | |
| CN108307248B (en) | Video broadcasting method, calculates equipment and storage medium at device | |
| JP4861371B2 (en) | Video quality estimation apparatus, method, and program | |
| Suárez et al. | Assessing the qoe in video services over lossy networks | |
| WO2016164874A1 (en) | System and method for determinig and utilizing priority maps in video | |
| EP2954677B1 (en) | Method and apparatus for context-based video quality assessment | |
| US11659217B1 (en) | Event based audio-video sync detection | |
| Nguyen et al. | Quality-delay tradeoff optimization in multi-bitrate adaptive streaming | |
| CN117459764B (en) | Multichannel intelligent screen display method, device, equipment and storage medium | |
| Arsenović et al. | Quality of Experience Assessment for HTTP Based Adaptive Video Streaming. | |
| Becerra Martinez | A three layer system for audio-visual quality assessment | |
| JP4460548B2 (en) | User experience quality estimation apparatus, method, and program | |
| CN120017644A (en) | WebRTC audio and video stream parameter dynamic adjustment method and device and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |