CN114025185B

CN114025185B - Video playback method and device, electronic equipment and storage medium

Info

Publication number: CN114025185B
Application number: CN202111260144.7A
Authority: CN
Inventors: 郑锦昂; 孙有军; 刘启钧; 陈丽; 徐杭生
Original assignee: Hangzhou Netease Zhiqi Technology Co Ltd
Current assignee: Hangzhou Netease Zhiqi Technology Co Ltd
Priority date: 2021-10-28
Filing date: 2021-10-28
Publication date: 2024-06-25
Anticipated expiration: 2041-10-28
Also published as: CN114025185A

Abstract

The disclosure relates to the technical field of video processing, and in particular relates to a video playback method, a device, an electronic device and a storage medium, wherein each piece of multimedia resource data associated with a target video and at least one piece of play event data corresponding to each piece of multimedia resource data are obtained from a server in response to a playback request for the target video, wherein the play event data are used for controlling the play state of the multimedia resource data, and each piece of multimedia resource data are obtained by respectively storing data streams corresponding to different multimedia elements; respectively determining target playing areas of the multimedia resource data according to the playing area layout information of the client; and playing the multimedia resource data based on the target playing areas and the playing event data, so that the live broadcast scene is restored based on the multimedia resource data and the playing event data, and the user experience can be improved.

Description

Video playback method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of video processing, and in particular relates to a video playback method, a video playback device, electronic equipment and a storage medium.

Background

With the development of the internet, the network live broadcast technology is widely applied to more and more scenes. For example, the method can be applied to the scenes of teaching, finance, meeting, entertainment and the like. When a viewer cannot watch live video in real time, the live content can be known by watching playback video in the client.

In the related art, in the live broadcast process, the live broadcast client sends the obtained live video stream to the server for storage, and when video playback is required, the playback client can obtain the live video stream from the server for playing, so that live video playback is realized.

However, in this manner in the related art, since the live video stream is a standardized audio/video file that is transmitted from the live client to the server and processed and generated by the server, the live video stream generated in this manner can only ensure that the video picture is not distorted when the size of the playing interface of the other client is the same as the size of the playing interface of the live client. However, when video playback is required in different sized playback interfaces, distortion of the video picture may occur, thereby degrading the user experience.

Disclosure of Invention

The embodiment of the disclosure provides a video playback method, a video playback device, electronic equipment and a storage medium, so as to avoid distortion of a live video played back and improve user experience.

The specific technical scheme provided by the embodiment of the disclosure is as follows:

a video playback method, applied to a client, comprising:

responding to a playback request aiming at a target video, acquiring each piece of multimedia resource data associated with the target video and at least one piece of play event data corresponding to each piece of multimedia resource data from a server, wherein the play event data are used for controlling the play state of the multimedia resource data, and each piece of multimedia resource data are respectively obtained by storing data streams corresponding to different multimedia elements;

Respectively determining target playing areas of the multimedia resource data according to the playing area layout information of the client;

and playing the multimedia resource data based on the target playing areas and the playing event data.

Optionally, the method further comprises:

Respectively obtaining multimedia resource data and at least one play event data corresponding to each multimedia element of the video to be processed;

and sending the multimedia resource data and the play event data to a server for associated storage.

Optionally, respectively obtaining multimedia resource data and at least one play event data corresponding to each multimedia element of the video to be processed, which specifically includes:

respectively storing data streams corresponding to each multimedia element of the video to be processed to obtain each multimedia resource data;

And respectively acquiring at least one play event corresponding to each multimedia element according to a preset data format to obtain at least one play event data.

Optionally, each piece of multimedia resource data associated with the target video and at least one piece of play event data corresponding to each piece of multimedia resource data are obtained from a server, which specifically includes:

The playback request is sent to a server, so that the server responds to the playback request and returns the searched multimedia resource data and at least one play event data corresponding to the multimedia data to a client;

wherein, the multimedia resource data and the play event data are obtained by searching the multimedia resource data and the play event data associated with the target video identifier based on the target video identifier in the playback request;

And receiving the multimedia resource data and the play event data sent by the server.

Optionally, determining the target playing area of each multimedia resource data according to the playing area layout information of the client, which specifically includes:

acquiring the size parameter of a playing interface of the client;

Determining playing area layout information of a playing interface of the client based on the size parameter, wherein the playing area layout information comprises playing areas of all multimedia elements in the playing interface;

And respectively determining target playing areas corresponding to the multimedia resource data according to the playing area layout information.

Optionally, playing the multimedia resource data based on each target playing area and each playing event data specifically includes:

according to a preset ordering mode, ordering the playback sequence of each piece of multimedia resource data and each piece of play event data to obtain each piece of ordered multimedia resource data and each piece of play event data;

And sequentially playing the sequenced multimedia resource data based on the target playing areas, and sequentially executing the playing event data.

Optionally, the sorting of the playback order of each multimedia resource data and each play event data according to a preset sorting mode specifically includes:

Respectively determining the corresponding time stamp of each multimedia resource data and the corresponding time stamp of each play event data;

And sequencing the playback sequence of each multimedia resource data and each play event data based on each determined time stamp.

Optionally, playing the sequenced multimedia resource data sequentially, and executing the playing event data sequentially, which specifically includes:

Determining multimedia resource data or play event data meeting the condition of the target time stamp from the sequenced multimedia resource data and the play event data based on the target time stamp in the acquired play progress control instruction;

And sequentially playing the sequenced multimedia resource data by taking the determined multimedia resource data or the determined playing event data as initial data, and sequentially executing the playing event data.

Optionally, the method further comprises:

responding to a play control request, and determining a play control type based on a control type identifier in the play control request, wherein the play control request at least comprises the control type identifier;

Generating a control instruction for the play control type;

and controlling the playing state of each piece of the sequenced multimedia resource data and each piece of the playing event data based on the control instruction.

A video playback device, for use with a client, comprising:

The system comprises an acquisition module, a storage module and a storage module, wherein the acquisition module is used for responding to a playback request aiming at a target video, acquiring each piece of multimedia resource data associated with the target video and at least one piece of play event data corresponding to each piece of multimedia resource data from a server, wherein the play event data are used for controlling the play state of the multimedia resource data, and each piece of multimedia resource data are obtained by storing data streams corresponding to different multimedia elements respectively;

The determining module is used for respectively determining target playing areas of the multimedia resource data according to the playing area layout information of the client;

And the playing module is used for playing the multimedia resource data based on the target playing areas and the playing event data.

Optionally, the system further comprises an association storage module, wherein the association storage module is used for:

Optionally, when the multimedia resource data and at least one play event data corresponding to each multimedia element of the video to be processed are respectively obtained, the associated storage module is specifically configured to:

Optionally, when each piece of multimedia resource data associated with the target video and at least one piece of play event data corresponding to each piece of multimedia resource data are acquired from a server, the acquiring module is specifically configured to:

Optionally, the determining module is specifically configured to:

acquiring the size parameter of a playing interface of the client;

Optionally, the playing module is specifically configured to:

Optionally, when the playback sequence of each multimedia resource data and each play event data is ordered according to a preset ordering mode, the play module is specifically configured to:

Optionally, the playing module is specifically configured to:

Optionally, the system further comprises a control module, wherein the control module is used for:

Generating a control instruction for the play control type;

An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the video playback method described above when the program is executed.

A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the video playback method described above.

The beneficial effects of the embodiment of the disclosure are as follows:

According to the video playback method, the video playback device, the electronic equipment and the storage medium, in response to a playback request for a target video, multimedia resource data associated with the target video and at least one play event data corresponding to the multimedia resource data are obtained from a server, target play areas of the multimedia resource data are respectively determined according to play area layout information of a client, and the multimedia resource data are played based on the target play areas and the play event data.

In this way, the obtained multimedia resource data and the obtained play event data are played in the target play area of the multimedia resource data, and the play size requirements of different play interfaces can be adapted in the playback process, so that the user experience is improved. And when the video is played back, each piece of multimedia resource data associated with the target video and at least one piece of play event data corresponding to each piece of multimedia resource data are acquired from the server, and the play of the multimedia resource data is controlled through the play event data, so that the playback scene can be restored and recorded more abundantly, and the user experience is improved.

Drawings

The above, as well as additional purposes, features, and advantages of exemplary embodiments of the present disclosure will become readily apparent from the following detailed description when read in conjunction with the accompanying drawings. Several embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which:

fig. 1 is an application scenario schematic diagram of a video playback method in an embodiment of the present disclosure;

FIG. 2 is a flow chart of a method of generating multimedia asset data and play event data in an embodiment of the present disclosure;

FIG. 3 is a flowchart of a method for obtaining multimedia asset data and playback event data in an embodiment of the present disclosure;

FIG. 4 is a flow chart of a video playback method in an embodiment of the present disclosure;

FIG. 5 is a flowchart of a method for searching multimedia asset data and playback event data in an embodiment of the present disclosure;

fig. 6 is a flowchart illustrating determining a target playing area according to an embodiment of the disclosure;

FIG. 7 is a first schematic diagram of a playback interface according to an embodiment of the disclosure;

FIG. 8 is a second schematic diagram of a playback interface according to an embodiment of the disclosure;

FIG. 9 is a flowchart of a method for playing multimedia asset data according to an embodiment of the present disclosure;

FIG. 10 is a flowchart of a method for ordering multimedia asset data and playback event data in an embodiment of the present disclosure;

FIG. 11 is a third schematic diagram of a playback interface according to an embodiment of the disclosure;

FIG. 12 is a flowchart of a method for modifying a playback progress in an embodiment of the present disclosure;

FIG. 13 is a flowchart of a method for controlling a play status in an embodiment of the present disclosure;

FIG. 14 is another flow chart of a video playback method in an embodiment of the present disclosure;

fig. 15 is a schematic structural view of a video playback apparatus in an embodiment of the present disclosure;

Fig. 16 is a schematic structural diagram of an electronic device in an embodiment of the disclosure.

Detailed Description

The following description of the technical solutions in the embodiments of the present disclosure will be made clearly and completely with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments that a person of ordinary skill in the art would obtain without making any inventive effort are within the scope of protection of this disclosure.

Embodiments of the present disclosure may be implemented as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the following forms, namely: complete hardware, complete software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.

In this document, it should be understood that any number of elements in the drawings is for illustration and not limitation, and that any naming is used only for distinction and not for any limitation.

For convenience of understanding, technical terms involved in the embodiments of the present disclosure are explained:

multimedia resource data: is data associated with the multimedia element stored during the live broadcast.

The multimedia resource data may be audio, interactive whiteboard, video, animation, document, etc., which is not limited in the embodiment of the present disclosure.

Playing event data: event data characterizing a play state for controlling multimedia asset data.

The play event data may be animation start, animation stop, visual restoration control, audio opening, etc., which is not limited in the embodiments of the present disclosure.

Playing area layout information: the layout information representing each playing area contained in the playing interface refers to setting the size and the position of a user window for displaying multimedia contents such as audio and video, an interactive whiteboard, a screen sharing video and the like on the electronic equipment and the platform through parameters agreed by a protocol.

It should be noted that, each play area is used for playing the multimedia resource data corresponding to one multimedia element.

Interactive whiteboard: the interactive whiteboard is mainly applied to an online classroom teaching scene, and a user can draw the key points while demonstrating content by means of functional tools such as graffiti, texts, lines, laser pens and the like.

The interactable component: the interactive component is a user-defined component which can be operated by a user, and can receive interaction feedback after the user operates, for example, a praise icon, the user can display praise animation effect when clicking, and meanwhile, the praise quantity is displayed and accumulated, for example, a questionnaire or a test question is displayed, and the user can interact through options and filling text.

Play speed multiplying power: the playback rate of the stream is generally 0.5, 1, 1.5, 2, etc. times the playback rate of the stream in the natural time, for example, the rate is 2 times, the playback takes 1 second, the playback rate of the record is 2 times, and the 1-time rate indicates the rate corresponding to the natural time.

Data: the method is used for storing and transmitting data concepts of audio and video, interactive white boards and events, and the data stream comprises information such as a start time stamp, a duration, an ID, a user UID, a data stream type and the like. The audio-video data stream comprises audio-video addresses or audio-video data, the interactive whiteboard data stream comprises interactive whiteboard playback history track data, and the playing event data comprises information such as event types, data content for executing playing events, event description and the like.

The principles and spirit of the present disclosure are explained in detail below with reference to several representative embodiments thereof.

Summary of The Invention

The inventors have found that live broadcast can be applied in many different scenarios, such as teaching, finance, conferencing, entertainment, etc. In the electronic equipment, the current online live broadcast scene mainly records videos, and a server generates standardized audio and video files for quality inspection, playback and archiving and leaving evidence. In the related art, in the live broadcast process, the live broadcast client transmits the acquired video stream, audio stream and the like to the server for storage, and the server generates a standardized audio/video file, so that in the playback process, the audio/video file is acquired from the server, and video playback can be performed. However, in this manner in the related art, since the audio/video file is generated by the server, only the size parameter of the playing interface of the playback client can be guaranteed, and the problem of losing video elements and guaranteeing that the video picture is not distorted can be avoided under the condition that the size parameter of the playing interface of the live client is the same as the size parameter of the playing interface of the live client. If video playback is required in different playback interfaces, the picture may be distorted due to the change of the size parameter (e.g., aspect ratio), which reduces the user experience.

In order to solve the above-mentioned problems, the present disclosure proposes a video playback method, apparatus, electronic device, and storage medium, in response to a playback request for a target video, each piece of multimedia resource data associated with the target video and at least one piece of play event data corresponding to each piece of multimedia resource data are obtained from a server, a target play area of each piece of multimedia resource data is determined according to play area layout information of a client, and each piece of multimedia resource data is played based on each target play area and each piece of play event data. In this way, in the embodiment of the present disclosure, each multimedia resource data is played in each target playing area, and each playing event data is executed, so that in the video playback process, the playing size requirements and customized viewing requirements of different playing interfaces can be adapted, thereby improving the user experience. In addition, by the method in the embodiment of the disclosure, the playing state of the multimedia resource data is controlled by playing the event data, and the live broadcast scene can be restored, so that the user experience is improved.

Having described the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are specifically described below.

Application scene overview

Referring to fig. 1, an application scenario diagram of a video playback method according to an embodiment of the present disclosure is shown. The application scenario may include a client 101 and a server 102, where the client 101 and the server 102 are connected through a communication network. The server 102 sends each multimedia asset data associated with the target video and at least one play event data corresponding to each multimedia asset data to the client 101, so that the client 101 determines a target play area of each multimedia asset data according to the play area layout information, and plays each multimedia asset data based on each target play area and each play event data, thereby realizing video playback.

The communication network may be a local area network, a wide area network, a mobile internet, etc., and the client 101 may be, but is not limited to,: the server 102 may be an independent physical server, or may be a cloud server that provides basic cloud computing services such as a cloud server, a cloud database, and a cloud storage, which is not limited in the embodiments of the present disclosure.

Exemplary method

A video playback method according to an exemplary embodiment of the present disclosure is described below in conjunction with the above application scenario. It should be noted that the above application scenario is only shown for the convenience of understanding the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in any way in this respect. Rather, embodiments of the present disclosure may be applied to any scenario where applicable.

First, a process of generating multimedia resource data and play event data in a live broadcast process in an embodiment of the present disclosure will be described in detail, with reference to fig. 2, which is a flowchart of a method of generating multimedia resource data and play event data in an embodiment of the present disclosure, and specifically includes:

s20: and respectively obtaining the multimedia resource data and at least one play event data corresponding to each multimedia element of the video to be processed.

In the embodiment of the disclosure, multimedia resource data and at least one play event data corresponding to each multimedia element of a video to be processed are respectively obtained.

Optionally, in the embodiment of the present disclosure, a possible implementation manner is provided for obtaining each piece of multimedia resource data and each piece of play event data, by storing a data stream of each piece of multimedia element and collecting a play event of each piece of multimedia element, each piece of multimedia resource data and each piece of play event data are obtained, referring to fig. 3, a flowchart of a method for obtaining each piece of multimedia resource data and each piece of play event data in the embodiment of the present disclosure is shown, and a detailed description is given below for a process for obtaining each piece of multimedia resource data and each piece of play event data in the embodiment of the present disclosure with reference to fig. 3, where the detailed description is provided below:

s201: and respectively storing data streams corresponding to the multimedia elements of the video to be processed to obtain the multimedia resource data.

In the embodiment of the disclosure, in a live broadcast process, since a video to be processed is composed of a plurality of multimedia elements, each multimedia element continuously generates data, so that data streams corresponding to the multimedia elements of the video to be processed are stored respectively, and thus, multimedia resource data generated in the live broadcast process are obtained.

The video to be processed is live video in a live broadcast process, and the multimedia resource data comprises various data generated by multimedia elements in the live broadcast process.

For example, assuming that the video to be processed is composed of audio, video, an interactive whiteboard and animation, in a live broadcast process, data streams corresponding to the audio are stored to obtain audio data, data streams corresponding to the video are stored to obtain video data, data streams corresponding to the interactive whiteboard are stored to obtain interactive whiteboard data, and data streams corresponding to the animation are stored to obtain animation data, wherein the audio data, the video data, the interactive whiteboard data and the animation data are all multimedia resource data.

It should be noted that, in the embodiment of the present disclosure, when data streams corresponding to different multimedia elements are stored, the data streams may be stored simultaneously; the storage may also be performed according to the presentation time of the multimedia element, for example, when the animation starts to be presented in the playing interface, the data stream corresponding to the animation is stored, which is not limited in the embodiment of the present disclosure.

In this way, the multimedia resource data in the embodiment of the disclosure includes animation, interface elements that can be operated by a user, and the like, so that the requirements of the user on animation and interaction can be met, and the user experience is improved.

S202: and respectively acquiring at least one play event corresponding to each multimedia element according to a preset data format to obtain at least one play event data.

In the embodiment of the disclosure, in a live broadcast process, at least one play event corresponding to each piece of multimedia element data is obtained according to a agreed data format, so as to generate at least one play event data containing a play event field.

It should be noted that, in the embodiment of the present disclosure, each play event data is composed of each play event field, where the play event field may be a unique user identification code, an event type, other data, and other basic information, and may also be an auxiliary attribute field such as a timestamp, which is not limited in the embodiment of the present disclosure.

For example, referring to table 1, a play event field meaning comparison table in an embodiment of the disclosure is shown.

Table 1.

Field name	Meaning of field
		userUid	User unique identification code
type	Event type
		Timetamp	Time stamp
data	Other data

The user unique identification code is used for uniquely identifying the identity information of the live broadcast object; the event type characterizes the type of the play event, such as that the animation starts to play, the control is invisible, etc.; the time stamp represents the occurrence time of the playing event and is used for aligning the playing time of the multimedia resource data so as to synchronize the playing progress; other data is used to supplement the additional data required for the execution of the play event, such as the animation content and the option operations of the interactable component, etc.

Further, in the embodiment of the present disclosure, the play event may be further distinguished according to the event type, and the table 2 is referred to as a play event type table in the embodiment of the present disclosure.

Table 2.

It should be noted that the play event in the embodiments of the present disclosure is not limited to the above types, and is not limited thereto.

S21: and sending the multimedia resource data and the play event data to a server for associated storage.

In the embodiment of the disclosure, each multimedia resource data and each play event data are sent to a server, so that the server generates a historical playback record based on the multimedia resource data and each play event data, generates a video identifier for the historical playback record, and stores the video identifier, each multimedia resource data and each play event data in a video database in association with each other, wherein a plurality of historical playback records are stored in the video database.

Therefore, each piece of to-be-processed video corresponds to each piece of multimedia resource data and each piece of play event data, each piece of to-be-processed video corresponds to a video identifier, and in the playback process, each piece of multimedia resource data and each piece of play event data associated with the video identifier can be determined to be played back according to the video identifier.

Further, in the embodiment of the present disclosure, the history playback record further includes a record ID, live room information, live layout parameters of a live interface, and the like, which is not limited in the embodiment of the present disclosure.

The live broadcast layout parameters represent live broadcast area information of each multimedia element of a live broadcast interface in a live broadcast process.

In the embodiment of the disclosure, when storing the video to be processed, the data streams of each multimedia element are respectively stored. In this way, when playing back video, redundant content is not included, and the storage volume and transmission bandwidth can be reduced.

Based on the foregoing embodiments, the embodiments of the present disclosure provide a video playback method, and referring to fig. 4, which is a flowchart of a video playback method in the embodiments of the present disclosure, specifically includes:

S40: and responding to the playback request aiming at the target video, acquiring each piece of multimedia resource data associated with the target video and at least one piece of play event data corresponding to each piece of multimedia resource data from a server.

The playing event data are used for controlling the playing state of the multimedia resource data, and each multimedia resource data is obtained by respectively storing data streams corresponding to different multimedia elements.

In the embodiment of the disclosure, when a user needs to play back a video, a target video to be played back may be selected in an operation interface of a client, a playback request for the target video is triggered in the client, the client responds to the triggered playback request, and sends the triggered playback request to a server, so that the server searches for each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data, and then the server returns the searched multimedia resource data and at least one play event data to the client, so that the client can acquire each multimedia resource data associated with the target video and at least one corresponding play event data from the server after responding to the playback request.

Optionally, in the embodiment of the present disclosure, when each piece of multimedia resource data and each piece of play event data are acquired from a server, each piece of multimedia resource data and each piece of play event data may be found in a database set by the server based on a target video identifier corresponding to a target video, and the following describes in detail the manner of acquiring each piece of multimedia resource data and each piece of play event data, as shown in fig. 5, a flowchart of a method for finding each piece of multimedia resource data and each piece of play event data in the embodiment of the present disclosure specifically includes:

s401: and sending the playback request to the server so that the server responds to the playback request and returns the searched multimedia resource data and at least one play event data corresponding to the multimedia data to the client.

The method comprises the steps of searching each multimedia resource data and each play event data associated with a target video identifier based on the target video identifier in a playback request.

In the embodiment of the disclosure, a playback request including a target video identifier is sent to a server, so that the server responds to the received playback request after receiving the playback request, obtains the target video identifier, searches for each piece of multimedia resource data and each piece of play event data associated with the target video identifier in a database based on the target video identifier, and then returns each piece of found multimedia resource data and each piece of play event data to a client.

For example, referring to table 3, in the embodiment of the disclosure, the table 3 includes each of the video identifiers A1, A2, and A3, and the multimedia asset data and the play event data associated with each of the video identifiers.

Table 3.

Assuming that the video identifier contained in the playback request is A2, based on the association relationship between the video identifier and the multimedia resource data and each play event data, the multimedia resource data Y1 and the play event data Y2 can be found, wherein the multimedia resource data Y1 comprises play control data and interactive whiteboard data; the play event data Y2 includes a visible play control, an interactive whiteboard start, and an interactive whiteboard stop, so that the play state of the multimedia asset data Y1 can be controlled using the play event data Y2 during video playback.

Optionally, in the embodiment of the present disclosure, the related multimedia resource data and the play event data may be queried based on the live room identifier, which is not limited in the embodiment of the present disclosure.

Further, in the embodiment of the present disclosure, when searching for each multimedia resource data and each play event data, searching may also be performed through a video name of the target video, which is not limited in the embodiment of the present disclosure.

S402: and receiving the multimedia resource data and the play event data sent by the server.

In the embodiment of the disclosure, after the server sends the searched multimedia resource data and the searched play event data to the client, the client receives the multimedia resource data and the play event data sent by the server.

S41: and respectively determining target playing areas of the multimedia resource data according to the playing area layout information of the client.

In the embodiment of the disclosure, the playing area layout information of the client is determined, and because the playing area layout information is used for indicating the playing area of each multimedia resource data in the playing interface, the target playing area of each multimedia resource data is respectively determined based on the playing area layout information.

It should be noted that in the embodiment of the present disclosure, the playing area layout information between different types of clients is different, for example, the playing area layout information between the mobile terminal and the portable computer is different, so, since the device and the platform used by the user in live broadcast and record playback generally include a tablet computer, a mobile phone, a smart tv, a PC, a browser, and the like, an appropriate playing area layout may be combined based on the characteristics of the platform and the device, so as to enhance the playback viewing experience of the user.

Optionally, in the embodiment of the present disclosure, the corresponding playing layout information may be determined according to the device identifier of the client, specifically, the device identifier of the client is determined, and the corresponding playing area layout information is determined according to the device identifier of the client and the association relationship between the device identifier and the playing area layout information.

Of course, when determining the corresponding layout information, the user may set the playing area layout information by himself, which is not limited in the embodiment of the present disclosure.

Optionally, in the embodiment of the present disclosure, since the sizes of the playing interfaces of the clients of the same type may be different, the playing area layout information of the client may be determined according to the size parameter of the playing interface of the client, and the process of determining the target playing area in the embodiment of the present disclosure will be described in detail below by taking the determining of the playing area layout information based on the size parameter as an example, referring to fig. 6, which is a schematic flow chart of determining the target playing area in the embodiment of the present disclosure, specifically includes:

S411: and acquiring the size parameter of the playing interface of the client.

In the embodiment of the disclosure, since the playing interface of each client has a size parameter, the interface size of the playing interface is identified, so as to obtain the size parameter of the playing interface.

The size parameter of the playing interface may be an aspect ratio, for example, referring to fig. 7, which is a first schematic diagram of the playing interface in the embodiment of the disclosure, as shown in fig. 7, the aspect ratio of the playing interface is 9:16; the dimensional parameters of the playing interface may also be width and height, for example, width 9 and height 16, which is not limited in the embodiments of the present disclosure.

S412: and determining playing area layout information of a playing interface of the client based on the size parameter.

The playing area layout information comprises playing areas of the multimedia elements in a playing interface.

In the embodiment of the disclosure, based on the size parameter and the association relationship between the size parameter and the playing area layout information, the playing area layout information associated with the size parameter of the playing interface is determined.

It should be noted that the playing area layout information includes the playing area of each multimedia element in the playing interface.

S413: and respectively determining target playing areas corresponding to the multimedia resource data according to the playing area layout information.

In the embodiment of the disclosure, the playing area layout information includes the position information of each target playing area on the playing interface, so that the target playing area corresponding to each multimedia resource data is determined according to the playing area layout information.

For example, referring to fig. 8, which is a second schematic diagram of a playing interface in the embodiment of the present disclosure, the multimedia elements are respectively an interactive whiteboard, a playing control, a rate control, a progress bar control, an a user video and a B user video, as can be seen from fig. 8, a playing area of the interactive whiteboard is an area 1, a playing area of the playing control is an area 2, a playing area of the rate control is an area 3, a playing area of the progress bar control is an area 4, a playing area of the a user video is an area 5, and a playing area of the B user video is an area 6.

Because the playing area of the interactive whiteboard is the area 1, in the playback process, playing interactive whiteboard data in the area 1; because the playing area of the playing control is area 2, in the playback process, playing control data in area 2; because the play area of the rate control is area 3, during playback, the rate control is played in area 3; because the playing area of the progress bar control is the area 4, in the playback process, the progress bar control data is played in the area 4; because the playing area of the A user video is the area 5, in the playback process, the A user video data is played in the area 5; since the play area of the B-user video is area 6, during playback, the B-user video data is played in area 6.

S42: and playing each multimedia resource data based on each target playing area and each playing event data.

In the embodiment of the disclosure, the multimedia resource data are played in the playing areas corresponding to the multimedia resource data respectively, and meanwhile, the playing event data are executed to control the playing state of the multimedia resource data, so that video playback is realized.

Specifically, in the embodiment of the present disclosure, a possible implementation manner is provided for playing each multimedia resource data, and referring to fig. 9, a flowchart of a method for playing each multimedia resource data in the embodiment of the present disclosure specifically includes:

S421: and sequencing the playback sequence of each multimedia resource data and each play event data according to a preset sequencing mode to obtain sequenced each multimedia resource data and each play event data.

In the embodiment of the disclosure, a preset ordering mode is determined, and the playback sequence of each multimedia resource data and each play event data is ordered based on the preset ordering mode, so that each multimedia resource data and each play event data after ordering are obtained.

The preset sorting manner may be, for example, sorting according to the size of the timestamp, or sorting according to the type priority of the multimedia resource data and each play event data, which is not limited in the embodiment of the present disclosure.

In the following, taking the sorting manner as an example of sorting according to the size of the timestamp, S421 in the embodiment of the disclosure is described in detail, referring to fig. 10, which is a flowchart of a method for sorting each multimedia resource data and each play event data in the embodiment of the disclosure, specifically including:

s4211: and respectively determining the corresponding time stamp of each multimedia resource data and the corresponding time stamp of each play event data.

In the embodiment of the disclosure, in the live broadcast process, each piece of stored multimedia resource data has a corresponding time stamp, and each piece of stored play event data also has a corresponding time stamp, so that the corresponding time stamp of each piece of multimedia resource data and the corresponding time stamp of each piece of play event data are obtained.

For example, the multimedia resource data is animation data, the time stamp corresponding to the animation is 00:00, the play event data corresponding to the animation data is respectively animation start and animation stop, wherein the time stamp corresponding to the animation start is 00:00, and the time stamp corresponding to the animation stop is 00:15.

It should be noted that, each multimedia resource data corresponds to a time stamp, and each play event data corresponds to a time stamp.

S4212: and sequencing the playback sequence of each multimedia resource data and each play event data based on each determined time stamp.

In the embodiment of the disclosure, the playback sequence of each multimedia resource data and each play event data is ordered based on the determined size of the timestamp, so as to obtain each ordered multimedia resource data and each play event data.

In the following, a specific example is adopted to describe the process of ordering each multimedia resource data and each play event data in detail in the embodiments of the present disclosure, for example, it is assumed that each multimedia resource data is animation data and interactive whiteboard data, a timestamp corresponding to the animation data is 00:00, play event data corresponding to the animation data is animation start and animation stop, a timestamp corresponding to the animation start is 00:00, a timestamp corresponding to the animation stop is 00:15, a timestamp corresponding to the interactive whiteboard data is 00:14, play event data corresponding to the interactive whiteboard data is interactive whiteboard start play and interactive whiteboard stop play, a timestamp corresponding to the interactive whiteboard start play is 00:14, and a timestamp corresponding to the interactive whiteboard stop play is 00:30. Therefore, based on the size of the time stamp, each multimedia resource data and each playing event data are ordered, and the data are animation data, animation start, interactive whiteboard data, interactive whiteboard start playing, animation stop and interactive whiteboard stop playing.

Wherein, since the animation data and the time stamp of the animation start are the same, the multimedia asset data animation data and the play event data animation start are played and executed simultaneously.

S422: and sequentially playing the sequenced multimedia resource data based on the target playing areas, and sequentially executing the playing event data.

In the embodiment of the disclosure, in each target playing area, each piece of sequenced multimedia resource data is played in sequence, and in each target playing area, each piece of playing event data is executed in sequence, so as to control the playing state of each piece of multimedia resource data.

For example, referring to fig. 11, a third schematic diagram of a playing interface in the embodiment of the disclosure is shown in fig. 11, in which in an interactive whiteboard playing area, playing interactive whiteboard data, and controlling a playing state of the interactive whiteboard data by playing event data, in an a user audio/video call area, playing a user audio/video call data, and controlling a playing state of the a user audio/video call data by playing event data, in an a user audio/video call area, playing B user audio/video call data, and controlling a playing state of the B user audio/video call data by playing event data, in a C user audio/video call area, playing control data, and controlling a playing state of the C user audio/video call data by playing event data, playing control data in a playing control area, and starting or pausing playing by playing event data, playing control data in a playing rate control area, and controlling a display and a size of a playing rate control by playing event data.

Optionally, in the embodiment of the present disclosure, when playing the multimedia resource data and executing the playing event data, the playing progress may be further modified, and in the embodiment of the present disclosure, the process of modifying the playing progress is described in detail, and referring to fig. 12, a flowchart of a method for modifying the playing progress in the embodiment of the present disclosure specifically includes:

s4221: and determining the multimedia resource data or the play event data meeting the condition of the target time stamp from the sequenced multimedia resource data and the play event data based on the target time stamp in the acquired play progress control instruction.

In the embodiment of the disclosure, after a play progress control instruction is acquired, in response to the play progress control instruction, since the play progress control instruction includes a target timestamp, each piece of multimedia resource data and each piece of play event data satisfying a target timestamp condition are determined from each piece of ordered multimedia resource data and each piece of play event data.

The multimedia data and the play event data meeting the target timestamp condition may be multimedia resource data or play event data corresponding to a timestamp with the minimum difference value between the target timestamps.

For example, assuming that the target timestamp is 00:15, two timestamps adjacent to the target timestamp are determined, namely, a timestamp 00:17 corresponding to the animation stop and a timestamp 00:14 corresponding to the interactive whiteboard data, so that a timestamp with the smallest difference value with the target timestamp is determined to be the timestamp 00:14 corresponding to the interactive whiteboard data, and the timestamp corresponds to the multimedia resource data, so that the determined multimedia resource data is acquired, the multimedia resource data is taken as initial data, and each piece of subsequent multimedia resource data and each piece of play event data are sequentially played from the multimedia resource data.

S4222: sequentially playing the sequenced multimedia resource data by taking the determined multimedia resource data or the determined playing event data as initial data, and sequentially executing the playing event data.

In the embodiment of the disclosure, the determined multimedia resource data or the determined play event data are used as the initial data, and each piece of sequenced multimedia resource data is sequentially played from the initial data, and each piece of play event data is sequentially executed.

Optionally, in the embodiment of the present disclosure, in the process of playing each multimedia resource data and each play event data, other play states of each multimedia resource data and each play event data may be controlled, for example, a play rate, a start operation, a pause operation, etc., and in the embodiment of the present disclosure, the process of controlling the play states of each multimedia resource data and each play event data is described below, and referring to fig. 13, a flowchart of a method for controlling the play states in the embodiment of the present disclosure is shown, which specifically includes:

S43: in response to the play control request, a play control type is determined based on the control type identification in the play control request.

Wherein, the play control request at least comprises a control type identifier.

In the embodiment of the disclosure, a play control request is generated, and a corresponding play control type is determined based on a control type identifier in the play control request in response to the play control request.

The control type may be, for example, start playing, pause, drag progress, stop, change of playing speed magnification, etc., which is not limited in the embodiment of the present disclosure.

S44: a control instruction for a playback control type is generated.

In the embodiment of the disclosure, the determined play control type is encapsulated, so as to generate a control instruction containing the play control type.

It should be noted that, in the embodiment of the present disclosure, the purpose of generating the control instruction is to control the playing states of the sequenced multimedia resource data and the playing event data, and what kind of control is specifically performed is determined by the playing control type in the control instruction.

S45: and controlling the playing state of each piece of the sequenced multimedia resource data and each piece of the playing event data based on the control instruction.

In the embodiment of the disclosure, based on the control instruction, the playing state of each piece of ordered multimedia resource data and each piece of playing event data is controlled.

For example, each multimedia asset data and each play event data are synchronously played and executed, and the progress is paused and changed, stopped, and the play speed magnification is changed.

In addition, it should be noted that the control operation may affect one or more multimedia asset data and the play event data at the same time, for example, start, pause, stop, change progress, play speed magnification, etc., which is not limited in the embodiment of the present disclosure.

In the embodiment of the disclosure, in the recording process, multimedia material elements including data such as audio, video, animation, documents, interactive whiteboard tracks, screen sharing recording and the like are stored, in addition, playing event data is synchronously recorded, when video is played back, playing event data different from audio and video is adopted, the playing event data and the multimedia resource data are used together, the multimedia resource data and the playing event data are synchronized, a live broadcast recording scene is restored, and content can be controlled and displayed in an interface area containing a multimedia window and other interface areas by executing the playing event data, so that the recorded playback scene is abundantly restored.

Based on the foregoing embodiments, referring to fig. 14, another flowchart of a video playback method according to an embodiment of the disclosure specifically includes:

S1400: starting.

S1401: and acquiring video time length, video address and video time stamp corresponding to the video data.

In the embodiment of the disclosure, in response to a playback request for a target video and sending the playback request to a server, the server may determine, based on a video identifier in the playback request, a video duration, a video address, and a video timestamp associated with the video identifier, and send the determined video duration, video address, and video timestamp to a client, where the video address is used to obtain video data.

S1402: and acquiring the interactive whiteboard data, the interactive whiteboard time length and the interactive whiteboard time stamp.

In the embodiment of the disclosure, the server may determine, based on the video identifier, interactive whiteboard data, interactive whiteboard duration, and interactive whiteboard timestamp associated with the video identifier, and send the interactive whiteboard data, interactive whiteboard duration, and interactive whiteboard timestamp to the client.

S1403: and acquiring each play event data, and the corresponding time stamp and event type.

In the embodiment of the disclosure, the video data and the interactive whiteboard data are respectively corresponding to the play event data, and the number and the types of the play event data are not limited.

S1404: and ordering the video data, the interactive whiteboard data and the playing event data according to the size of the time stamp to generate a playing queue.

In the embodiment of the disclosure, video data, interactive whiteboard data and each play event data are arranged from small to large based on the time stamp, so that a play queue is generated.

The playing queue comprises ordered video data, interactive whiteboard data and playing event data.

S1405: the timestamp with the smallest value is determined.

In the embodiment of the disclosure, the numerical value of each time stamp is determined, and the time stamp with the smallest numerical value is determined.

S1406: starting from the determined time stamp, the timing is started.

In the embodiment of the disclosure, video data, interactive whiteboard data or play event data corresponding to the timestamp with the smallest value is taken as a starting point, and timing is started.

S1407: and executing the drag progress operation.

In the embodiment of the disclosure, the drag progress operation corresponds to a target timestamp.

S1408: and searching corresponding data according to the target time stamp.

In the embodiment of the disclosure, the corresponding video data playing progress, interactive whiteboard data playing progress or playing event data is found according to the target timestamp.

S1409: if the data type is video data, S1410 is executed, if the data type is interactive whiteboard data, S1411 is executed, and if the data type is play event data, S1412 is executed.

In the embodiment of the disclosure, the data corresponding to the dragged time stamp can be determined based on the data type, if the data type is video data, the playing progress of the video data is modified, if the data type is interactive whiteboard data, the playing progress of the interactive whiteboard data is modified, and if the data type is playing event data, the playing process is skipped to the corresponding playing event data to start playing.

S1410: modifying the video data playing progress.

In the embodiment of the disclosure, if the data type is video data, the playing progress of the video data is modified, that is, the playing is started from the video data corresponding to the target timestamp.

S1411: and modifying the playing progress of the interactive whiteboard data.

In the embodiment of the disclosure, if the data type is interactive whiteboard data, the playing progress of the interactive whiteboard data is modified, that is, the interactive whiteboard data is played from the target timestamp.

S1412: jump to the corresponding play event data.

In the embodiment of the disclosure, if the data type is the play event data, the method jumps to the corresponding play event data and starts to execute the play event data.

S1413: continuing to play the video data, the interactive whiteboard data and the playing event data.

In the embodiment of the disclosure, after the corresponding video data, interactive whiteboard data or playing event data are determined, playing and executing are continued according to the ordered video data, interactive whiteboard data and playing event data.

S1414: and executing normal play operation.

In the embodiment of the disclosure, when playing is started, each multimedia resource data and each playing event data are played according to the same playing rate.

It should be noted that, if the normal play operation is performed, the video data, the interactive whiteboard data, and each play event data are ordered from the time stamp.

S1415: and playing the video data, the interactive whiteboard data and the playing event data according to the time stamp.

In the embodiment of the disclosure, the video data, the interactive whiteboard data and the playing event data are ordered according to the size of the time stamp, the ordered video data, the interactive whiteboard data and the playing event data are obtained, and the ordered video data, the ordered interactive whiteboard data and the ordered playing event data are sequentially played according to the time stamp.

S1416: whether all data are played is judged to be finished, if yes, S1417 is executed, and if not, S1415 is executed.

In the embodiment of the disclosure, whether all data are played is judged to be finished, if yes, the video playback is finished, and if not, the video data, the interactive whiteboard data and the playing event data are continuously played according to the time stamp.

S1417: and (5) ending.

In the embodiment of the disclosure, in the video playback process, the playback scene is generated by matching the multimedia resource data and the play event data, so that the size requirements and layout customization requirements of different clients on a play interface can be met, and the user experience is improved.

Exemplary apparatus

Based on the same inventive concept, a video playback device is also provided in the embodiments of the present disclosure, where the video playback device may be a hardware structure, a software module, or a combination of a hardware structure and a software module, and the embodiments of the video playback device may inherit the content described in the foregoing method embodiments, and based on the foregoing embodiments, as shown in fig. 15, is a schematic structural diagram of a video playback device 150 in the embodiments of the present disclosure. In one embodiment, video playback device 150 includes: an acquisition module 1501, a determination module 1502, a play module 1503, an associated storage module 1504, and a control module 1505.

An obtaining module 1501, configured to obtain, from a server, each piece of multimedia resource data associated with a target video and at least one piece of play event data corresponding to each piece of multimedia resource data in response to a playback request for the target video, where the play event data is used to control a play state of the multimedia resource data, and each piece of multimedia resource data is obtained by storing data streams corresponding to different multimedia elements respectively;

a determining module 1502, configured to determine, according to the playing area layout information of the client, target playing areas of the multimedia resource data respectively;

A playing module 1503, configured to play the multimedia resource data based on each target playing area and each playing event data.

Optionally, the system further includes an association storage module 1504, where the association storage module 1504 is configured to:

Optionally, when the multimedia resource data and at least one play event data corresponding to each multimedia element of the video to be processed are respectively obtained, the association storage module 1504 is specifically configured to:

Optionally, when each piece of multimedia resource data associated with the target video and at least one piece of play event data corresponding to each piece of multimedia resource data are acquired from a server, the acquiring module 1501 is specifically configured to:

Optionally, the determining module 1502 is specifically configured to:

acquiring the size parameter of a playing interface of the client;

Optionally, the play module 1503 is specifically configured to:

Optionally, when the playback order of the multimedia resource data and the playback event data is ordered according to a preset ordering manner, the playback module 1503 is specifically configured to:

Optionally, when the sequenced multimedia resource data are sequentially played, and the playing event data are sequentially executed, the playing module 1503 is specifically configured to:

Optionally, a control module 1505 is further included, where the control module 1505 is configured to:

Generating a control instruction for the play control type;

Based on the above embodiments, referring to fig. 16, a schematic structural diagram of an electronic device in an embodiment of the disclosure is shown.

Embodiments of the present disclosure provide an electronic device that may include a processor 1610 (Center Processing Unit, CPU), a memory 1620, an input device 1630, an output device 1640, and the like, where the input device 1630 may include a keyboard, a mouse, a touch screen, and the like, and the output device 1640 may include a display device, such as a Liquid crystal display (Liquid CRYSTAL DISPLAY, LCD), a Cathode Ray Tube (CRT), and the like.

The memory 1620 may include Read Only Memory (ROM) and Random Access Memory (RAM) and provides processor 1610 with program instructions and data stored in memory 1620. In the disclosed embodiments, the memory 1620 may be used to store a program of any of the video playback methods of the disclosed embodiments.

Processor 1610 is configured to execute any one of the video playback methods of the embodiments of the present disclosure in accordance with the obtained program instructions by calling the program instructions stored by memory 1620.

Based on the above embodiments, in the embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the video playback method in any of the method embodiments described above.

It will be apparent to those skilled in the art that embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to the disclosure. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present disclosure without departing from the spirit or scope of the disclosure. Thus, the present disclosure is intended to include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. A video playback method, applied to a client, comprising:

Determining playing area layout information of the client according to at least one of the type of the client, the equipment identifier and the size parameter of a playing interface, wherein the playing area layout information is used for adapting to the playing size requirement of the playing interface, and comprises playing areas of each multimedia element in the playing interface, and each playing area is used for playing multimedia resource data corresponding to one multimedia element;

Respectively determining target playing areas of the multimedia resource data according to the playing area layout information;

2. The method as recited in claim 1, further comprising:

3. The method according to claim 2, wherein the obtaining the multimedia resource data and the at least one play event data corresponding to each multimedia element of the video to be processed respectively specifically includes:

4. The method according to claim 1, wherein obtaining, from a server, each multimedia asset data associated with the target video, and at least one play event data corresponding to each multimedia asset data, specifically comprises:

Sending the playback request to a server, so that the server responds to the playback request and returns the searched multimedia resource data and at least one play event data corresponding to the multimedia resource data to a client;

5. The method of claim 1, wherein playing each multimedia asset data based on each target playing area and each playing event data, specifically comprises:

6. The method of claim 5, wherein the ordering the playback sequence of each multimedia asset data and each play event data according to a preset ordering manner, specifically comprises:

7. The method of claim 5, wherein playing each of the sequenced multimedia asset data sequentially, and sequentially executing each of the playing event data, specifically comprises:

sequentially playing the sequenced multimedia resource data by taking the determined multimedia resource data or the determined playing event data as initial data, and sequentially executing the playing event data.

8. The method of any one of claims 5-7, further comprising:

Generating a control instruction for the play control type;

9. A video playback device, for application to a client, comprising:

The determining module is used for determining playing area layout information of the client according to at least one of the type of the client, the equipment identifier and the size parameter of the playing interface, and respectively determining target playing areas of the multimedia resource data according to the playing area layout information; the playing area layout information is used for adapting to the playing size requirement of the playing interface, and comprises playing areas of each multimedia element in the playing interface, wherein each playing area is used for playing multimedia resource data corresponding to one multimedia element;

10. The apparatus of claim 9, further comprising an associative memory module to:

11. The apparatus of claim 10, wherein the association storage module is specifically configured to, when respectively obtaining the multimedia resource data and the at least one play event data corresponding to each multimedia element of the video to be processed:

12. The apparatus of claim 9, wherein, when each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data are acquired from a server, the acquiring module is specifically configured to:

13. The apparatus of claim 9, wherein the playing module is specifically configured to:

14. The apparatus of claim 13, wherein when the playback order of each multimedia asset data and each play event data is ordered according to a preset ordering manner, the play module is specifically configured to:

15. The apparatus of claim 13, wherein the playing module is specifically configured to, when sequentially playing the sequenced multimedia asset data and sequentially executing the playing event data:

16. The apparatus of any one of claims 13-15, further comprising a control module to:

Generating a control instruction for the play control type;

17. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of claims 1-8 when the program is executed by the processor.

18. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program implementing the steps of the method of any of claims 1-8 when executed by a processor.