CN112711459B

CN112711459B - Multimedia data processing method, device, equipment and computer readable storage medium

Info

Publication number: CN112711459B
Application number: CN202110053910.6A
Authority: CN
Inventors: 高嘉俊; 徐冬博; 肖鹏; 罗杰
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-01-15
Filing date: 2021-01-15
Publication date: 2022-06-28
Anticipated expiration: 2041-01-15
Also published as: CN112711459A

Abstract

The application provides a multimedia data processing method, a device, equipment and a computer readable storage medium; the method comprises the following steps: acquiring a multimedia data stream to be played, and playing a multimedia picture corresponding to the multimedia data stream on a playing interface; receiving an adjusting operation for adjusting a picture, and determining an adjusting parameter corresponding to the adjusting operation; adjusting the multimedia picture played by the playing interface based on the adjustment parameter, wherein the adjustment comprises zooming and/or moving, and the playing interface is kept unchanged; and playing the adjusted multimedia picture on the playing interface. By the method and the device, the multimedia picture can be zoomed and/or moved on the premise of keeping the playing interface unchanged, so that the interactivity and entertainment of multimedia playing are improved.

Description

Multimedia data processing method, device, equipment and computer readable storage medium

Technical Field

The present application relates to data processing technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for processing multimedia data.

Background

With the development of intelligent terminals and mobile communication technology, the mobile terminal has far exceeded the functions defined by the mobile terminal itself as ー communication tools, and more entertainment and interactivity are brought to the user, for example, the user can use the mobile terminal to perform online shopping, video chat, live broadcast or video watching anywhere and anywhere.

When a user watches live broadcast or video through the intelligent terminal, a video picture is limited by the screen size of the intelligent terminal, the video picture can only be as large as the screen size of the intelligent terminal to the maximum extent, the details of the video picture are difficult to see for the intelligent terminal with smaller screen size, and the interactivity and entertainment are poor.

Disclosure of Invention

The embodiment of the application provides a multimedia data processing method, a multimedia data processing device and a computer readable storage medium, which can zoom and/or move a multimedia picture on the premise of keeping a playing interface unchanged, so that interactivity and entertainment of multimedia playing are improved.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a multimedia data processing method, which comprises the following steps:

acquiring a multimedia data stream to be played, and playing a multimedia picture corresponding to the multimedia data stream on a playing interface;

receiving an adjusting operation for adjusting a picture, and determining an adjusting parameter corresponding to the adjusting operation;

adjusting the multimedia picture played by the playing interface based on the adjustment parameter, wherein the adjustment comprises zooming and/or moving, and the playing interface is kept unchanged;

and playing the adjusted multimedia picture on the playing interface.

An embodiment of the present application provides a multimedia data processing apparatus, including:

the first acquisition module is used for acquiring a multimedia data stream to be played and playing a multimedia picture corresponding to the multimedia data stream on a playing interface;

the first determining module is used for receiving an adjusting operation for adjusting the picture and determining an adjusting parameter corresponding to the adjusting operation;

the picture adjusting module is used for adjusting the multimedia picture played by the playing interface based on the adjusting parameter, wherein the adjustment comprises zooming and/or moving, and the playing interface is kept unchanged;

and the picture playing module is used for playing the adjusted multimedia picture on the playing interface.

In some embodiments, the screen playing module is further configured to:

determining a played target picture when an adjustment operation is received;

displaying the adjusted target picture on the playing interface until a preset time length is reached;

and playing other adjusted multimedia pictures on the playing interface, wherein the other multimedia pictures are a plurality of multimedia pictures behind the target picture.

In some embodiments, when the adjustment operation is a zoom operation, the adjustment parameter comprises a zoom scale, the apparatus further comprising:

a second determination module to determine a real-time scaling based on the adjustment operation during a continued contribution of the adjustment operation;

and the first presentation module is used for presenting the real-time zooming scale in the playing interface.

In some embodiments, when the adjusting operation is a moving operation, the apparatus further comprises:

the second acquisition module is used for acquiring the current scaling of the multimedia picture;

the second presentation module is used for keeping the current zoom scale unchanged during the continuous action of the adjustment operation and presenting the current zoom scale in the playing interface;

correspondingly, the picture playing module is further configured to: and playing the multimedia picture which is not presented before the mobile operation is received in the playing interface.

In some embodiments, the first determining module is further configured to:

when the adjustment operation is determined to be a zooming operation, acquiring a current zooming ratio;

acquiring the operation direction and the operation distance of the adjustment operation, and determining an adjustment proportion based on the operation direction and the operation distance;

and determining a target adjustment ratio based on the current scaling ratio and the adjustment ratio, and determining the target adjustment ratio as an adjustment parameter corresponding to the adjustment operation.

In some embodiments, the first determining module is further configured to:

determining a product of the current scaling and the adjusted scaling as a reference scaling;

when the reference scaling is smaller than a preset first scaling threshold, determining a target adjustment scale based on the first scaling threshold and the current scaling;

when the reference scaling is larger than a preset second scaling threshold value, determining a target adjustment scale based on the second scaling threshold value and the current scaling;

determining the adjustment ratio as a target adjustment ratio when the reference scaling ratio is greater than or equal to the first ratio threshold and the reference ratio is less than or equal to the second ratio threshold.

In some embodiments, the screen adjustment module is further configured to:

acquiring first position information of a first central point of the playing interface, and acquiring second position information of a second central point corresponding to the adjustment operation;

determining a first transformation matrix based on the first location information and the second location information, the first transformation matrix being used to move the first center point to a second center point;

determining a second transformation matrix based on the adjustment parameter, wherein the second transformation matrix is used for zooming the multimedia picture in the playing interface;

determining a third transformation matrix based on the first position information and the second position information, wherein the third transformation matrix is used for reducing the first central point to a position corresponding to the first position information;

and adjusting each pixel point in the multimedia picture based on a preset player transformation matrix, the first transformation matrix, the second transformation matrix and the third transformation matrix.

In some embodiments, the first determining module is further configured to:

when the adjustment operation is determined to be a moving operation, determining a moving direction and a moving distance of the adjustment operation;

determining a reference offset value based on the moving direction and the moving distance;

acquiring first size information of an operation response area and second size information of a multimedia picture played in a current playing interface;

determining a target offset value based on the reference offset value, the first size information, and the second size information, and determining the target offset value as an adjustment parameter.

In some embodiments, the reference offset value includes a reference width offset value and a reference height offset value, the first size information includes a first width and a first height of the manipulation response area, the second size information includes a vertex coordinate of the multimedia picture, a second width and a second height,

correspondingly, the first determining module is further configured to:

determining the difference between the first width and the second width as a first width coordinate threshold value, and determining the difference between the first height and the second height as a first height coordinate threshold value;

determining a sum of the reference width offset value and a width coordinate in the vertex coordinates as a reference width coordinate, and determining a sum of the reference width offset value and a height coordinate in the vertex coordinates as a reference width coordinate;

determining a preset value as a target width offset value when it is determined that the reference width coordinate exceeds the moving boundary based on the reference width coordinate, the first width coordinate threshold, and a preset second width coordinate threshold;

determining a preset value as a target height offset value when it is determined that the reference height coordinate exceeds the moving boundary based on the reference height coordinate, the first height coordinate threshold value, and a preset second height coordinate threshold value.

In some embodiments, the first determining module is further configured to:

determining the reference width offset value as a target width offset value when it is determined that the reference width coordinate does not exceed the moving boundary based on the reference width coordinate, the first width coordinate threshold value and a preset second width coordinate threshold value;

determining the reference height offset value as a target height offset when it is determined that the reference height coordinate does not exceed the movement boundary based on the reference height coordinate, the first height coordinate threshold, and a preset second height coordinate threshold.

In some embodiments, the picture adjustment module is further configured to:

determining a fourth transformation matrix based on the target offset value;

and adjusting the multimedia picture played by a playing interface by each pixel point in the multimedia picture based on a preset player transformation matrix and a fourth transformation matrix.

a memory for storing executable instructions;

and the processor is used for realizing the method provided by the embodiment of the application when executing the executable instructions stored in the memory.

Embodiments of the present application provide a computer-readable storage medium, which stores executable instructions for causing a processor to implement the method provided by the embodiments of the present application when the processor executes the executable instructions.

The embodiment of the application has the following beneficial effects:

after the terminal acquires the multimedia data stream to be played and plays the multimedia picture corresponding to the multimedia data stream on the playing interface, when receiving an adjusting operation for adjusting the picture, determining an adjusting parameter corresponding to the adjusting operation, the adjustment operation may be a zoom operation and/or a move operation, and then the multimedia picture played by the playing interface is zoomed and/or moved based on the adjustment parameter, and the playing interface remains unchanged, and play the zoomed and/or moved multimedia picture on the play interface, that is to say in the process of multimedia playing, the size of the playing interface can be kept unchanged, and the multimedia picture can be zoomed and/or moved, therefore, not only can the detail information in the multimedia picture be more conveniently watched, but also the interactivity and the entertainment of multimedia data playing can be improved.

Drawings

FIG. 1 is a network architecture diagram illustrating an architecture of a multimedia data processing system 100 according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of a terminal 400 according to an embodiment of the present disclosure;

fig. 3 is a schematic flowchart of an implementation of a multimedia data processing method according to an embodiment of the present application;

fig. 4 is a schematic flow chart illustrating an implementation process of playing an adjusted multimedia picture on a playing interface according to an embodiment of the present application;

fig. 5 is a schematic flowchart of another implementation of a multimedia data processing method according to an embodiment of the present application;

fig. 6 is a schematic flowchart of another implementation of multimedia data processing according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an interface for watching live broadcast in full screen;

FIG. 8 is a schematic view of an interface for displaying a zoom ratio when adjusting a screen according to an embodiment of the present disclosure;

FIG. 9 is a schematic view of an enlarged, simultaneously moving interface provided in an embodiment of the present application;

FIG. 10 is a schematic diagram of an interface for moving a screen after zooming in according to an embodiment of the present application;

fig. 11 is a schematic view of a playing interface after adjusting a screen according to an embodiment of the present application.

Detailed Description

In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the attached drawings, the described embodiments should not be considered as limiting the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the following description, references to the terms "first \ second \ third" are only to distinguish similar objects and do not denote a particular order, but rather the terms "first \ second \ third" are used to interchange specific orders or sequences, where appropriate, so as to enable the embodiments of the application described herein to be practiced in other than the order shown or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

Before further detailed description of the embodiments of the present application, terms and expressions referred to in the embodiments of the present application will be described, and the terms and expressions referred to in the embodiments of the present application will be used for the following explanation.

1) Gesture operation: refers to a finger action that can interact with an application;

2) the multimedia data stream is a code stream which is formed by coding the video and is suitable for transmission in a network, and the multimedia data stream can be instantly decoded and played by a receiving end without waiting for receiving all data.

3) The live broadcast service, the video and/or audio content released by the main user in the live broadcast room, has the limitation in time length, capacity and the like according to the service rule of the live broadcast platform.

An exemplary application of the multimedia data processing apparatus provided by the embodiment of the present application is described below, and the multimedia data processing apparatus provided by the embodiment of the present application may be implemented as various types of user terminals such as a notebook computer, a tablet computer, a desktop computer, a set-top box, a mobile device (e.g., a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device), and the like. In the following, an exemplary application will be explained when the device is implemented as a terminal.

Referring to fig. 1, fig. 1 is a schematic diagram of a network architecture of a multimedia data processing system 100 according to an embodiment of the present application, and as shown in fig. 1, the multimedia data processing system 100 includes a server 200, a network 300, and a terminal 400, where the terminal 400 is connected to the server 200 through the network 300, and the network 300 may be a wide area network or a local area network, or a combination of both.

When a user watches a video on line or watches a video live broadcast through the terminal 400, the terminal 400 acquires a multimedia data stream transmitted by the server 200 through the network 300 and plays a multimedia picture in a playing interface of the player, and during the playing process, if the adjustment operation for adjusting the multimedia picture is received, the terminal 400 zooms and/or moves the multimedia picture based on the adjustment operation, presents the zoomed and/or moved multimedia picture in the playing interface, scaling the multimedia picture may include enlarging or reducing the multimedia picture, where reducing may refer to reducing the multimedia picture after enlargement, without reducing the original multimedia picture, thereby enabling a more intuitive viewing of the details of interest and improving the interactivity and entertainment of the user when viewing multimedia data.

In some embodiments, the server 200 may be an independent physical server, may also be a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server that provides basic cloud computing services such as cloud services, a cloud database, cloud computing, cloud functions, cloud storage, a network service, cloud communication, middleware services, domain name services, security services, a CDN, and a big data and artificial intelligence platform. The terminal 400 may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited in the embodiment of the present invention.

Referring to fig. 2, fig. 2 is a schematic diagram of a composition structure of a terminal 400 according to an embodiment of the present application, where the terminal 400 shown in fig. 2 includes: at least one processor 410, memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together by a bus system 440. It is understood that the bus system 440 is used to enable communications among the components. The bus system 440 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 440 in fig. 2.

The Processor 410 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor, or the like.

The user interface 430 includes one or more output devices 431, including one or more speakers and/or one or more visual displays, that enable the presentation of media content. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like. Memory 450 optionally includes one or more storage devices physically located remote from processor 410.

The memory 450 includes either volatile memory or nonvolatile memory, and may include both volatile and nonvolatile memory. The nonvolatile memory may be a Read Only Memory (ROM), and the volatile memory may be a Random Access Memory (RAM). The memory 450 described in embodiments herein is intended to comprise any suitable type of memory.

In some embodiments, memory 450 is capable of storing data, examples of which include programs, modules, and data structures, or a subset or superset thereof, to support various operations, as exemplified below.

An operating system 451, including system programs for handling various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and handling hardware-based tasks;

a network communication module 452 for communicating to other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: bluetooth, wireless compatibility authentication (WiFi), and Universal Serial Bus (USB), etc.;

a presentation module 453 for enabling presentation of information (e.g., user interfaces for operating peripherals and displaying content and information) via one or more output devices 431 (e.g., display screens, speakers, etc.) associated with user interface 430;

an input processing module 454 for detecting one or more user inputs or interactions from one of the one or more input devices 432 and translating the detected inputs or interactions.

In some embodiments, the apparatus provided in the embodiments of the present application may be implemented in software, and fig. 2 illustrates a multimedia data processing apparatus 455 stored in the memory 450, which may be software in the form of programs and plug-ins, and the like, and includes the following software modules: the first acquiring module 4551, the first determining module 4552, the picture adjusting module 4553 and the picture playing module 4554, which are logical and thus may be arbitrarily combined or further divided according to the implemented functions.

The functions of the respective modules will be explained below.

In other embodiments, the apparatus provided in the embodiments of the present Application may be implemented in hardware, and for example, the apparatus provided in the embodiments of the present Application may be a processor in the form of a hardware decoding processor, which is programmed to execute the multimedia data processing method provided in the embodiments of the present Application, for example, the processor in the form of the hardware decoding processor may be one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components.

The multimedia data processing method provided by the embodiment of the present application will be described in conjunction with exemplary applications and implementations of the terminal provided by the embodiment of the present application.

Referring to fig. 3, fig. 3 is a schematic flow chart of an implementation of a multimedia data processing method according to an embodiment of the present application, and will be described with reference to the steps shown in fig. 3.

Step S101, obtaining a multimedia data stream to be played, and playing a multimedia picture corresponding to the multimedia data stream on a playing interface.

Here, when the terminal acquires the multimedia data stream to be played, the video data stream may be acquired from the server when the terminal watches a video online or a live broadcast online, or the multimedia data stream may be acquired from the local storage space when the terminal plays a locally stored multimedia file.

After the multimedia data stream is acquired, the terminal decodes the multimedia data stream to acquire each multimedia picture frame data of the multimedia data stream, and renders and plays the corresponding multimedia picture in the playing interface based on the multimedia picture frame data.

Step S102, receiving an adjusting operation for adjusting the picture, and determining an adjusting parameter corresponding to the adjusting operation.

Here, the adjustment operation is an operation of adjusting the multimedia screen in the playing interface, and is not an operation of adjusting the playing interface, that is, the adjustment operation is not an operation of displaying the playing interface in a full screen or in a non-full screen.

In the embodiment of the present application, the adjustment operation may be an operation of scaling the multimedia screen and/or an operation of moving the scaled multimedia screen. In practical implementation, the operation of zooming the multimedia screen may be a pinch-in or pinch-out operation using at least two fingers, and the operation of moving the multimedia screen may be a single-finger drag operation or a double-finger or multi-finger drag operation. When the zooming operation is triggered by two fingers, the moving directions of the two fingers are different; when the moving operation is triggered with two fingers, the moving directions of the two fingers are the same.

When the adjustment operation is an operation of scaling the multimedia picture, and when an adjustment parameter corresponding to the adjustment operation is determined, an adjustment ratio for adjusting the picture can be determined based on an operation direction and an operation distance of the adjustment operation, and a target adjustment ratio (i.e., an adjustment parameter) is determined according to a current scaling ratio and the adjustment ratio of the multimedia picture; when the adjustment operation is an operation of moving the multimedia screen, and when an adjustment parameter corresponding to the adjustment operation is determined, a target offset (i.e., an adjustment parameter) of moving the screen may be determined based on the operation direction and the operation distance of the adjustment operation.

And step S103, adjusting the multimedia picture played by the playing interface based on the adjusting parameters.

Here, the adjustment includes zooming and/or moving, and the playing interface remains unchanged. That is, in step S103, the multimedia screen is zoomed and/or moved without adjusting the playing interface.

When the step S103 is implemented, a transformation matrix for adjusting the multimedia picture may be determined by adjusting the parameter, and the multimedia picture data is adjusted based on the transformation matrix.

It should be noted that the adjustment operation may be one operation, for example, only a zoom operation, or only a move operation, or may be a combination of a series of operations, for example, zoom first and then move, or move first and then zoom then move, and the like. When the adjustment operation is a combination of a series of operations, sub-adjustment parameters corresponding to each sub-operation included in the adjustment operation are sequentially adjusted, and the multimedia picture is sequentially adjusted according to each sub-adjustment parameter.

And step S104, playing the adjusted multimedia picture on the playing interface.

Here, in step S103, all the multimedia frames played after receiving the adjustment operation are adjusted, so that in step S104, the respective zoomed and/or moved multimedia frames after receiving the adjustment operation are played in the playing interface.

In the multimedia data processing method provided in the embodiment of the application, after acquiring a multimedia data stream to be played, and playing a multimedia picture corresponding to the multimedia data stream on a playing interface, when receiving an adjustment operation for adjusting the picture, a terminal determines an adjustment parameter corresponding to the adjustment operation, where the adjustment operation may be a zoom operation and/or a move operation, and then zooms and/or moves the multimedia picture played on the playing interface based on the adjustment parameter, and the playing interface remains unchanged, and plays the zoomed and/or moved multimedia picture on the playing interface, that is, during the multimedia playing process, the size of the playing interface may remain unchanged, while zooming and/or moving the multimedia picture, so that not only can the detailed information in the multimedia picture be viewed more conveniently, and the interactivity and entertainment of multimedia data playing can be improved.

In some embodiments, the step S104 "play the adjusted multimedia screen on the play interface" shown in fig. 3 can be implemented by the steps S1041 to S1043 shown in fig. 4, and the steps are described below with reference to fig. 4.

In step S1041, the played target screen when the adjustment operation is received is determined.

Here, the adjustment operation is generally continued for a certain period of time, and the target picture is a multimedia picture played at the time when the adjustment operation is received.

Step S1042, presenting the adjusted target frame on the play interface until a preset duration is reached.

Here, the target picture corresponding to the moment when the adjustment operation is received is a picture that the user wants to zoom and/or move, and the zoomed and/or moved target picture has a viewing object that the user is interested in, so that in order to allow the user enough time to view the viewing object in the zoomed and/or moved target picture, the zoomed and/or moved target picture is continuously presented in the playing interface for a certain time period, for example, for 5 seconds or for 10 seconds.

And step S1043, playing the adjusted other multimedia pictures on the playing interface.

Here, the other multimedia pictures are multiple multimedia pictures after the target picture, that is, after the target picture is continuously presented for a certain time, the other multimedia pictures after the target picture are played at a normal frame rate.

In the embodiment of steps S1041 to S1043, after the adjustment operation is received and the multimedia picture is adjusted, the target picture played when the adjustment operation is received is continuously presented for a certain duration, so that the user has enough time to carefully watch the watching object in the target picture.

In some embodiments, the adjustment operation may be a zoom operation or may be a move operation. When the adjustment operation is a zoom operation, as shown in fig. 5, after step S104, the following steps may also be performed:

step S105A, determining a real-time scaling based on the adjustment operation during the continued functioning of the adjustment operation.

Here, when the adjustment operation is a zoom operation, for example, a gesture operation of pinching or expanding a double finger, the adjustment parameter, that is, the zoom ratio is changed in real time according to the distance of the pinching or expanding during the pinching or expanding of the finger, and thus the real-time zoom ratio can be determined based on the operation direction and the operation distance of the adjustment operation at this time.

Step S106A, presenting the real-time zoom ratio in the playing interface.

Here, when implemented, step S106A may be to present the real-time zoom ratio in a preset display area of the play interface.

In some embodiments, when the adjusting operation is a moving operation, as shown in fig. 5, after step S104, the following steps may also be performed:

in step S105B, the current scaling of the multimedia frame is obtained.

In the embodiment of the present application, after the multimedia screen is zoomed, the adjustment operation for moving the multimedia screen can be responded, that is, when the moving operation is received, the multimedia screen is not in the original size, and at this time, the current zoom ratio of the multimedia screen can be obtained.

Step S106B, during the continuous action of the adjustment operation, keeping the current zoom ratio unchanged, and presenting the current zoom ratio in the playing interface.

Here, since when the adjustment operation is a move operation, the move operation can only move the multimedia screen and cannot change the zoom ratio, the current zoom ratio is kept unchanged and presented in the play interface during the continuous action of the move operation. Similar to step S106A, when the current zoom ratio is presented in the play interface, the current zoom ratio may be presented in a preset display area in the play interface.

When the multimedia picture is enlarged and the playing interface is kept unchanged, the complete multimedia picture cannot be displayed in the playing interface, and when the user wants to watch the part of the multimedia picture which is not presented, the multimedia picture can be moved by triggering the moving operation, so that the multimedia picture which is not presented before the moving operation is received is presented in the playing interface. That is, when the adjustment operation is a moving operation, the step S104 "playing the adjusted multimedia picture on the playing interface" is to play the multimedia picture that is not shown before the moving operation is received on the playing interface when the step S104 "playing the adjusted multimedia picture on the playing interface" is implemented. In practical implementation, whether all the multimedia frames which are not displayed or part of the multimedia frames which are not displayed are displayed in the playing interface is determined based on the movement parameters corresponding to the movement operation.

Through the steps S105A and S106A and the steps S105B and S106B, during the continuous action of the adjustment operation, the real-time zoom ratio or the current zoom ratio is presented in the playing interface, so that the user can intuitively know the adjustment degree of the multimedia picture, and a subsequent adjustment strategy for the multimedia picture is determined. In some embodiments, when the adjustment operation no longer acts on the terminal, the zoom scale is no longer presented on the playing interface, thereby avoiding the occlusion of the multimedia picture when the zoom scale is presented.

In some embodiments, the step S102 of fig. 3 of "determining the adjustment parameter corresponding to the adjustment operation" includes:

in step S1021A, when it is determined that the adjustment operation is a zoom operation, a current zoom ratio is acquired.

Here, the current scaling is a scaling compared to the original multimedia screen, and when the setting is only capable of enlarging the original multimedia screen, the current scaling must be greater than or equal to 1, and when the setting is capable of enlarging and reducing the original multimedia screen, the current scaling may be greater than or equal to 1, or may be less than 1.

In the embodiment of the present application, it is assumed that the current scaling is 1.5, that is, the current multimedia picture is 1.5 times of the original multimedia picture.

Step S1022A, acquiring the operation direction and the operation distance of the adjustment operation, and determining the adjustment ratio based on the operation direction and the operation distance.

Here, step S1022, when implemented, may first determine whether to zoom in or zoom out a screen, for example, a pinch-in operation, as an operation to zoom out a screen, and a pinch-out operation, as an operation to zoom in a screen, according to the operation direction; and then determining the adjustment proportion according to the operation distance. In actual implementation, a first transformation relationship between the operation distance and the adjustment ratio when the picture is enlarged and a second transformation relationship between the operation distance and the adjustment ratio when the picture is reduced may be preset, and after the picture is determined to be enlarged or reduced, the adjustment ratio may be determined based on the corresponding transformation relationship and the operation distance.

In the embodiment of the present application, the adjustment ratio may be a numerical value greater than 1 or a numerical value less than 1, and when the adjustment ratio is a numerical value greater than 1, it indicates that the multimedia screen is to be continuously enlarged, and when the adjustment ratio is a numerical value less than 1, it indicates that the multimedia screen is to be reduced.

Step S1023A, determining a target adjustment ratio based on the current scaling ratio and the adjustment ratio, and determining the target adjustment ratio as an adjustment parameter corresponding to the adjustment operation.

In some embodiments, after the target adjustment ratio is determined, the product of the target adjustment ratio and the current scaling ratio may be updated to the latest current scaling ratio.

Here, step S1023A may be implemented by:

in step S23a1, a product of the current scaling and the adjustment scaling is determined as a reference scaling.

In step S23a2, it is determined whether the reference scaling ratio is smaller than a preset first scaling threshold.

Here, the first scale threshold is the minimum scale threshold, and when the reference scaling is smaller than the preset first scale threshold, the process proceeds to step S23a 3; when the reference scaling is greater than or equal to the first scaling threshold, the flow proceeds to step S23a 4.

Step S23a3, determining a target adjustment ratio based on the first ratio threshold and the current scaling ratio.

Here, since it is generally impossible to allow infinite enlargement or reduction when scaling the multimedia screen, the maximum scaling ratio and the minimum scaling ratio are set. When the reference scaling is smaller than the first scaling threshold, it is described that the reference scaling is already smaller than the set minimum scaling, and at this time, a value obtained by dividing the first scaling threshold by the current scaling is determined as a target adjustment scale, so that it is ensured that the scaling of the adjusted multimedia picture is not smaller than the minimum scaling after the multimedia picture is adjusted based on the target adjustment scale.

In step S23a4, it is determined whether the reference scaling is greater than a preset second scaling threshold.

Here, the second scale threshold is a maximum scale for limiting the scaling of the multimedia picture, and when the reference scale is larger than a preset second scale threshold, the step S23a5 is proceeded to; when the reference scaling is less than or equal to the second scaling threshold, the flow proceeds to step S23a 6.

Step S23a5, determining a target adjustment ratio based on the second ratio threshold and the current scaling ratio.

Here, when implemented, step S23a5 may be implemented by dividing the second ratio threshold by the current scaling ratio to determine the target adjustment ratio.

When the reference scaling is larger than the second scaling threshold, the reference scaling is already larger than the set maximum scaling, and at this time, a value obtained by dividing the second scaling threshold by the current scaling is determined as a target adjustment scaling, so that after the multimedia picture is adjusted based on the target adjustment scaling, the scaling of the adjusted multimedia picture is not larger than the maximum scaling.

Step S23a6, determining the adjustment ratio as a target adjustment ratio.

Here, when the reference scaling is less than or equal to the second scaling threshold and the reference scaling is greater than or equal to the first scaling threshold, it is described that the multimedia picture is adjusted by using the adjustment scale, and the scaling of the adjusted multimedia picture does not exceed the preset scaling range, that is, not greater than the maximum scaling threshold, nor is it less than the minimum scaling threshold, so that the adjustment scale does not need to be adjusted again, and at this time, the adjustment scale is directly determined as the target adjustment scale.

In steps S23a1 to S23a6, a scaling threshold, i.e., a minimum scaling and a maximum scaling, may be set in advance, and when it is determined that the reference scaling exceeds the scaling threshold range based on the adjustment scale and the current scaling, the target adjustment scale is determined by the minimum scaling or the maximum scaling, so as to ensure that the final scaling of the multimedia image is within the scaling threshold range.

In the embodiment of steps S1021A through S1023A, when the adjustment operation is a zoom operation, an adjustment ratio may be determined based on the operation direction and the operation distance, and a final target adjustment ratio may be determined based on the current zoom ratio and the adjustment ratio and a preset zoom ratio boundary, so that a subsequent adjustment process is performed based on the target adjustment ratio.

In some embodiments, after step S1022A, the following steps may also be performed:

step S1024A, it is determined whether the adjustment ratio is smaller than 1.

Here, when the adjustment ratio is smaller than 1, it is described that the current multimedia screen needs to be reduced, and the process proceeds to step S1025A; when the adjustment ratio is greater than 1, it indicates that the current multimedia screen needs to be enlarged, and then the process proceeds to step S1026A.

In step S1025A, it is determined whether the current scaling ratio is equal to the first scaling threshold.

Here, if the current scaling is equal to the first scaling threshold, it indicates that the current scaling has reached the minimum scaling, and no adjustment operation is responded; if the current scaling is larger than the first scaling threshold, which indicates that the current multimedia picture can be scaled down, then step S1023A is entered.

In step S1026A, it is determined whether the current scaling ratio is equal to the second ratio threshold.

Here, if the current scaling is equal to the second scaling threshold, it indicates that the current scaling has reached the maximum scaling, and no adjustment operation is responded; if the current zoom ratio is less than the second ratio threshold, it means that the current multimedia picture can be enlarged, and the process proceeds to step S1023A.

Correspondingly, when the adjusting operation is a zooming operation, the step S103 "adjusting the multimedia picture played on the playing interface based on the adjusting parameter" may be implemented by the following steps:

step S1031A obtains first position information of a first center point of the play interface, and obtains second position information of a second center point corresponding to the adjustment operation.

Here, the first position information, i.e., the coordinates of the first center point, is denoted as (playerX, playeryy), and the second position information, i.e., the coordinates of the second center point, is denoted as (pinchX, pinchY).

Step S1032A, a first transformation matrix is determined based on the first location information and the second location information.

Here, the first transformation matrix is used to transform the first image into the second imageThe first center point moves to the second center point, and in practical implementation, the first transformation matrix is

Step S1033A, a second transformation matrix is determined based on the adjustment parameter.

Here, the second transformation matrix is used for scaling the multimedia picture in the playing interface, and in practical implementation, the second transformation matrix is

Wherein scale is the target adjustment ratio.

Step S1034A, determining a third transformation matrix based on the first location information and the second location information.

Here, the third transformation matrix is used for reducing the first center point to a position corresponding to the first position information, and in actual implementation, the third transformation matrix is

Step S1035A, adjusting each pixel point in the multimedia picture based on a preset player transformation matrix, the first transformation matrix, the second transformation matrix, and the third transformation matrix.

Here, the preset player transform matrix is transform, and at this time, each pixel point in the multimedia picture can be adjusted by using formula (1-1):

wherein [ x y 1] is each pixel point in the multimedia picture before adjustment, and [ x 'y' 1] is each pixel point in the multimedia picture after adjustment.

Through the steps S1031A to S1035A, the plurality of pixels included in the multimedia image are scaled and transformed from one coordinate system to another coordinate system, and an adjusted multimedia image is obtained.

In some embodiments, when the adjusting operation is a moving operation, the step S102 of "determining the adjusting parameter corresponding to the adjusting operation" may further be implemented by:

step S1021B, when it is determined that the adjustment operation is a movement operation, determines a movement direction and a movement distance of the adjustment operation.

Step S1022B, a reference offset value is determined based on the moving direction and the moving distance.

Here, positive directions of the X direction and the Y direction may be preset, for example, a positive direction of the X direction toward the right and a positive direction of the Y direction toward the up may be set, and a transformation relationship between the operation distance and the offset value may also be set, when step S1022B is implemented, firstly, an offset value corresponding to the movement distance is determined according to the transformation relationship between the operation distance and the offset value, and then, based on the movement direction, a final reference offset value is determined, that is, if the movement direction is the positive direction, the calculated offset value is the reference offset value; if the direction of movement is the opposite direction, then the opposite of the calculated offset value is the reference offset value.

In step S1023B, first size information of the operation response area and second size information of the multimedia screen currently being played in the play interface are acquired.

Here, the first size information includes a first width and a first height of an operation response area, the operation response area is also an area where the playing interface is located, and the second size information includes a vertex coordinate, a second width, and a second height of the multimedia frame.

Step S1024B, determining a target offset value based on the reference offset value, the first size information and the second size information, and determining the target offset value as an adjustment parameter.

Here, the reference offset value includes a reference width offset value and a reference height offset value. Step S1024B may be implemented by:

step S24B1, determining the difference between the first width and the second width as a first width coordinate threshold value, and determining the difference between the first height and the second height as a first height coordinate threshold value;

a step S24B2 of determining a sum of the reference width offset value and a width coordinate in the vertex coordinates as a reference width coordinate, and determining a sum of the reference width offset value and a height coordinate in the vertex coordinates as a reference width coordinate;

and step S24B3, when the reference width coordinate is determined to exceed the moving boundary based on the reference width coordinate, the first width coordinate threshold and the preset second width coordinate threshold, determining the preset value as the target width deviation value.

Here, when the step S24B3 is implemented, when the moving operation is a forward movement in the width direction, that is, when the reference width offset value is a numerical value greater than 0, if the reference width coordinate is greater than or equal to the preset second width coordinate threshold value, it is described that the screen has been moved to the reverse most boundary, and at this time, the preset value is determined as the target width offset value, and when actually implemented, the preset value may be 0. Or when the moving operation is a reverse moving operation, that is, the reference width offset is a numerical value smaller than 0, if the reference width coordinate is smaller than or equal to the first width coordinate threshold, it indicates that the frame has been moved to the forward most boundary, and at this time, the preset value is also determined as the target width offset value.

And step S24B4, when the reference height coordinate is determined to exceed the moving boundary based on the reference height coordinate, the first height coordinate threshold and the preset second height coordinate threshold, determining the preset value as the target height offset value.

Here, the preset value may also be 0, that is, the target height offset value is 0 when it is determined that the reference height coordinate exceeds the movement boundary based on the reference height coordinate, the first height coordinate threshold value, and the preset second height coordinate threshold value.

In implementation, when the moving operation is forward movement in the height direction, that is, when the reference height offset value is a numerical value greater than 0, if the reference height coordinate is greater than or equal to the preset second height coordinate threshold value, it is described that the frame has been moved to the reverse frontmost boundary in the height direction, and at this time, the preset value is determined as the target height offset value, and in actual implementation, the preset value may be 0. Or when the moving operation is a reverse movement in the height direction, that is, the reference height offset amount is a numerical value smaller than 0, if the reference height coordinate is smaller than or equal to the first height coordinate threshold, it indicates that the screen has been moved to the forward most boundary in the height direction, and at this time, the preset value is also determined as the target height offset value.

And step S24B5, determining the reference width offset value as a target width offset value when it is determined that the reference width coordinate does not exceed the moving boundary based on the reference width coordinate, the first width coordinate threshold value, and a preset second width coordinate threshold value.

Here, when the moving operation does not exceed the moving boundary, then the reference width offset value is directly determined as the target width offset.

And step S24B6, when the reference height coordinate is determined not to exceed the moving boundary based on the reference height coordinate, the first height coordinate threshold and the preset second height coordinate threshold, determining the reference height offset value as the target height offset.

Correspondingly, when the adjustment operation is a moving operation, the step S103 "adjusting the multimedia picture played on the playing interface based on the adjustment parameter" may be implemented by:

in step S1031B, a fourth transformation matrix is determined based on the target offset value.

Here, the target offset value includes a target width offset value and a target height offset value, and in practical implementation, the fourth transformation matrix may be

Step S1032B, adjusting the multimedia image played by the playing interface based on the preset player transformation matrix and the fourth transformation matrix for each pixel point in the multimedia image.

Here, the preset player transform matrix is transform, and at this time, each pixel point in the multimedia picture can be adjusted by using formula (1-2):

Through the steps S1031B to S1032B, the plurality of pixels included in the multimedia frame are moved from one coordinate system to another coordinate system, so as to obtain an adjusted multimedia frame.

Next, an exemplary application of the embodiment of the present application in a practical application scenario will be described.

Fig. 6 is a schematic diagram of a further implementation process of multimedia data processing according to an embodiment of the present application, where as shown in fig. 6, the process includes:

in step S601, a touch gesture is detected in the multimedia playing process.

Step S602, determining whether the multimedia screen is in a zoom state.

Here, when the multimedia screen is in the zoom state, the process proceeds to step S603; when the multimedia screen is not in the zoom state, the process proceeds to step S606.

In step S603, it is determined whether the touch gesture is a single finger or a double finger.

Here, it is assumed in the embodiment of the present application that a single finger is a moving operation and a double finger is a zooming operation, and therefore when the touch gesture is a single finger, the process proceeds to step S604; when the touch gesture is a double finger, the process proceeds to step S605,

step S604, moving the screen based on the touch gesture.

Here, when implemented, the step S604 may determine a movement offset of the screen according to the movement distance of the touch gesture, and then move the screen based on the movement offset of the screen. After step S604, the process proceeds to step S609.

In step S605, the screen is zoomed and moved based on the touch gesture.

Here, when step S605 is implemented, an adjustment ratio may be determined according to a distance of pinch-in or pinch-out of two fingers, and then a final target adjustment ratio is determined according to a current zoom ratio and the adjustment ratio, and when the touch gesture includes a movement operation in addition to a pinch-in or pinch-out operation, a movement distance of a finger may also be determined, and then an offset of a screen is determined, and finally the screen is zoomed and moved according to the target adjustment ratio and the screen offset.

In the present embodiment, after step S605, the process proceeds to step S609.

In step S606, it is determined whether the touch gesture is a single finger or a double finger.

Here, in the embodiment of the present application, it is assumed that a single finger is a moving operation, and a double finger is a zooming operation, and when the touch gesture is a single finger, the process proceeds to step S607; when the touch gesture is a double finger, the process proceeds to step S608.

In step S607, the screen is not operated.

Here, when the touch gesture is detected and it is determined that the screen is not zoomed, it is described that the current screen and the screen are the same in size, and therefore, the movement adjustment cannot be performed, so when the touch gesture is a single-finger operation, the touch gesture is not responded, that is, the screen is not operated. After step S607, the process proceeds to step S609.

In step S608, the screen is zoomed and moved based on the touch gesture.

Here, similarly to the implementation process of step S605, the target adjustment ratio and the screen shift amount are determined based on the touch gesture in step S608, and the scaling and the movement of the screen are performed.

In step S609, when the finger lift is detected, the movement is stopped.

In the embodiment of the application, the picture is zoomed by monitoring the pinch gesture (PinchGesture), the picture is moved by monitoring the drag gesture (PanGesture), and the zooming and the moving are both realized by adopting a matrix transformation mode, and the principle is as follows:

the three-dimensional transformation matrix is recorded as:

transforming a set of points in space from a coordinate system by left-multiplying a vector by a transformation matrixTransformation to another coordinate system is shown in equation (2-1):

from equation (2-1), x '═ ax + cy + tx, y' ═ bx + dy + ty, where tx controls x-axis direction translation, ty controls y-axis direction translation, a controls x-axis direction scaling, d controls y-axis direction scaling, and a, b, c, d collectively control rotation.

In practical implementation, zooming the screen based on the touch gesture in step S605 can be implemented by:

step S6051, obtain the current zoom ratio, which is recorded as curScale, the pinch gesture zoom ratio, which is recorded as scale, and the preset zoom threshold value is recorded as [ minScale, maxScale ].

The current scaling currScale is a numerical value greater than or equal to 1; the pinch gesture zoom ratio may be understood as an adjustment ratio of a current zoom ratio, and the pinch gesture zoom ratio may be a real number greater than 1 or a real number less than 1, when the pinch gesture zoom ratio is greater than 1, it is considered that the current screen is continuously enlarged, and when the pinch gesture zoom ratio is less than 1, it is considered that the current screen is reduced.

For example, minScale can be preset to 1, maxScale can be preset to 2, that is, the multimedia picture cannot be reduced to a smaller scale than the original picture, and can be enlarged to 2 times of the original picture at most.

In step S6052, it is determined whether or not the zoom boundary is reached.

Here, when the following condition is satisfied, it is determined that the scaling boundary is reached, at which time scaling cannot be performed any more, and the process ends directly: (scale < 1 ^ curScale ═ minScale) (scale > 1 ^ curScale ═ maxScale).

In step S6053, the boundary scaling value is adjusted.

When implemented, the rule for adjusting the boundary scaling value is as follows:

when scale < 1 ^ curScale × scale < minScale, which indicates that the adjustment to the multimedia picture is less than the minimum scale, the adjustment value of scale is determined according to equation (2-2):

scale＝minScale÷curScale (2-2)；

for example, if the curScale is 1.6, the scale determined by the touch gesture is 0.5, the curScale × scale is 0.8, and the minScale is 1, then the scale is adjusted to 1/1.6, that is, to 0.625.

When scale > 1 ^ curScale × scale > maxScale indicates that the adjustment to the multimedia picture is larger than the maximum scale, the adjustment value of scale is determined according to the formula (2-3):

scale＝maxScale÷curScale (2-3)；

in step S6054, the center point of the rectangular area of the current player is obtained and recorded as (playerX, playerY).

In step S6055, a gesture center point is obtained and recorded as (pinchX, PinchY).

Here, since the scaling is a two-finger operation, the gesture center point may be an intermediate point determined based on the coordinates of the two finger action points.

Step S6056, acquiring a player transformation matrix, which is denoted as transform, and translating the zoom center to the gesture center through matrix transformation, where the transformation matrix is as shown in formula (2-4):

step S6057, scaling the scaling region by matrix transformation, where the transformation matrix is as shown in equation (2-5):

step S6058, restoring the center of the transformation matrix of the player through matrix transformation, wherein the transformation matrix is shown as a formula (2-6):

step S6059, the current scaling is updated according to equation (2-7).

curScale′＝curScale×scale (2-7)；

In this embodiment of the application, the step S604 "moving the screen based on the touch gesture" may be implemented by:

step S6041, acquiring a drag offset value, which is recorded as (translation x, translation y), a play area, which is recorded as (playrx, playry, playrw, playerH), and a gesture response size which is recorded as (geturew, gettrue H);

step S6042, adjusting the boundary values according to the following rules:

assuming that the gesture towards the right in the X direction is a positive direction gesture, the touch gesture towards the right generates a positive offset, the touch gesture towards the left generates a negative offset, and when the relationship X is more than 0 ^ planerX + relationship X is more than or equal to 0, the picture is moved to the left boundary, and the relationship X is set to be 0;

when the translation X is less than 0 ^ planerX + translation X is less than or equal to getureW-planerW, the picture is moved to the right boundary, and the translation X is set to be 0;

assuming that the upward gesture is a positive direction gesture in the Y direction, the upward touch gesture generates a positive offset, the downward touch gesture generates a negative offset, and when the translation Y is more than 0 ^ player + translation Y is more than or equal to 0, the picture is moved to the lower boundary, and the translation Y is set to be 0;

when translation y < 0 ^ planery + translation y ≦ getureh-planeh, it indicates that the screen has been moved to the upper boundary, and translation y is set to 0.

Step S6043, acquiring a player transformation matrix, and recording as transform;

step S6044, translating the player through matrix transformation, where the transformation matrix is as shown in formula (2-8):

the following describes a processing method of multimedia data provided in an embodiment of the present application with reference to drawings.

When a user watches a video, for example, when the user enters a live broadcast room to watch a live broadcast in a full screen mode, a playing interface of the terminal is as shown in fig. 7, and there is a limitation of a size of a mobile phone to watch a picture, so that some picture contents of the user are not clearly seen.

When a user wants to watch a certain point or a certain area of a live streaming picture carefully, the multimedia data processing method provided by the embodiment of the present application may be used to enlarge the live streaming picture through a gesture operation, and as shown in fig. 8, an enlargement ratio 801 is shown in the enlargement process, which is 160%, in the embodiment of the present application, the maximum enlargement ratio may be set to 200%.

When the picture is enlarged, the point of the finger and the touch screen is taken as the center of enlargement, and since the enlargement operation is a double-finger operation in the embodiment of the application, the middle point of the connection line of the two fingers can be determined as the center of enlargement, and after the picture is enlarged, the live stream can still be played normally.

As shown in fig. 9, the screen may be dragged to move up, down, left, and right during the process of enlarging the screen, as shown in fig. 10, after the screen is enlarged, the screen may be dragged to move up, down, left, and right by a single finger operation, and the screen still appears an enlarged scale during the movement, and disappears after the finger is released, and the screen size is fixed at the currently enlarged size, and still does not affect the live streaming.

No matter the picture is moved in the process of amplifying or after the picture is amplified, the moving range is limited by a boundary, and the user cannot move the picture continuously after pulling the boundary.

The user does not influence the playing of the live stream in the whole operation process, and after the user stops operating, the picture size is fixed to the picture after the user operates, as shown in fig. 11, the live picture can still be played.

By the multimedia data processing method provided by the embodiment of the application, a user can zoom and/or move a multimedia picture through gestures in the multimedia data playing process, exploration content display is realized through self-behavior operation of the user, and interestingness and audience participation are improved.

Continuing with the exemplary structure of the multimedia data processing apparatus 455 provided by the embodiment of the present application implemented as software modules, in some embodiments, as shown in fig. 2, the software modules stored in the multimedia data processing apparatus 455 of the memory 450 may include:

a first obtaining module 4551, configured to obtain a multimedia data stream to be played, and play a multimedia picture corresponding to the multimedia data stream on a play interface;

a first determining module 4552, configured to receive an adjustment operation for performing screen adjustment, and determine an adjustment parameter corresponding to the adjustment operation;

a picture adjusting module 4553, configured to adjust a multimedia picture played by the play interface based on the adjustment parameter, where the adjustment includes zooming and/or moving, and the play interface remains unchanged;

and the picture playing module 4554 is configured to play the adjusted multimedia picture on the playing interface.

In some embodiments, the picture playing module 4554 is further configured to:

determining a played target picture when an adjustment operation is received;

a second determination module for determining a real-time scaling based on the adjustment operation during a continuous effect of the adjustment operation;

the second acquisition module is used for acquiring the current zoom ratio of the multimedia picture;

In some embodiments, the first determining module is further configured to:

In some embodiments, the screen adjustment module is further configured to:

In some embodiments, the first determining module is further configured to:

when the adjustment operation is determined to be a movement operation, determining a movement direction and a movement distance of the adjustment operation;

correspondingly, the first determining module is further configured to:

determining the difference value between the first width and the second width as a first width coordinate threshold value, and determining the difference value between the first height and the second height as a first height coordinate threshold value;

In some embodiments, the first determining module is further configured to:

In some embodiments, the screen adjustment module is further configured to:

determining a fourth transformation matrix based on the target offset value;

and adjusting the multimedia picture played by the playing interface by each pixel point in the multimedia picture based on a preset player transformation matrix and a fourth transformation matrix.

Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the multimedia data processing method according to the embodiment of the present application.

Embodiments of the present application provide a computer-readable storage medium having stored therein executable instructions that, when executed by a processor, cause the processor to perform a method provided by embodiments of the present application, for example, the method as illustrated in fig. 3, 4 and 5.

In some embodiments, the computer-readable storage medium may be memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

In some embodiments, executable instructions may be written in any form of programming language (including compiled or interpreted languages), in the form of programs, software modules, scripts or code, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

By way of example, executable instructions may correspond, but do not necessarily have to correspond, to files in a file system, and may be stored in a portion of a file that holds other programs or data, such as in one or more scripts in a hypertext Markup Language (H TML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

By way of example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.

The above description is only an example of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for processing multimedia data, comprising:

receiving an adjusting operation for adjusting a picture, and acquiring a current zooming ratio when the adjusting operation is determined to be a zooming operation;

determining a target adjustment ratio based on the reference scaling ratio, the first ratio threshold and the second ratio threshold, and determining the target adjustment ratio as an adjustment parameter corresponding to the adjustment operation;

and playing the adjusted multimedia picture on the playing interface.

2. The method according to claim 1, wherein the playing the adjusted multimedia picture on the playing interface comprises:

determining a played target picture when an adjustment operation is received;

3. The method of claim 1, further comprising:

determining a real-time scaling based on the adjustment operation during a continued effort of the adjustment operation;

presenting the real-time zoom scale in the playback interface.

4. The method of claim 1, wherein when the adjusting operation is a moving operation, the method further comprises:

acquiring the current scaling of the multimedia picture;

during the continuous action of the adjusting operation, keeping the current zoom scale unchanged, and presenting the current zoom scale in the playing interface;

correspondingly, the playing the adjusted multimedia picture on the playing interface includes: and playing the multimedia picture which is not presented before the mobile operation is received on the playing interface.

5. The method of claim 1, wherein determining a target scaling ratio based on the reference scaling ratio, a first scaling threshold and a second scaling threshold comprises:

when the reference scaling is smaller than the preset first scaling threshold, determining the target adjustment scale based on the first scaling threshold and the current scaling;

when the reference scaling is larger than the preset second scaling threshold, determining the target adjustment scale based on the second scaling threshold and the current scaling;

determining the adjustment ratio as the target adjustment ratio when the reference scaling ratio is greater than or equal to the first ratio threshold and the reference scaling ratio is less than or equal to the second ratio threshold.

6. The method according to claim 1 or 5, wherein the adjusting the multimedia picture played by the playing interface based on the adjustment parameter comprises:

7. The method of claim 1, further comprising:

8. The method according to claim 7, wherein the reference offset value includes a reference width offset value and a reference height offset value, the first size information includes a first width and a first height of the operation response area, the second size information includes a vertex coordinate of the multimedia picture, a second width and a second height,

correspondingly, the determining a target offset value based on the reference offset value, the first size information and the second size information includes:

determining a preset value as a target height offset value when it is determined that the reference height coordinate exceeds the movement boundary based on the reference height coordinate, the first height coordinate threshold, and a preset second height coordinate threshold.

9. The method of claim 8, wherein determining a target offset value based on the reference offset value, the first size information, and the second size information comprises:

10. The method according to any one of claims 7 to 9, wherein the adjusting the multimedia picture played by the playing interface based on the adjustment parameter comprises:

determining a fourth transformation matrix based on the target offset value;

11. A multimedia data processing apparatus, comprising:

the device comprises a first determining module, a second determining module and a display module, wherein the first determining module is used for receiving an adjusting operation for adjusting the picture, and acquiring the current zooming ratio when the adjusting operation is determined to be the zooming operation; acquiring the operation direction and the operation distance of the adjustment operation, and determining an adjustment proportion based on the operation direction and the operation distance; determining a product of the current scaling and the adjusted scaling as a reference scaling; determining a target adjustment ratio based on the reference scaling ratio, the first ratio threshold and the second ratio threshold, and determining the target adjustment ratio as an adjustment parameter corresponding to the adjustment operation;

12. A multimedia data processing apparatus, characterized by comprising:

a memory for storing executable instructions;

a processor for implementing the method of any one of claims 1 to 10 when executing executable instructions stored in the memory.

13. A computer-readable storage medium having stored thereon executable instructions for, when executed by a processor, implementing the method of any one of claims 1 to 10.