CN103916620B - Video call method and device, mobile terminal - Google Patents

Abstract

The invention discloses a video call method, comprising: performing frame sampling on video data sent within a set period of time, and determining that the rate of change between the sampled video frames is less than or equal to a set threshold, no longer sending video data, The receiving end is notified to repeatedly play the video data in the previous set period. The invention also discloses a video call device and a mobile terminal. The technical scheme of the invention not only ensures good video call experience for users, but also greatly saves network resource occupation.

Description

Video call method and device, mobile terminal

technical field

The present invention relates to video communication technology, in particular to a video call method and device, and a mobile terminal.

Background technique

With the rapid development of third-generation mobile communications, more and more mobile terminals provide video call services. However, since video calls have high requirements on network bandwidth, they occupy a large amount of mobile communication network resources. When many people use it at the same time, the network transmission data will be large, which will affect the transmission quality of the video call and cause a decline in user experience.

At present, there are technologies that use certain methods to reduce the occupation of the network by video calls and ensure user experience. For example, in the prior art, when network congestion affects video data transmission, the user manually transmits still images to ensure the effect of video calls; Set still image. In the prior art, when the network transmission quality is poor, the still pictures are transmitted manually by the user, and the video call is converted into a still picture call, so as to reduce the occupation of network resources.

In the prior art, a still image is used instead of a video during a video call, which cannot provide a user with a real video call experience. In addition, the way in which the user manually completes the picture sending is poor in user experience.

Contents of the invention

In view of this, the main purpose of the present invention is to provide a video call method and device, and a mobile terminal, which can save wireless resources without transmitting video data when the call video is close.

In order to achieve the above object, technical solution of the present invention is achieved in that way:

A method for video calling, comprising:

Perform frame sampling on the video data sent within the set period, and when it is determined that the rate of change between the sampled video frames is less than or equal to the set threshold, no longer send video data, and notify the receiving end to repeatedly play the previous set period the video data within.

Preferably, the method also includes:

The received video data within the set period is cached, and when the repeat playback notification sent by the sending end is received, the cached video data within the set period is played.

Preferably, after notifying the receiving end to repeatedly play the video data within the previous set period, the method further includes:

Waiting for the set period, resending the currently captured video data, performing frame sampling on the video data sent in the next set period, and re-determining whether the rate of change between the sampled video frames is less than or equal to the set Set the threshold.

Preferably, the method also includes:

When it is determined that the rate of change between the sampled video frames is greater than or equal to or greater than the set threshold, continue to send the currently captured video data, and perform frame sampling on the video data sent within the next set period, and re-determine the sampled video Whether the rate of change between frames is less than or equal to the set threshold.

Preferably, the frame sampling of the video data sent within the set period includes:

Sampling the first video frame and the last video frame of the video data sent within the set period of time;

Or, sampling the video frames sent within the set period of time in an equal interval;

Alternatively, the video frames sent within the set period of time are sampled at equal intervals, and the sampled video frames include the first video frame and the last video frame of the video data.

Preferably, said determining that the rate of change between sampled video frames is less than or equal to a set threshold includes:

When it is determined that the brightness difference between every two video frames sampled is less than or equal to the set threshold, it is determined that the rate of change between the sampled video frames is less than or equal to the set threshold;

Or, compare the gray levels of the pixels at the set positions of the video frames respectively, and when the gray level difference of the pixels at each set position is less than or equal to the set threshold, determine the difference between the sampled video frames The rate of change is less than or equal to the set threshold.

A video call device, including a sending unit, a sampling unit, a determining unit and a notification unit, wherein:

a sending unit, configured to send video data;

The sampling unit is used to perform frame sampling on video data sent within a set period of time;

A determining unit, configured to determine whether the rate of change between sampled video frames is less than or equal to a set threshold, and trigger the sending unit to no longer send video data, and trigger the notification unit;

The notification unit is used to notify the receiving end to repeatedly play the video data in the previous set period.

Preferably, the device further includes a receiving unit, a buffer unit and a playback unit, wherein:

The receiving unit is used to receive video data within the set period of time; and receives the repeat playback notification sent by the sending end, and triggers the playback unit;

a buffer unit, configured to buffer video data within a set period of time received by the receiving unit;

The playing unit is used to play the video data within the set period of time in the cache.

Preferably, the device also includes:

The timing unit is used to wait for the set period of time after the notification unit notifies the receiving end to repeatedly play the video data in the previous set period, and re-trigger the sending unit to send the currently captured video data, the The sampling unit performs frame sampling on the video data sent in the next set period, and the determination unit re-determines whether the rate of change between the sampled video frames is less than or equal to the set threshold.

Preferably, when the determination unit determines that the rate of change between the sampled video frames is greater than or equal to or greater than a set threshold, the sending unit is triggered to continue sending the currently captured video data, and the sampling unit performs the next setting The video data sent within a certain period of time is subjected to frame sampling, and the determining unit re-determines whether the rate of change between the sampled video frames is less than or equal to a set threshold.

Preferably, the sampling unit is further configured to sample the first video frame and the last video frame of the video data sent within the set period; or, sample the video frames sent within the set period at equal intervals; or , sampling the video frames sent within the set time period at equal intervals, where the sampled video frames include the first video frame and the last video frame of the video data.

Preferably, the determining unit is further configured to determine that when the brightness difference between every two sampled video frames is less than or equal to a set threshold, determine that the rate of change between the sampled video frames is less than or less than or equal to a set threshold threshold;

Compare the gray levels of the pixels at the set positions of the video frames respectively, and determine the rate of change between the sampled video frames when the gray level difference of the pixels at each set position is less than or equal to the set threshold Less than or equal to the set threshold.

A mobile terminal includes the aforementioned video call device.

In the present invention, frame sampling is performed on the video data sent within a set period of time, and when it is determined that the rate of change between the sampled video frames is less than or equal to the set threshold, the video data is no longer sent, and the receiving end is notified to repeatedly play the previous video. The video data within the set period described above. According to the technical solution of the present invention, when the user is in the same posture as the mobile terminal during the call, the videos taken during the call should be basically the same, at this time, the sending of the video data can be suspended; and when the video When one party to the call moves frequently and the update frequency of the video data is faster, new video information needs to be sent all the time; thus, the technical solution of the present invention not only ensures a good video call experience for the user, but also greatly saves the occupation of network resources.

Description of drawings

Fig. 1 is the flowchart of the video calling method of the embodiment of the present invention;

FIG. 2 is a schematic diagram of the composition and structure of a video call device according to an embodiment of the present invention.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail by citing the following embodiments and referring to the accompanying drawings.

During a video call, usually, the two parties in the call talk to the camera, the image captured by the camera does not change much, the video image seen by the two parties in the call is almost unchanged, and the transmitted video information has a lot of redundancy. Based on this In the video call method of the embodiment of the present invention, the video transmission is carried out uninterruptedly only when the frequency of the video captured by the camera changes rapidly. The present invention detects the video frames collected by the camera at regular intervals. When it is found that the change between two video frames is lower than the set threshold within a certain period of time, it is considered that the call video has not changed, and the sending end suspends the video information transmission, and the receiving end The terminal repeatedly plays the video content of the previous period; when the change between two video frames is higher than the threshold, it is considered that the call video has changed, and the video needs to be sent in real time. During the whole process, voice transmission is not affected. The video call method of the embodiment of the present invention can suspend the transmission of video information when the user is in the same posture during the video call process, which not only ensures the user's good video call experience, but also greatly saves the occupation of network resources.

Fig. 1 is the flow chart of the video call method of the embodiment of the present invention, as shown in Fig. 1, the video call method of this example comprises the following steps:

Step 101, the two parties in the call establish a video call.

In this step, the mobile terminals of the two parties that support the video call enable the video call function, and start the video call.

Step 102, both parties in the call send video to each other;

Video calls are two-way, and both parties need to send video data to each other.

In step 103, it is determined that both parties in the conversation continue, and if so, execute step 104, otherwise end the current processing flow.

Step 104, sampling the sent video data.

In this step, the main means of sampling video data are: the first video frame and the last video frame of the video data sent in the sampling setting period such as T; that is, only the first frame and the last frame of the video data sent in T are sampled, by For the change of the first frame and the last frame of the video within T, determine whether the rate of change of the video exceeds the set threshold.

Alternatively, the video frames sent within the set period T are sampled at equal intervals. That is, the video frames in T are sampled according to the time interval of T/n, and n sampling points are obtained, and whether the change rate of the video frames corresponding to the n sampling points exceeds the set threshold is determined to determine whether the video in T occurs Variety.

Alternatively, the video frames sent within the set time period T are sampled at equal intervals, and the sampled video frames include the first video frame and the last video frame of the video data. This method is basically similar to the aforementioned method of sampling video frames at equal intervals, but when the sampling points do not include the first frame and the last frame, the first frame and the last frame also need to be used as sampling points.

In the present invention, T can be set to 1 second, 3 seconds or 5 seconds, etc. Of course, it can also be set to other time values.

Step 105, judge whether the change of the sampled video frame exceeds the threshold, if yes, execute step 107; otherwise, execute step 106.

In the present invention, the brightness change between sampled video frames is used as the basis for judging whether the video frame changes. Specifically, when it is determined that the brightness difference between every two sampled video frames is less than or equal to the set threshold, it is determined that the rate of change between the sampled video frames is less than or equal to the set threshold.

Or, compare the gray levels of the pixels at the set positions of the video frames respectively, and when the gray level difference of the pixels at each set position is less than or equal to the set threshold, determine the difference between the sampled video frames The rate of change is less than or equal to the set threshold. For example, select 5 position points at the center position and four corners of the video frame, and obtain the gray levels of the pixels at the corresponding 5 positions for every two video frames sampled, and the gray level difference at the 5 positions If they are all less than or equal to the set threshold value, it is determined that the video frames are the same; otherwise, it is determined that the video frames are different.

Of course, other indicators of the video frames can also be used to determine the changes between the video frames, and the present invention does not limit the manner of determining whether the video frames change.

Step 106, clear frame records, and return to step 102.

In this step, the currently cached video record is deleted to re-cache the newly sent video data.

Step 107, suspend the video transmission, and notify the calling party to repeat playing the video data in the previous set period T.

When it is determined that the video has basically no change, the currently captured video is no longer sent to the mobile terminal of the calling party, but the video data in the previous set period T is notified to be played repeatedly. In the present invention, the mobile terminal as the receiver needs to buffer the received video data, so as to repeatedly play the buffered video data after receiving the repeat playback notification from the sender.

In the present invention, when the mobile terminal of the party sending the video data determines that the video has not changed much, it does not need to notify the mobile terminal of the other party, and the mobile terminal of the other party can repeat the video immediately after not receiving the video stream of the other party's mobile terminal. Play the previously cached video data in T.

Step 108, clear frame records, wait for a set period of time T, and return to step 103 to resend the video data currently captured by the mobile terminal.

In the present invention, although the video changes in T are small, it cannot guarantee that the video remains unchanged for a long period of time, and if the same video is played repeatedly, the call experience will be poor. Therefore, repeated video playback After that, the currently captured video will be sent to the mobile terminal of the other party again.

FIG. 2 is a schematic diagram of the composition and structure of a video calling device according to an embodiment of the present invention. As shown in FIG. 2 , the video calling device of this example includes a sending unit 20, a sampling unit 21, a determining unit 22, and a notification unit 23, wherein:

A sending unit 20, configured to send video data;

The sampling unit 21 is used to perform frame sampling on the video data sent within the set period of time;

A determining unit 22, configured to determine whether the rate of change between the sampled video frames is less than or equal to a set threshold, and trigger the sending unit to no longer send video data, and trigger the notification unit 23;

The notification unit 23 is configured to notify the receiving end to repeatedly play the video data within the previous set period.

On the basis of the video call device shown in Figure 2, the video call device of this example also includes a receiving unit (not shown in Figure 2), a buffer unit (not shown in Figure 2) and a playback unit (not shown in Figure 2 shown), where:

The receiving unit is used to receive video data within the set period of time; and receives the repeat playback notification sent by the sending end, and triggers the playback unit;

a buffer unit, configured to buffer video data within a set period of time received by the receiving unit;

The playing unit is used to play the video data within the set period of time in the cache.

On the basis of the video calling device shown in Figure 2, the video calling device of this example also includes:

A timing unit (not shown in FIG. 2 ), used to wait for the set period after the notification unit notifies the receiving end to repeatedly play the video data in the previous set period, and re-trigger the sending unit 20 Send the currently shot video data, the sampling unit 21 performs frame sampling on the video data sent in the next set period, and the determination unit 22 re-determines whether the rate of change between the sampled video frames is less than or equal to Set the threshold.

When the determination unit determines that the rate of change between the sampled video frames is greater than or equal to or greater than the set threshold, the sending unit 20 is triggered to continue sending the currently captured video data, and the sampling unit 21 sets the next The video data sent within the period is subjected to frame sampling, and the determination unit 22 re-determines whether the rate of change between the sampled video frames is less than or equal to a set threshold.

The above-mentioned sampling unit is also used to sample the first video frame and the last video frame of the video data sent within the set period; or, sample the video frames sent within the set period at equal intervals; or, at equal intervals The method samples the video frames sent within the set time period, and the sampled video frames include the first video frame and the last video frame of the video data.

The above determination unit 22 is also used to determine that the rate of change between the sampled video frames is less than or equal to the set threshold when the brightness difference between every two sampled video frames is less than or equal to the set threshold;

Compare the gray levels of the pixels at the set positions of the video frames respectively, and determine the rate of change between the sampled video frames when the gray level difference of the pixels at each set position is less than or equal to the set threshold Less than or equal to the set threshold.

Those skilled in the art should understand that the functions of the processing unit involved in the video calling device shown in FIG. 2 can be realized by hardware circuits, or by executing corresponding software by a processor. The functions of the above processing units can be understood in combination with the descriptions of the above-mentioned related embodiments and examples of the video call method of the present invention.

The present invention also records a mobile terminal, including the flow control device shown in FIG. 2 .

Obviously, those skilled in the art should understand that each processing unit or each step of the present invention described above can be implemented by a general-purpose computing device, which can be concentrated on a single computing device, or distributed across a plurality of computing devices. Optionally, it can be implemented with program codes executable by a computing device, thus, they can be stored in a storage device and executed by a computing device, or they can be made into individual integrated circuit modules, or they can be A plurality of modules or steps in the process are realized by making a single integrated circuit module. As such, the present invention is not limited to any specific combination of hardware and software.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention.

Claims (11)

1. A video call method, characterized in that the method comprises: During the video call, frame sampling is performed on the video data sent within the set period, and when the change rate between the sampled video frames is determined to be less than or equal to the set threshold, no more video data is sent, and the receiving end is notified to play the previous one repeatedly video data within the set time period; Waiting for the set period, resending the currently captured video data, performing frame sampling on the video data sent in the next set period, and re-determining whether the rate of change between the sampled video frames is less than or equal to the set Set the threshold. 2. The method according to claim 1, characterized in that the method further comprises: The received video data within the set period is cached, and when the repeat playback notification sent by the sending end is received, the cached video data within the set period is played. 3. The method according to claim 1, characterized in that the method further comprises: When it is determined that the rate of change between the sampled video frames is greater than or equal to or greater than the set threshold, continue to send the currently captured video data, and perform frame sampling on the video data sent within the next set period, and re-determine the sampled video Whether the rate of change between frames is less than or equal to the set threshold. 4. The method according to claim 1, 2 or 3, wherein the frame sampling of the video data sent within the set period of time comprises: Sampling the first video frame and the last video frame of the video data sent within the set period of time; Or, sampling the video frames sent within the set period of time in an equal interval; Alternatively, the video frames sent within the set period of time are sampled at equal intervals, and the sampled video frames include the first video frame and the last video frame of the video data. 5. The method according to claim 1, 2 or 3, wherein the rate of change between the determined sampled video frames is less than or equal to a set threshold, comprising: When it is determined that the brightness difference between every two video frames sampled is less than or equal to the set threshold, it is determined that the rate of change between the sampled video frames is less than or equal to the set threshold; Or, compare the gray levels of the pixels at the set positions of the video frames respectively, and when the gray level difference of the pixels at each set position is less than or equal to the set threshold, determine the difference between the sampled video frames The rate of change is less than or equal to the set threshold. 6. A video call device, characterized in that the device comprises a sending unit, a sampling unit, a determining unit and a notification unit, wherein: a sending unit, configured to send video data; The sampling unit is used to perform frame sampling on the video data sent within the set period during the video call; A determining unit, configured to determine whether the rate of change between sampled video frames is less than or equal to a set threshold, and trigger the sending unit to no longer send video data, and trigger the notification unit; The notification unit is used to notify the receiving end to repeat playing the video data in the previous set period; The device also includes: The timing unit is used to wait for the set period of time after the notification unit notifies the receiving end to repeatedly play the video data in the previous set period, and re-trigger the sending unit to send the currently captured video data, the The sampling unit performs frame sampling on the video data sent in the next set period, and the determination unit re-determines whether the rate of change between the sampled video frames is less than or equal to the set threshold. 7. The device according to claim 6, further comprising a receiving unit, a buffer unit and a playback unit, wherein: The receiving unit is used to receive video data within the set period of time; and receives the repeat playback notification sent by the sending end, and triggers the playback unit; a buffer unit, configured to buffer video data within a set period of time received by the receiving unit; The playing unit is used to play the video data within the set period of time in the cache. 8. The device according to claim 6, wherein when the determination unit determines that the rate of change between sampled video frames is greater than or equal to or greater than a set threshold, the sending unit is triggered to continue sending the currently captured video data, the sampling unit performs frame sampling on the video data sent in the next set period, and the determination unit re-determines whether the rate of change between the sampled video frames is less than or equal to the set threshold. 9. The device according to claim 6, 7 or 8, wherein the sampling unit is also used to sample the first video frame and the last video frame of the video data sent within the set period of time; or, with Sampling the video frames sent in the set period in equal intervals; or sampling the video frames sent in the set period in equal intervals, including the first video frame and the tail video frame of the video data in the sampled video frames . 10. The device according to claim 6, 7 or 8, wherein the determining unit is further configured to determine that when the brightness difference between every two video frames sampled is less than or equal to a set threshold, Determine that the rate of change between sampled video frames is less than or equal to a set threshold; Compare the gray levels of the pixels at the set positions of the video frames respectively, and determine the rate of change between the sampled video frames when the gray level difference of the pixels at each set position is less than or equal to the set threshold Less than or equal to the set threshold. 11. A mobile terminal, characterized in that the mobile terminal comprises the video call device according to any one of claims 6 to 10.