JPWO2020095728A1 - Information processing device and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device and an information processing method capable of reducing a feeling of delay given to a user.
An information processing device includes an acquisition unit, an encoding unit, a context recognition unit, a priority data extraction unit, and a communication unit. The acquisition unit acquires data related to the transmission point. The encoding unit encodes the data related to the transmission point. The context recognition unit sets the data to be preferentially transmitted from the data related to the transmission point based on the situation of the transmission point recognized by using the data related to the transmission point. The priority data extraction unit extracts the data to be preferentially transmitted as the priority data based on the setting in the context recognition unit. The communication unit transmits the data encoded by the encoding unit and the unencoded priority data to the information processing device at the receiving point.
[Selection diagram] Fig. 1

Description

The present technology relates to an information processing device and an information processing method.

It has become possible to bidirectionally transmit video and audio data between different points using a communication network such as a video conference and exchange information in real time (see, for example, Patent Document 1).

In Patent Document 1, the low delay priority of the video / audio data is determined based on the conversation volume of the video conference and the degree of excitement of the conversation.

Japanese Unexamined Patent Publication No. 2009-76952

In the technique described in Patent Document 1, for example, in a situation where there is no conversation between users and video synchronization between two parties at different points is required, an appropriate video is provided to the user by delaying the video / audio data. It's difficult.

In view of the above circumstances, an object of the present technology is to provide an information processing device and an information processing method capable of reducing a feeling of delay given to a user.

In order to achieve the above object, the information processing apparatus according to one form of the present technology includes an acquisition unit, an encoding unit, a context recognition unit, a priority data extraction unit, and a communication unit.
The acquisition unit acquires data related to the transmission point.
The encoding unit encodes data related to the transmission point.
The context recognition unit sets data to be preferentially transmitted from the data related to the transmission point based on the situation of the transmission point recognized by using the data related to the transmission point.
The priority data extraction unit extracts the data to be preferentially transmitted as priority data based on the settings in the context recognition unit.
The communication unit transmits the data encoded by the encoding unit and the unencoded priority data to the information processing device at the receiving point.

According to such a configuration, data to be preferentially transmitted to the information processing device at the receiving point is extracted based on the situation at the transmitting point, and the data is not encoded and is sent to the information processing device at the receiving point. Will be sent. As a result, the data to be preferentially transmitted does not require the time required for the encoding process, so that the data can be transmitted to the information processing device at the receiving point earlier than the data to be encoded.

The priority data extraction unit may extract the data to be preferentially transmitted, the status of the transmission point, and the reproduction time of the data to be preferentially transmitted as the priority data.

A storage unit for storing the priority data and a priority data prediction unit for predicting data to be preferentially transmitted based on the priority data stored in the storage unit may be further provided.

The data regarding the transmission point may include video data.
The data regarding the transmission point may further include at least one of sound data and depth data.

In order to achieve the above object, the information processing device according to one embodiment of the present technology includes a communication unit, a decoding unit, a determination unit, a reproduction data generation unit, and an output unit.
The communication unit receives from the information processing device at the transmission point the data in which the data relating to the transmission point is encoded and the priority data extracted from the data relating to the transmission point and not encoded.
The decoding unit decodes the encoded data.
The determination unit determines the reproduction time and reproduction method of the unencoded priority data.
The reproduction data generation unit generates the reproduction data of the priority data based on the determination in the determination unit.
The output unit outputs the data decoded by the decoding unit and the reproduction data of the priority data.

According to such a configuration, the unencoded priority data does not need to be decoded, so that it can be reproduced faster than the encoded data.

Of the decoded data, the data that has already been reproduced by the reproduction data of the priority data by referring to the storage unit that stores the content of the decision in the determination unit and the determination content stored in the storage unit. When the reproduced data of the priority data has already been reproduced by the reproduced confirmation unit for confirming whether or not there is, the reproduced data of the priority data and the decoded data are connected. It may further include an interpolation data generation unit that generates interpolation data for the purpose.

In order to achieve the above object, the information processing method according to one form of the present technology is
The information processing device at the transmission point
Get data about the transmission point,
Encode the data related to the above transmission point and
Based on the situation of the transmission point recognized by using the data related to the transmission point, the data to be preferentially transmitted is extracted as the priority data from the data related to the transmission point.
The encoded data and the unencoded priority data are transmitted to the information processing device at the receiving point.
The information processing device at the above reception point
Receive the above encoded data and the above unencoded priority data,
Decode the above encoded data and
Determine the playback time and playback method of the above priority data that has not been encoded.
Based on the above determination, the reproduction data of the above priority data is generated, and the reproduction data is generated.
The decoded data and the reproduced data of the priority data are output.

It is a figure which shows the structure of the information processing system and the information processing apparatus which used the information processing apparatus which concerns on one Embodiment of this technique. It is a flow chart of the information processing method related to the delay control in the information processing apparatus at the transmission point. It is a flow chart of the information processing method related to the delay control in the information processing apparatus at the receiving point. It is a figure explaining the specific example of the delay control in the information processing method which concerns on one Embodiment of this technique. It is a figure explaining the specific example of the image delay in the information processing method which concerns on a comparative example.

An information processing device according to an embodiment of the present disclosure and an information processing system using the same will be described. The information processing system relates to a communication system that bidirectionally transmits video data and audio data between two information processing devices installed at different points.

In the information processing system of the present embodiment, the situation of the point where the information processing device is installed (hereinafter referred to as a scene) using sensing data such as video data and sound data related to the information of the point acquired by the information processing device at the point. In some cases.) Is recognized.

The sensing data acquired by the information processing device at the transmission point is encoded through the AV codec and transmitted to the information processing device at the reception point.
Further, in the information processing device at the transmission point, data to be preferentially sent to the information processing device at the reception point (hereinafter, may be referred to as priority data) is extracted from the sensing data according to the scene. The extracted priority data is transmitted to the information processing device at the receiving point separately from the encoded data without passing through the AV codec.

The information processing device at the receiving point reproduces the encoded data and the unencoded priority data.

As described above, in the information processing system of the present embodiment, the priority data is separately transmitted and reproduced without passing through the AV codec, so that the delay for the AV codec is eliminated and the priority data is reproduced promptly. As a result, the information processing device at the receiving point receives video data and audio data with reduced delay depending on the situation, and can reduce the feeling of delay given to the user.
Hereinafter, a detailed description will be given.

(Configuration of information processing system)
FIG. 1 shows the configuration of the information processing system 50 according to the present embodiment.
As shown in FIG. 1, the information processing system 50 includes a first information processing system 20A and a second information processing system 20B. These two information processing systems 20A and 20B can communicate in both directions via the network 30.

In the present embodiment, the user who uses the first information processing system 20A is Mr. A, and the user who uses the second information processing system 20B is Mr. B. The first information processing system 20A is installed at the point A where Mr. A is. The second information processing system 20B is installed at the point B where Mr. B is. It is in a different place from point A and point B. When point A is a transmission point, point B is a reception point, and when point B is a transmission point, point A is a reception point.

The network 30 may include a public line network such as the Internet, a telephone line network, a satellite communication network, various LANs (Local Area Network) including Ethernet (registered trademark), and a WAN (Wide Area Network). Further, the network 30 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network). Further, the network 30 may include a wireless communication network such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

The first information processing system 20A and the second information processing system 20B have the same configuration. Here, A is added to the end of the code given to each configuration constituting the first information processing system 20A, and B is added to the end of the code unnecessary for each configuration constituting the second information processing system 20B. To distinguish.
In the following description, if it is not necessary to particularly separate the first information processing system 20A and the second information processing system 20B, the description of A and B will be omitted.

The information processing system 20 includes an information processing device 21, a sensor group 22, and a reproduction unit 23. The configuration of the information processing device 21 will be described later.

The sensor group 22 includes a video camera 221, a microphone 222 as a sound collecting unit, a depth sensor 223, and various sensors such as an infrared camera 224. Sensing data acquired by various sensors includes information (data) about points. Specifically, the data related to the point is video data, sound data, depth data, etc. of the person at the point and the surroundings of the person.

In the present embodiment, the situation (scene) of the point is recognized by using the sensing data. Scene recognition is recognized using sensing data for at least one of a plurality of points communicating with each other.

Scene recognition may be performed using sensing data obtained at each of a plurality of points, based on the sensing data obtained by the information processing device of the communication partner and the sensing data acquired by itself (information processing device). , The scene being performed between the two parties may be recognized.

The video camera 221 acquires video data at a point.
The microphone 222 collects sound at a point, for example, a human voice or an environmental sound, and acquires sound data.
The depth sensor 223 uses, for example, infrared light to acquire depth data indicating the distance of a person or an object at a point from the depth sensor. Any method such as a TOF (Time of Flight) method, a pattern irradiation method, and a stereo camera method can be adopted as the depth sensor method.
The infrared camera 224 acquires infrared image data of a person, an object, or the like. Human skeleton can be estimated from infrared image data.

The playback unit 23 includes a video player 231 and an audio player 232, a display unit 233, and a speaker 234 which is an audio output unit.

The video player 231 reproduces the reproduction data such as D / A conversion and amplification with respect to the reproduction data output from the reproduction data output unit 15 of the information processing apparatus 21 described later and based on the input priority data, prediction priority data, and non-priority data. The processing is performed, and the image is displayed on the display unit 233.

The voice player 232 reproduces the reproduced data such as D / A conversion and amplification with respect to the reproduced data output from the reproduction data output unit 15 of the information processing apparatus 21 described later and based on the input priority data, prediction priority data, and non-priority data. The processing is performed, and the sound is output from the speaker 234.
Priority data, prediction priority data, and non-priority data will be described later.

Whether the data reproduction process is performed by the video player 231 or the audio player 232 is determined by the playback time / playback method determination unit 9 of the information processing device 21 described later.

The display unit 233 displays the video reproduced by the video player 231.
The display unit 233 is composed of a display device such as a liquid crystal display, a plasma display, or an OELD (Organic Electro Luminescence Display). The display unit 233 is configured to be capable of displaying both the image of the communication partner point, the image of its own point, or the image of the communication partner point and the image of its own point.

For example, the image acquired by the second information processing system 20B is displayed on the display unit 233A of the first information processing system 20A, and the first information is displayed on the display unit 233B of the second information processing system 20B. The image acquired by the processing system 20A is displayed.

The speaker 234 outputs the sound reproduced and processed by the sound player 232.

The sound acquired by the second information processing system 20B is output to the speaker 234A of the first information processing system 20A, and the sound acquired by the second information processing system 20B is output to the speaker 234B of the second information processing system 20B by the first information processing system 20A. The acquired audio is output.

(Configuration of information processing device)
The information processing device 21 includes a sensing data acquisition unit 1, a data encoding unit 2, a context recognition unit 3, a priority data extraction unit 4, a short-term priority data storage unit 5, a priority data prediction unit 6, and a communication unit 7. , Priority data sorting unit 8, reproduction time / reproduction method determination unit 9, data decoding unit 10, priority data reproduction storage unit 11, reproduction data generation unit 12, priority data reproduction completed confirmation unit 13, and interpolation. It has a data generation unit 14 and a reproduction data output unit 15.

The sensing data acquisition unit 1 as an acquisition unit acquires the sensing data acquired by various sensors constituting the sensor group 22. Sensing data includes video data, sound data, depth data, and infrared image data. The sensing data is data related to the point where the sensing data is acquired. Here, a case where video data, sound data, and depth data are used as sensing data will be described.

The sensing data acquired by the sensing data acquisition unit 1 is output to the data encoding unit 2 and the context recognition unit 3.
Further, the acquired sensing data is stored in a sensing data storage unit (not shown in time series).

The data encoding unit 2 as the encoding unit encodes the sensing data acquired by the sensing data acquisition unit 1. All the data (here, video data, sound data, depth data) acquired by the sensing data acquisition unit 1 are encoded. The encoded data is output to the communication unit 7. The encoded data is non-preferred data.

The data encoding unit 2 and the data decoding unit 10 described later are provided in an AV codec (not shown).

The context recognition unit 3 recognizes the situation (scene) of the point based on the sensing data acquired by the sensing data acquisition unit 1.

For example, in the context recognition unit 3, there are a plurality of people at a point, a conversation is being held by a person, what the person is doing, what the object at the point is, and the object is moving. Recognize whether the object is stationary, what it is doing if the object is moving, and so on.

This recognition is performed by inputting the sensing data into the corresponding analyzer and analyzing it.

Of the various sensing data, sound data is discriminated into human voice and environmental sound by voice recognition by an analyzer. Thereby, the sound data of the human voice can be extracted.

Further, in voice recognition, language recognition is performed based on sound data, words included in sound data are recognized word by word, and keywords are extracted. The extracted keywords include nouns representing the names of objects, demonstrative words such as "stop" and "go", and words expressing emotions such as "happy" and "fun". The situation (scene) of the point can be recognized by using the keywords extracted in this way.

As a voice recognition method, for example, there is a method of accumulating voice features from learning data, comparing the features with the supplied voice, and extracting keywords, and a known method can be used.

Of the various sensing data, the video data is obtained by the analyzer for each pixel as the difference in pixel value from the previously acquired frame image.
Of the various sensing data, the depth data is different from the previously acquired depth data by the analyzer, and the amount of movement of the object in the depth direction and the vertical direction is obtained based on this difference.
For the previously acquired frame image and depth data, the data stored in the sensing data storage unit is used.
The movement information of a person or an object can be obtained from the difference between these pixel values and the amount of movement of the object.

The scene is estimated and recognized based on the analysis result of each of the sensing data.

As an example of the scene, two different places are used by using communication between information processing systems (first information processing system 20A and second information processing system 20B in the present embodiment) installed in a plurality of different places. A scene of information processing, a scene of a video conference, a scene of remote operation, and a remote pointing performed between two people in each (between Mr. A and Mr. B in this embodiment). There are scenes where you are doing information processing. The scene is not limited to these scenes.

Further, the context recognition unit 3 sets an identification flag of data to be synchronized between two points and between two parties based on the recognized scene. Data that should be synchronized between two points and between two parties is data that should be sent preferentially to the other party's information processing device (information processing device at the transmission point), which is important for smooth communication between the two parties in the scene. It is informational. An identification flag is set so that it can be identified whether or not the data should be sent with priority.
What data should be preferentially sent to the information processing device of the communication partner is set according to the scene. Specific examples will be described later.

The data to be synchronized (data to be sent preferentially) include sound data such as human voice determined by analysis of sound data, pixel values obtained by analysis of video data, and objects obtained by analysis of depth data. There is the amount of movement of.
Specific examples will be described later.

The scene name recognized by the context recognition unit 3, the information of the identification flag set for the scene name, and the reproduction time of the data to be sent preferentially are output to the priority data extraction unit 4.

The priority data extraction unit 4 extracts information to be synchronized between two points and two parties, that is, data to be preferentially transmitted, based on the set identification flag. The priority data extraction unit 4 combines the data to be preferentially transmitted, the scene name, the identification flag information, and the reproduction time, associates them with each other, and outputs the priority data to the short-term priority data storage unit 5 and the communication unit 7.

The short-term priority data storage unit 5 stores the priority data extracted by the priority data extraction unit 4 in chronological order in a short period of time.

The priority data prediction unit 6 predicts future priority data based on the priority data stored in the short-term priority data storage unit 5, and generates prediction priority data. The prediction priority data is output to the communication unit 7.

More specifically, the priority data prediction unit 6 continuously operates a person or an object in the video with a recognized scene name based on the time-series priority data saved by the short-term priority data storage unit 5. To determine if you are doing.
Further, when the priority data prediction unit 6 determines that the person or object is continuously moving, the priority data prediction unit 6 predicts the movement of the person or object from the time-series priority data saved by the short-term priority data storage unit 5. Determine if it is a possible phase.
Then, when it is determined that the phase can be predicted, the prediction priority data, which is the future priority data, is predicted and generated based on the information stored in the short-term priority data storage unit 5.

By providing the priority data prediction unit in this way, it is possible to preferentially present important data according to the scene to the communication partner, and it is possible to facilitate communication between the two parties.

As an example, in the rock-paper-scissors scene, the information of the sender is important, and the pixel value of the video data of the sender is the priority data.
In rock-paper-scissors, it is possible to predict which form the issuer will take from the change in the form of the hand before the issuer completely takes one of the forms of "goo", "choki", and "par". Therefore, it is possible to predict and generate predictive priority data, which is future priority data, based on the time-series priority data saved by the short-term priority data storage unit 5 before the issuer is completely released. Become.

The prediction priority data includes predicted priority transmission data such as sound data, pixel values, and movement amount of an object, a scene name, an identification flag, and a playback time. The prediction priority data is transmitted to the information processing device of the communication partner without being encoded.

The communication unit 7 transmits / receives data to / from the information processing device of the communication partner. In the present embodiment, the information processing device of the communication partner in the first information processing device 20A (second information processing device 20B) is the second information processing device 20B (first information processing device 20A).

The communication unit 7 transmits the priority data and the non-priority data obtained based on the sensing data acquired by its own information processing device to the information processing device of the communication partner. Further, the communication unit 7 receives the priority data and the non-priority data obtained based on the sensing data acquired by the information processing device of the communication partner.

In this way, the priority data is transmitted to the information processing device of the communication partner without being encoded without passing through the AV codec. As a result, compared to non-priority data encoded through the AV codec and transmitted to the information processing device of the communication partner, the priority code does not require time for encoding, so there is no delay for encoding by the AV codec, and communication is performed. It becomes possible to quickly transmit to the information processing device of the other party.
As a result, video data and audio data with reduced delay can be transmitted to the information processing device at the receiving point, and smooth communication between the two parties becomes possible.

The priority data sorting unit 8 separates the data received by the communication unit 7 from the information processing device of the communication partner into priority data or prediction priority data and non-priority data. The priority data and the prediction priority data are unencoded data. Non-priority data is encoded data.

The priority data sorting unit 8 outputs non-priority data to the data decoding unit 10.
The priority data sorting unit 8 outputs the priority data and the prediction priority data to the reproduction time / reproduction method determination unit 9.

Playback time / playback method as a determination unit The determination unit 9 determines the reproduction time of the priority data (prediction priority data) sorted by the priority data sorting unit 8 and the image which is the priority data (prediction priority data) in which player. Decide whether to play data (predicted video data) or sound data (predicted sound data). The determined content is stored in the priority data reproduction / storage unit 11. Further, the determined content is output to the reproduction data generation unit 12.

The data decoding unit 10 as the decoding unit decodes the non-priority data sorted by the priority data sorting unit 8. The decoded non-priority data is output to the priority data reproduction completed confirmation unit 13.

Priority data reproduction as a storage unit The storage unit 11 determines the content of the priority data, the reproduction time of performing the reproduction using the priority data, and the information of the reproduction machine used for the reproduction as the determination contents by the reproduction time / reproduction method determination unit 9. To save.

The reproduction data generation unit 12 generates reproduction data of priority data and prediction priority data based on the determination contents in the reproduction time / reproduction method determination unit 9. The generated reproduction data is output to the reproduction data output unit 15.

Priority data as reproduced confirmation unit The reproduced confirmation unit 13 refers to the decision content saved in the priority data reproduction storage unit 11 before reproducing the data decoded by the data decoding unit 10, and the communication partner. Among the decoded data of the non-priority data received from the information processing device, it is confirmed whether or not there is already reproduced by the reproduction using the priority data.

The priority data reproduction completed confirmation unit 13 outputs the decoded non-priority data that has not yet been reproduced to the reproduction data output unit 15. Further, the priority data reproduction completed confirmation unit 13 outputs the confirmation result to the interpolation data generation unit 14.

The interpolation data generation unit 14 generates interpolation data for interpolating and joining the priority data confirmed to have already been reproduced using the priority data and the decoded non-priority data. The generated interpolated data is output to the reproduction data output unit 15.

By providing the interpolation data generation unit 14 in this way, it is possible to display an image with less discomfort in the flow of movement of a person or the like projected on the image, and a voice with less discomfort in the flow of voice of the output person. Can be output.

The reproduction data output unit 15 as an output unit inputs the reproduction data from the reproduction data generation unit 12, inputs the decoded data from the priority data reproduction completion confirmation unit 13, and inputs the interpolation data from the interpolation data generation unit 14. Is received, and the reproduced data is output to the reproduction unit 23.

In the information processing device 21 at the receiving point, since the priority data is not encoded, it is not necessary to decode it through the AV codec, so that there is no delay for decoding by the AV codec and the data can be reproduced faster than the non-priority data. ..
As a result, the data transmitted from the information processing device 21 at the transmission point can be reproduced with a reduced delay, so that smoother communication between the two parties becomes possible.

(Information processing method)
Next, the information processing method executed by the above-mentioned information processing system 50 will be described separately for the transmitting side and the receiving side with reference to FIGS. 2 and 3.
Here, for convenience, the point A will be the transmission point and the point B will be the reception point. However, as a matter of course, the point A may be the reception point and the point B may be the transmission point. In such a case, the same processing is performed. Hereinafter, the configuration shown in FIG. 1 will be described as necessary.

[Operation flow in the information processing system at the transmission point]
FIG. 2 is a flow chart of an information processing method related to delay control in the information processing system on the transmitting side (here, the first information processing system 20A). Hereinafter, an information processing method in the information processing system on the transmitting side will be described with reference to FIG.

As shown in FIG. 2, the sensing data acquisition unit 1A of the first information processing device 21A acquires the sensing data acquired by the various sensors of the sensor group 22A as the data related to the point A (S1). In the present embodiment, the sensing data includes sound data, video data, and depth data.

The sensing data (sound data, video data, depth data) acquired by the sensing data acquisition unit 1A by the data encoding unit 2A is processed and encoded by a general-purpose codec after synchronizing the sensing data acquisition time (S2). ).

In the codec processing, the sound data, the video data, and the depth data are subjected to the codec processing having a short processing time. For example, sound data and video data are encoded by VP9, which is for real-time communication and has a short processing time.

The encoded data, that is, the non-priority data is transmitted to the second information processing device 21B via the communication unit 7A (S12).

The context recognition unit 3A performs voice recognition of the acquired sound data (S3). In voice recognition, human voice and environmental sound are discriminated.

The context recognition unit 3A obtains a pixel value using the acquired video data, and further calculates the difference in pixel value from the previous frame for each pixel (S4).

Using the acquired depth data, the context recognition unit 3A takes a difference from the depth information of the previous frame, and obtains the amount of movement of the object in the depth direction and the vertical direction (S5).
It is possible to acquire motion information of a person or an object from the difference result of these pixel values and the amount of movement of the object.

The context recognition unit 3A recognizes the scene based on the voice recognition result, the pixel value difference result, and the movement amount result of the object (S6).
Next, the context recognition unit 3A sets an identification flag for the data so that the data to be preferentially transmitted can be identified based on the recognized scene (S7).

Next, the priority data extraction unit 4A extracts data to be preferentially transmitted to the second information processing device 21B from the sound data, the pixel value, the movement amount of the object, and the like, based on the set identification flag. NS. The extracted data is extracted as priority data together with the scene name, identification flag information, and playback time (S8).

Next, the extracted priority data is written and stored in the short-term priority data storage unit 5A (S9).

Next, a person or an object is continuously moving with a recognized scene name based on the time-series priority data saved by the priority data prediction unit 6 and the short-term priority data storage unit 5. From the time-series priority data saved by the short-term priority data storage unit 5, it is determined whether or not the phase is such that the movement of a person or an object can be predicted (S10).

If No is determined in S10, the process proceeds to S12. The priority data is transmitted to the second information processing device 21B via the communication unit 7A (S12).

If it is determined in S10 that the person or object is continuously moving and the phase is such that the movement of the person or object can be predicted (Yes), the process proceeds to S11.

In S11, the prediction priority data is generated based on the information stored in the short-term priority data storage unit 5. The generated prediction priority data and the priority data are transmitted to the second information processing device 21B via the communication unit 7A (S12).

[Operation flow in the information processing system at the receiving point]
FIG. 3 is a flow chart of an information processing method related to delay control in the information processing system on the receiving side (here, the second information processing system 20B). Hereinafter, an information processing method in the information processing system on the receiving side will be described with reference to FIG.

As shown in FIG. 3, the communication unit 7B of the second information processing device 21B receives priority data, non-priority data, and prediction priority data from the first information processing device 21A via the network 30 (S31). ).

Next, the priority data sorting unit 8B determines whether or not the received data is priority data or prediction priority data (S32).
If it is determined in S32 that the data is not priority data or prediction priority data, that is, it is non-priority data (No), the process proceeds to S33.
If it is determined in S32 that the data is priority data or prediction priority data (Yes), the process proceeds to S38.

In S38, the reproduction time / reproduction method determination unit 9 determines the reproduction time and the reproduction method of the priority data to be reproduced or the predicted priority data. The reproduction method indicates which reproduction machine is used for reproduction.

The reproduction time and the reproduction method are stored in the priority data reproduction storage unit 11B, and the reproduction information of the priority data or the prediction priority data is stored (S39).

Next, the reproduction data generation unit 12B generates reproduction data using the priority data or the prediction priority data according to the determined reproduction method (S40). The generated reproduction data is output to the reproduction data output unit 15B, and the process proceeds to S36.

In S33, the data decoding unit 10B decodes the non-priority data which is the encoded data.

Next, the priority data reproduction completion confirmation unit 13B refers to the data stored in the priority data reproduction storage unit 11B, the reproduction time is used as a key, and the content included in the decoded data uses the priority data. It is confirmed by the reproduction whether it has already been reproduced (S34).

If it is confirmed that the data has not been reproduced in S34 (No), the decoded data is output to the reproduction data output unit 15B, and the process proceeds to S36.

When it is confirmed that the data is being reproduced in S34 (Yes), the interpolation data generation unit 14B generates the interpolation data so that the reproduction by the priority data performed in advance and the reproduction by the decoded data are successfully connected (yes). S35). The generated interpolated data is output to the reproduction data output unit 15B, and proceeds to S36.

In S36, the reproduction data output unit 15B sorts the data according to the reproduction time of the data, and then outputs the data to the determined player (video player 231B or audio player 232B) in order. The preferentially separately transmitted priority data is superimposed on the decoded non-priority data and output as output data according to the determined playback time.

As a specific example, in the case of a rock-paper-scissors scene, data in which the video data of the output part, which is the priority data, is superimposed on the hand part of the decoded video data is output.

The video player 231B and the audio player 232B perform playback processing based on the input data (S37), the video is displayed on the display unit 233B, and the audio is output from the speaker 234B.
In the case of a rock-paper-scissors scene, a video in which the video data of the output part, which is the priority data, is superimposed on the hand part of the decoded video data is displayed on the display unit 233B.

(Specific example of information processing method related to delay control)
Next, as an example of the information processing method related to delay control, a case where a scene in which Mr. A and Mr. B at different points are playing rock-paper-scissors is recognized will be described with reference to FIGS. 4 and 5.

"Rock-paper-scissors" is a game that uses only hands. Rock-paper-scissors is a means of deciding whether to win or lose by composing a trilemma with three types of fingering. "Rock-paper-scissors" is called Rock-paper-scissors in English-speaking countries, for example.

In Japan, there are three types of fingering, generally, "Goo", which takes the form of a fist by grasping all five fingers, and "Goo", which takes the form of extending the index finger and middle finger and grasping the other fingers. There are "choki" and "par" that stretches all five fingers.
"Goo" corresponds to Rock of Rock-paper-scissors. "Rock-paper-scissors" is equivalent to rock-paper-scissors scissors. "Par" corresponds to Rock-paper-scissors paper.

In "rock-paper-scissors", "goo" beats "choki" but loses to "par", "choki" wins to "par" but loses to "goo", and "par" wins to "goo". However, he loses to "Choki".

In Japan, the cliché "Saisho is goo, rock-paper-scissors" is often used as a shout when playing rock-paper-scissors. Here, an information processing method related to delay control will be described by taking the case of using this shout as an example.

In rock-paper-scissors, which uses the shout of "saisho wa goo, rock-paper-scissors", as a rule of rock-paper-scissors, the "goo" voice of "saisho wa goo" is uttered, and at the same time, all the rock-paper-scissors players are in the form of "goo". I'm supposed to get my hands on it.
Then, following "Saisho Hagu", a shout of "rock-paper-scissors" is made, and at the same time, a voice of "rock-paper-scissors" of "rock-paper-scissors" is made. Put out a hand in the form of "goo", "choki", or "par".

FIG. 4 is a diagram illustrating a specific example of an information processing method related to delay control in the information processing method according to the embodiment of the present technology. In the example shown in FIG. 4, there is a case where there is unencoded priority data to be transmitted preferentially.

FIG. 5 is a diagram illustrating a specific example of the information processing method related to delay control in the information processing method according to the comparative example. The example shown in FIG. 5 shows a case where there is no priority data to be preferentially transmitted and only non-priority data which is encoded data is transmitted to the information processing apparatus of the communication partner.

In the examples shown in FIGS. 4 and 5, Mr. A calls out, and in response to the call, Mr. A and Mr. B put out their hands in the form of goo, choki, or par according to each other's movements. , And.

First, a comparative example will be described with reference to FIG.
5 (A) to 5 (D) show the time-dependent changes of the user or the reproduced video at the time of shooting in a plurality of frames, respectively. In FIG. 5, the mover in the first frame when the mover of "Goo" is put out is surrounded by an ellipse of a chain line.

FIG. 5A shows the time course of Mr. A's movement when Mr. A is photographed by the first information processing system 20A. The video data, audio data, and depth data of Mr. A acquired by the first information processing system 20A are encoded and sent to the second information processing system 20B.

In the second information processing system 20B, the data sent from the first information processing system 20A is decoded, and the video and audio are reproduced.
FIG. 5B shows the time course of the reproduced video displayed on the display unit 233B of the second information processing system 20B based on the data sent from the first information processing system 20A. This reproduced video is a video of point A.

As shown in FIGS. 5A and 5B, due to the transmission delay, the image is reproduced on the second information processing system 20B side with a slight delay from the time of shooting taken by the first information processing system 20A. Will be done. In the example shown in the figure, the timing at which "goo" is output in the reproduced video displayed by the second information processing system 20B is one frame later than that when shooting with the first information processing system 20A. It has become.

Mr. B, who is a user on the second information processing system 20B side, sees the reproduced video as shown in FIG. 5B and plays rock-paper-scissors.

FIG. 5 (C) shows the operation of Mr. B when the second information processing system 20B is shooting Mr. B who is playing rock-paper-scissors by watching the reproduced video as shown in FIG. 5 (B). Shows the change over time.

Looking at the playback video shown in Fig. 5 (B), Mr. B sees the "Goo" when Mr. A's "Saisho wa Goo" is "Goo", and as shown in Fig. 5 (C), " Goo "is issued. As shown in FIGS. 5 (A) and 5 (C), the timing of issuing Mr. A's "goo" and the timing of issuing Mr. B's "goo" are out of sync.

The video data, audio data, and depth data of Mr. B acquired by the second information processing system 20B are encoded and sent to the first information processing system 20A.

In the first information processing system 20A, the data sent from the second information processing system 20B is decoded, and the video and audio are reproduced.
FIG. 5D shows the time course of the reproduced video displayed on the display unit 233A of the first information processing system 20A based on the data sent from the second information processing system 20B.

As shown in FIGS. 5C and 5D, due to the transmission delay, the point on the display unit 233A of the first information processing system 20A is delayed from the time of shooting taken by the second information processing system 20B. The image of B is played back. In the example shown in the figure, the timing at which "goo" is displayed in the image of the state of the point B reproduced and displayed on the display unit 233A of the point A is when the first information processing system 20A takes a picture (FIG. It is 3 frames later than 5 (A).).

Mr. A, who is the user of the first information processing system 20A, will play rock-paper-scissors by watching the reproduced video shown in FIG. 5 (D).

In other words, Mr. A confirms the image of Goo's hand to be put out in accordance with Mr. B's "Saisho wa Goo" shout "Goo" on the display unit 233A, and emits the next shout "Jankenpon". It will be.

As a result, as shown in FIG. 5 (A), Mr. A has to wait for three frames from the shout of "Saisho wa Goo" to the next shout of "Rock-paper-scissors". become.

On the other hand, in the information processing method in which the delay control shown in FIG. 4 according to the present embodiment is executed, as shown in FIG. 4 (A), the waiting time of Mr. A is one frame, and the waiting time is a comparative example. It is shorter than.

Hereinafter, it will be described with reference to FIG. 4 (A) to 4 (D) show the time-dependent changes of the user or the reproduced video at the time of shooting in a plurality of frames, respectively. In FIG. 4, the mover in the first frame when the mover of "Goo" is put out is surrounded by an ellipse of a chain line.

In the example described here, it is assumed that the information processing device 21 has already recognized that the scene of the user's situation is a rock-paper-scissors scene based on the sensing data acquired by the sensing data acquisition unit 1. explain.

In the information processing device 21, an identification flag is set for the data so that the data (priority data) to be preferentially sent to the other information processing device can be identified based on the recognized scene. In the rock-paper-scissors scene, the identification flag is set so that the video data of the part of the video data in the form of "goo", "choki", or "par" becomes the priority data.

Video data, sound data, and depth data, which are sensing data, are encoded and transmitted as non-priority data.
In addition to the encoded data (non-priority data), in the rock-paper-scissors scene, the pixel value of the video data of the sender is not encoded and is transmitted to the information processing device of the communication partner as the priority data. That is, in the rock-paper-scissors scene, the video information of the sender is important, so the video data of the sender is the priority data. Priority data corresponds to information that if delayed, communication between Mr. A and Mr. B in rock-paper-scissors will not be successful.

In the information processing system 50, the priority data is output to the reproduction data output unit of the information processing device of the communication partner without being encoded and decoded, so that there is no delay for encoding and decoding by the AV codec. , It is possible to present to the communication partner faster than non-priority data.

In the present embodiment, since the case where the scene is recognized as a rock-paper-scissors scene is taken as an example, the video data (pixel value) of the part of the rock-paper-scissors player is preferentially sent separately. Then, according to the determined playback time, output data is generated in which the video data of the output part that is preferentially sent separately is superimposed on the video data that has been encoded, transmitted, and decoded.

In the example shown in FIG. 4, in order to make it easier to understand the effect of the present technology, a case where the delay control related to the present technology is executed when the video of Mr. B is sent to Mr. A is given as an example. ..

FIG. 4A shows the time course of the movement of Mr. A when the first information processing system 20A is photographing Mr. A. The video data, sound data, and depth data acquired by the first information processing system 20A are encoded and sent to the second information processing system 20B.

In the second information processing system 20B, video and audio are reproduced based on the data sent from the first information processing system 20A.
FIG. 4B shows the time course of the reproduced video displayed on the display unit 233B.

As shown in FIGS. 4A and 4B, due to the transmission delay, the video is reproduced on the second information processing system 20B side with a slight delay from the time of shooting taken by the first information processing system 20A. Will be done. In the example shown in the figure, the timing at which "goo" is output in the reproduced video on the second information processing system 20B side is delayed by one frame in time compared to when shooting with the first information processing system 20A. ing.

Mr. B, who is a user on the second information processing system 20B side, sees the reproduced video as shown in FIG. 4B and plays rock-paper-scissors.

FIG. 4 (C) shows the time course of the movement of Mr. B when the second information processing system 20B is taking a picture of Mr. B who is playing rock-paper-scissors by watching the reproduced video shown in FIG. 4 (B). show.

Looking at the playback video shown in Fig. 4 (B), Mr. B sees the "Goo" when Mr. A's "Saisho wa Goo" is "Goo", and as shown in Fig. 4 (C), " Goo "is issued. As shown in FIGS. 4 (A) and 4 (C), the timing of issuing Mr. A's "goo" and the timing of issuing Mr. B's "goo" are out of sync.

The video data, audio data, and depth data of Mr. B acquired by the second information processing system 20B are encoded and sent to the first information processing system 20A.

Here, since it is already recognized as a rock-paper-scissors scene, the identification flag is set so that the pixel value of the partial image of the sender part becomes the priority data.
In the second information processing system 20B, the partial video data (pixel value) of the part of Mr. B's sender is extracted as priority data from the video data based on the identification flag. The extracted priority data is preferentially sent to the first information processing system 20A as priority data without being encoded.

Further, when there is prediction priority data predicted by the priority data prediction unit 6B, this prediction priority data is also preferentially sent to the first information processing system 20A without being encoded. Here, the prediction priority data is the partial video data (pixel value) of the predicted output part.

In the first information processing system 20A, the reproduction time and reproduction method of the video data (pixel value) of the sender, which is the priority data or the prediction priority data sent from the second information processing system 20B, are determined, and based on this. The playback data of the video data of the sender, which is the priority data, is generated.

Further, even when there is prediction priority data, the reproduction time and reproduction method of the video data of the predicted sender which is the sent prediction priority data are determined, and based on this, the predicted sender which is the prediction priority data is determined. Playback data of the video data of is generated.

Further, in the first information processing system 20A, the encoded data which is the non-priority data sent from the second information processing system 20B is decoded.

If there is data that has already been reproduced using the priority data or the prediction priority data among the decoded data, the reproduction by the priority data or the prediction priority data performed in advance and the reproduction by the decoded data are performed. Interpolated data is generated so that it connects well.

The decoded data, the generated reproduction data, and the interpolation data are sorted according to the reproduction time of the data, and then output to the video player 231A and the audio player 232A for reproduction. As a result, the reproduced video is displayed on the display unit 233A as shown in FIG. 4 (D).
FIG. 4D shows a reproduced image at point B displayed on the display unit 233A.

As shown in FIG. 4 (D), the "goo" of the sender surrounded by the broken line ellipse is based on the video data (pixel value) of the sender sent preferentially as the priority data, and is based on the video data (pixel value) of the sender. The video parts other than the above are videos based on non-priority data. In this way, the video of the sender based on the priority data sent preferentially is superimposed and displayed on the video based on the non-priority data sent earlier.
As a result, the video of the rock-paper-scissors scene, which is important for the scene, is reproduced by the information processing device of the communication partner with the delay suppressed.

Mr. A, who is the user of the first information processing system 20A, will play rock-paper-scissors by watching the reproduced video shown in FIG. 4 (D). In other words, Mr. A confirms the image of Goo's hand that Mr. B puts out in accordance with "Goo" of "Saisho wa Goo" on the display unit 233A, and decides to issue the next call "rock-paper-scissors". Become.

As a result, as shown in FIG. 4 (A), the waiting time for Mr. A from the shout of "Saisho wa Goo" to the next shout of "Rock-paper-scissors" is one frame. That is, the information waiting time from Mr. A and Mr. B is shortened as compared with the comparative example described with reference to FIG.

By reducing the waiting time in this way, the efficiency per unit time is improved. Further, by shortening the waiting time, the feeling of delay felt by the user is reduced, and the communication between Mr. A and Mr. B can be made more natural and smooth.

As described above, in the present technology, in communication between a plurality of remote points, important data according to the scene is extracted and preferentially transmitted to the information processing device of the communication partner. Important information can be presented to the other party with reduced delay.

As a result, it is important for the communication partner according to the scene even in the communication situation where the transmission delay is likely to be large, such as communication in an area where the communication infrastructure is not developed, communication in a country with a large land area, communication between other countries, etc. Information can be presented faster, and the feeling of delay in communication given to the user can be reduced.

In addition, this technology uses an IoT (Internet of Things) terminal that automatically sends information acquired by a sensor to a server, and a plurality of servers (edge servers) provided at positions physically close to the IoT terminal. It can also be applied to edging computing that performs data distribution processing in.

(Example of application of information processing method related to delay control in other scenes)
In the above-described embodiment, the rock-paper-scissors scene has been described as an example, but the present invention is not limited to this. By differentiating the content of the data to be preferentially transmitted to the information processing system of the communication partner according to the recognized scene and preferentially transmitting important information according to the scene, the user feels a delay in communication. Can be reduced.
An example of application to other scenes other than rock-paper-scissors will be described below, but the present invention is not limited to these scenes.

For example, in the case of a video conference, the movement and voice of the person who is likely to start talking becomes important, so as priority data, the part of the mouth of the person who is likely to start talking is used as priority data. Video data and human voice data are extracted.

By preferentially transmitting such data to the information processing device of the transmission partner, it is possible to prevent users at different points from colliding with each other in the video conference. As a result, smoother communication is possible between a plurality of different points, and it is possible to reduce the feeling of communication delay given to the user.

Further, as another example, in the case of a remote control scene in which a person at point A moves an object at point B, for example, with a robot hand while listening to instructions from the voice of a person at point B. Also, this technology can be applied.

In the case of such a remote control scene, voice instructions such as "stop" (data of the voice of the instruction) and video data of the robot hand, which is an object approaching the object to be moved, are extracted as priority data. ..

In the example in which a person at point A moves an object at point B while listening to the instructions of a person at point B, when point B is the transmission point, the acquired sound data at point B is used. The data of the instruction voice issued by the person at the B point is extracted. Further, the video data of the robot hand portion is extracted from the acquired video data of the B point.
The extracted instruction voice data and the video data (pixel value) of the robot hand portion are preferentially transmitted to the information processing device at point A on the receiving side as priority data.

Further, as another example of the remote control scene, in the example in which the person at the point A performs the movement operation of the object at the point A while listening to the instruction of the person at the point B using the robot hand, the point B is used. When is the transmission point, the voice data of the instruction issued by the person at the B point is extracted from the sound data acquired at the B point. The voice data of this instruction is preferentially transmitted to the information processing device at point A, which is the receiving point, as priority data.
On the other hand, when the point A is the transmission point, the video data of the robot hand portion is extracted from the video data acquired at the point A. The video data of the robot hand portion is preferentially transmitted to the information processing device at point B, which is the receiving point, as priority data.

As described above, the communication delay feeling given to the user by preferentially transmitting the instruction voice data and the video data of the robot hand part, which are important in the remote control scene, to the information processing device of the communication partner. Can be reduced, and smoother remote control can be performed.

Further, as yet another example, the present technology can be applied to the case of a remote pointing scene.
The remote pointing scene is, for example, that the images of both the A point and the B point are displayed on the display unit 233A at the A point and the display unit 233B at the B point, respectively, and the person at the A point is displayed on the display unit 233A. A scene that points to an object projected in the image at point B. In this case, in the image of the point B displayed on the display unit 233B, the point pointed at the point A is displayed in the form of the pointing point.

In the case of a remote pointing scene, the movement of the finger on the image is important, so the image data (pixel value) of the part of the finger pointing is extracted from the image data acquired at point A. , As priority data, it is preferentially transmitted to the information processing device at point B.
By preferentially transmitting the video data of the finger portion in this way, the pointing direction of the finger on the transmitting side and the pointing point displayed on the receiving side are synchronized, and the feeling of delay in communication given to the user is reduced. Can be made to.

The embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.

For example, in the above-described embodiment, communication between two points is taken as an example, but it can also be applied to communication between three or more points.

Further, in the above-described embodiment, the sound data, the video data, and the depth data are given as examples as the sensing data to be acquired, but at least the video data may be present. By partially extracting data from the video data according to the scene and preferentially transmitting it to the information processing device of the communication partner, it is possible to reduce the feeling of delay in communication given to the user.

The present technology can have the following configurations.

(1)
An acquisition unit that acquires data related to the transmission point,
An encoding unit that encodes the data related to the above transmission point, and
A context recognition unit that sets priority transmission data from the data related to the transmission point based on the situation of the transmission point recognized using the data related to the transmission point.
Based on the settings in the context recognition unit, the priority data extraction unit that extracts the data to be preferentially transmitted as priority data, and the priority data extraction unit.
An information processing device including a communication unit that transmits data encoded by the encoding unit and unencoded priority data to an information processing device at a receiving point.

(2)
The information processing device according to (1) above.
The priority data extraction unit is an information processing device that extracts the data to be preferentially transmitted, the status of the transmission point, and the reproduction time of the data to be preferentially transmitted as the priority data.

(3)
The information processing device according to (1) or (2) above.
A storage unit that stores the above priority data,
An information processing device further including a priority data prediction unit that predicts data to be preferentially transmitted based on the priority data stored in the storage unit.

(4)
The information processing device according to any one of (1) to (3) above.
The data related to the transmission point is an information processing device including video data.

(5)
The information processing device according to (4) above.
The data related to the transmission point is an information processing device that further includes at least one of sound data and depth data.

(6)
A communication unit that receives encoded data related to the transmission point and priority data extracted from the data related to the transmission point and unencoded from the information processing device at the transmission point.
The decoding unit that decodes the encoded data and
A determination unit that determines the playback time and playback method of the unencoded priority data, and
A reproduction data generation unit that generates reproduction data of the priority data based on the determination in the determination unit, and a reproduction data generation unit.
An information processing device including an output unit that outputs data decoded by the decoding unit and playback data of the priority data.

(7)
The information processing device according to (6) above.
A storage unit that stores the contents of the decisions made by the above determination unit,
With reference to the decision contents saved in the storage unit, a reproduction confirmation unit for confirming whether or not there is data that has already been reproduced by the reproduction data of the priority data among the decoded data, and a reproduction confirmation unit.
When the reproduction data of the priority data has already been reproduced by the reproduction confirmation unit, the interpolation data generation unit that generates the interpolation data for joining the reproduction data of the priority data and the decoded data is further added. Information processing device to be equipped.

(8)
The information processing device at the transmission point
Get data about the transmission point,
Encode the data related to the above transmission point and
Based on the situation of the transmission point recognized by using the data related to the transmission point, the data to be preferentially transmitted is extracted as the priority data from the data related to the transmission point.
The encoded data and the unencoded priority data are transmitted to the information processing device at the receiving point.
The information processing device at the above reception point
Receive the above encoded data and the above unencoded priority data,
Decode the above encoded data and
Determine the playback time and playback method of the above priority data that has not been encoded.
Based on the above determination, the reproduction data of the above priority data is generated, and the reproduction data is generated.
An information processing method that outputs the reproduced data of the decoded data and the priority data.

1A, 1B ... Sensing data acquisition unit (acquisition unit)
2A, 2B ... Data encoding section (encoding section)
3A, 3B ... Context recognition unit 4A, 4B ... Priority data extraction unit 6A, 6B ... Priority data prediction unit 7A, 7B ... Communication unit 9A, 9B ... Playback time / playback method determination unit (decision unit)
10A, 10B ... Data decoding unit (decoding unit)
11A, 11B ... Priority data playback storage unit (storage unit)
12A, 12B ... Playback data generation unit 13A, 13B ... Priority data playback completed confirmation unit (reproduced confirmation unit)
14A, 14B ... Interpolation data generation unit 15A, 15B ... Reproduction data output unit (output unit)
20A ... First information processing device (information processing device at the transmission point, information processing device at the reception point)
20B ... Second information processing device (information processing device at the transmission point, information processing device at the reception point)
50 ... Information processing system

Claims (8)

An acquisition unit that acquires data related to the transmission point,
An encoding unit that encodes data related to the transmission point, and
A context recognition unit that sets data to be preferentially transmitted from the data related to the transmission point based on the situation of the transmission point recognized using the data related to the transmission point.
A priority data extraction unit that extracts the data to be preferentially transmitted as priority data based on the settings in the context recognition unit, and a priority data extraction unit.
An information processing device including a communication unit that transmits data encoded by the encoding unit and unencoded priority data to an information processing device at a receiving point. The information processing device according to claim 1.
The priority data extraction unit is an information processing device that extracts the data to be preferentially transmitted, the status of the transmission point, and the reproduction time of the data to be preferentially transmitted as the priority data. The information processing device according to claim 2.
A storage unit that stores the priority data and
An information processing device further comprising a priority data prediction unit that predicts data to be preferentially transmitted based on the priority data stored in the storage unit. The information processing device according to claim 3.
The data related to the transmission point is an information processing device including video data. The information processing device according to claim 4.
The data related to the transmission point is an information processing device that further includes at least one of sound data and depth data. A communication unit that receives encoded data related to the transmission point and priority data extracted from the data related to the transmission point and unencoded from the information processing device at the transmission point.
A decoding unit that decodes the encoded data, and
A determination unit that determines the playback time and playback method of the unencoded priority data, and
A reproduction data generation unit that generates reproduction data of the priority data based on the determination in the determination unit, and a reproduction data generation unit.
An information processing device including an output unit that outputs data decoded by the decoding unit and reproduced data of the priority data. The information processing device according to claim 6.
A storage unit that stores the content of the decision made by the determination unit,
With reference to the decision contents saved in the storage unit, a reproduction confirmation unit for confirming whether or not there is data already reproduced by the reproduction data of the priority data among the decoded data, and a reproduction confirmation unit.
When the reproduced data of the priority data has already been reproduced by the reproduced confirmation unit, an interpolation data generation unit that generates interpolation data for joining the reproduced data of the priority data and the decoded data is further added. Information processing device to be equipped. The information processing device at the transmission point
Get data about the transmission point,
Encode the data about the transmission point and
Based on the situation of the transmission point recognized by using the data related to the transmission point, the data to be preferentially transmitted is extracted as priority data from the data related to the transmission point.
The encoded data and the unencoded priority data are transmitted to the information processing device at the receiving point, and the data is transmitted.
The information processing device at the reception point
Upon receiving the encoded data and the unencoded priority data,
Decode the encoded data and
Determine the playback time and playback method of the unencoded priority data,
Based on the determination, the reproduction data of the priority data is generated, and the reproduction data is generated.
Outputs the decoded data and the reproduced data of the priority data.
Information processing method.

Images