FR2961919A1 - Method for processing 360 degree audiovisual scene of football match broadcasted in Internet by server, involves transmitting information relative to scene portion visualized by user to mobile telephone for utilization - Google Patents

Abstract

The method involves receiving video and audio streams (E1) of 360 degree multimedia scene. The streams are decoded (E2) progressive to their reception. Angular position of a user with respect to a visualization support reference is detected (E3). Coordinates representing a portion of the decoded scene to be displayed on 360 degree visualization support are calculated (E4) based on the position. The portion is sent (E5) to the support for displaying and listening the portion. Information relative to the portion visualized by the user are transmitted (E6) to a mobile telephone for utilization. Independent claims are also included for the following: (1) a method for distributing video and audio streams of 360 degree multimedia scene to user terminals connected to a telecommunications network (2) a device for distributing video and audio streams of 360 degree multimedia scene to user terminals, comprising an information receiving unit (3) a device for processing 360 degree multimedia scene distributed in a telecommunications network by a server, comprising a display unit (4) a computer program comprising instructions for performing a method for processing 360 degree multimedia scene distributed in a telecommunications network (5) a computer program comprising instructions for performing a method for distributing video and audio streams of 360 degree multimedia scene to user terminals.

Description

TECHNICAL FIELD OF THE INVENTION The field of the invention is that of telecommunications, and more particularly of the transmission of audio-visual contents. 360 degree type. Cameras are now commercially available to produce 360-degree films. These cameras are equipped with several lenses, usually 6 to 12, which film each according to an angle of view. The different shots are then transformed into a single planispherical video, according to a so-called "stitching" technique, intended to be "mapped" on a screen of a dedicated viewing device or not. Such a processing device is specifically adapted to processing for the purpose of viewing audiovisual contents of the 360 degree type. It is capable of processing a data stream representative of a 360-degree audiovisual content to "drop" the texture information of the 360-degree content onto a virtual sphere or cube and to restore a portion of that sphere or this cube on a display screen, for example a PC personal computer screen ("Persona / Computer", in English) or mobile terminal. By means of commands, for example by pressing certain keys of a computer keyboard, it is possible to rotate the sphere or the cube and thus modify the part displayed on the screen. The 360 degrees of audiovisual content can therefore be viewed piece by piece. It is also known to use specific screens, for example "head mounted display" or HMD, which are fixed on the head of the user. An HMD generally consists of: immersive goggles, with micro-display screens ("microdisplay" in English) on which the user can view part of the 360-degree scene; a gyroscope ("headtracker" in English) to detect the movements of the head and back to a suitable processing module. This is for example the FRCM-HC3 tracker model from Hillcrest; Headphones adapted to the sound reception.

In a known manner, an HMD connects to a PC-type computer or a mobile terminal via a VGA, HDMI or DVI type socket. A software client installed on the computer or the terminal is able to manage the connection of the HMD to the terminal and to control the sending of the image restored by the 360-degree audiovisual content processing device included in the computer in the glasses instead. than (or simultaneously) on the PC screen.

The movements of the head detected by the headtracker are reassembled at the level of the processing device which is able to calculate a new image to be displayed in the glasses according to the new position of the head of the user. Turning the head, the user has the opportunity to view in his spherical glasses a portion of the 360-degree scene corresponding to his field of vision.

It is particularly known from WO2005 / 017729 to use such an HMD device to display a virtual audiovisual object superimposed on a real scene observed by the user through his glasses. HMD devices are typically used for video game applications and for military simulations, in which the user is part of the scene viewed by the participants. In this regard, the document WO2009 / 117450 describes a method for producing an immersive audiovisual scene involving participants equipped with such HMD devices. The inventors have found a new need for users to share with others an immersive viewing experience of audiovisual content broadcast by a content provider, for example in streamed mode. According to a first aspect, the invention relates to a method of processing a 360-degree multimedia scene distributed in a telecommunications network by a server to a plurality of user terminals connected to said network, characterized in that said method comprises the following steps, intended to be implemented by a terminal of said plurality of terminals, said terminal being connected to a viewing and listening medium 360 degrees centered on a first user: -receiving at least one video stream and at least one audio stream constituting the 360-degree multimedia scene; decoding of said streams as and when they are received; detecting an angular positioning of said user with respect to a reference frame of the spherical support; displaying at least a portion of the 360 degree multimedia scene decoded on said display medium and rendering the associated sound, as a function of the angular position detected; transmission of information relating to the part of the scene viewed and listened to by the user for at least one second user terminal for use.

The invention allows a user to receive the constituent data streams of a 360-degree multimedia scene and to "play" a part thereof, corresponding to an angular sector of this scene, on a 360-degree support, for example glasses spheres of an HMD device. We understand that the field of vision of this first user does not allow him to view and listen to the entire 360-degree scene at one time. He must therefore turn his head in order to discover the scene 360 degrees angular sector by angular sector. The position of its head, in particular its angular position, is thus calculated with respect to a reference frame of the display medium and the angular sector of the scene to be displayed on the viewing medium, around this angular position, is deduced from this position. . Advantageously, the rendering of the sound associated with the projected image on the display medium may be spatialized, so as to reinforce the user's immersion sensation in the audiovisual scene. Note that the 360-degree viewing support can take various forms, for example be cubic or still consists of a plurality of screens oriented at different angles and arranged side by side.

The invention also makes it possible to trace back to at least a second user information relating to the part of the scene viewed and listened to by the first user. Indeed, as the audiovisual scene has a field of 360 degrees, it can not be embraced at one glance. As a result, two users, who at the same time receive audio and video streams from the 360-degree scene, do not necessarily view and listen to the same angular sector of that scene. Thanks to the information transmitted in the network by the terminal of the first user, the second user can deduce the part of the scene that the first user is seeing and listening. In this way, he can share, if he wishes, his immersion experience in the 360-degree scene. Thus, the invention proposes a new and inventive solution for immersive communication. According to one aspect of the invention, the information relating to the portion of the scene viewed and listened to by the first user comprises at least the angular sector of said scene portion and / or the angular orientation of the user. Such information may advantageously be used by the terminal of the second user to indicate with a visual signal the portion of the scene viewed and listened to by the first user. According to another aspect of the invention, the information relating to the portion of the scene viewed and listened to by the first user comprises at least one voice signal representative of a reaction of the user to the scene portion that he displays. For example, the user exclaims, "Oh, look right!" to draw other users' attention to the part of the scene they are viewing and interacting with. It will be understood that, according to this embodiment, a voice communication session has been established between the first and second users in parallel with the streaming video streaming session from the content server. Such communication may involve an application server of the telecommunications network, able to connect the two user terminals. According to one aspect of the invention, the processing method comprises a step of receiving information relating to a scene portion displayed by a third user and a step of adapting the displayed scene part using the information received. . As previously discussed in connection with the first user, this information may be of different types, such as, for example, the angular sector of the visualized scene part or a reaction speech signal. When the information takes the form of angular sector values and position, the adaptation step may consist of superimposing a direction indication on the projected scene, for example an arrow pointing to the scene part displayed by the third user. Thus the first user is encouraged to turn their heads towards this part of the scene and share the same audiovisual content as the third user. When the information takes the form of a reaction voice signal, such an adaptation may consist of sending a sound rendition of this speech signal into the spectacles in a representative angular orientation of the third user. Advantageously, it is possible to associate beforehand with each user participating in the broadcast session of the audiovisual scene, a sound angular orientation, so that each time this user emits a voice signal, he will be received by the others in their 10 processing device, with this angular orientation. It is understood that this associated angular orientation is a kind of sound signature of the user. According to one variant, the voice signal of reaction emitted by the third user is transmitted in the processing device of the other users with the angular orientation raised by the terminal of the third user as information relating to the part of the scene that he visualizes and listens. Advantageously, the first user will be prompted to turn his head in the direction of the sound he perceives and will face the same part of the scene as the third user. According to one aspect of the invention, the multimedia scene consists of a single audiovisual content represented in the form of audio stream and a video stream. According to another aspect of the invention, the multimedia scene consists of a plurality of contents or applications each represented as an audio stream and a video stream or a file depending on the type of content, and metadata describing a spatial organization of said contents relative to one another on the medium spherical. Advantageously, such metadata comprise an angular position in a repository of the user-centered display medium. According to this embodiment of the invention, the display step consists in displaying the content of the plurality of contents of the multimedia scene whose angular position is closest to that of the user and in rendering the its associated with this content. According to another aspect of the invention, the information received from a third user 30 during the receiving step includes an identifier of the content of the plurality of contents viewed and listened to by the third user and the adaptation step. includes a superposition on the projected scene to the first user of a visual or audible indication of the content viewed and listened to by the third user and identified from said received identifier. The invention also relates to a method of distributing at least one video stream and a constituent audio stream of a 360-degree multimedia scene to a plurality of user terminals connected to a telecommunications network, characterized in that said method comprises the following steps: distribution of said streams to the terminals, a terminal being able to decode the video and audio streams as and when they are received and to project at least a portion of the decoded multimedia scene on a 360 degree viewing medium centered on the user receiving, from at least one first terminal of said plurality of terminals, information relating to the scene portion displayed by the user of said first terminal; retransmission of said information to at least one second user terminal. Such a method is advantageously implemented by a service platform of the telecommunications network, capable of rendering the distribution service of the audio and video streams constituting the audiovisual scene 360 degrees by establishing for example a streaming session with the terminals of users involved and, where appropriate, to put them in contact.

The invention also relates to a device for processing a 360-degree multimedia scene. Such a device is intended to be connected to a user terminal of said plurality of terminals or integrated with this terminal and is connected to a 360 degree viewing medium centered on a first user. According to the invention, said processing device comprises the following means: reception of at least one video stream and at least one audio stream constituting the 360-degree multimedia scene; decoding of said streams as and when they are received; detecting an angular orientation of said user with respect to a repository of the visualization medium; displaying at least a portion of the decoded 360 degree multimedia scene on said spherical viewing medium and retransmitting the associated sound, as a function of the detected angular position; transmission of information relating to the part of the scene displayed by the user to at least one second user terminal for use. The invention also relates to a device for distributing at least one video stream and a constituent audio stream of a 360-degree multimedia scene to a plurality of user terminals connected to a telecommunications network, characterized in that said device is capable of implementing the following means: distribution of said streams to the terminals, a terminal being able to decode the video and audio streams as and when they are received and to project at least a portion of the decoded multimedia scene onto a user-centered visualization media; receiving, from at least one first terminal of said plurality of terminals, information relating to the scene portion displayed by the user of said first terminal; Retransmission of said information to at least one second user terminal. This device may of course include the various features relating to the treatment method according to the invention. Thus, the characteristics and advantages of this decoder are the same as those of the processing method, and are not detailed further. The invention also relates to a system for distributing a 360 degree multimedia scene 360 to a plurality of user terminals connected to a telecommunications network, characterized in that it comprises at least one device for distributing a multimedia scene. and a device for processing said multimedia scene according to the invention. The invention also relates to a computer program comprising instructions for implementing a method of processing as described above, when this program is executed by a processor. Such a program can use any programming language. It can be downloaded from a communication network and / or saved on a computer-readable medium. The invention finally relates to a computer program comprising instructions for the implementation of a distribution method as described above, when this program is executed by a processor. Such a program can use any programming language. It can be downloaded from a communication network and / or saved on a computer-readable medium.

List of Figures Other features and advantages of the invention will appear more clearly on reading the following description of a particular embodiment, given as a simple illustrative and nonlimiting example, and the appended drawings, among which: Figure 1 schematically shows a communication system of a 360-degree audiovisual scene to a plurality of user terminals according to the invention; FIG. 2 presents the main steps implemented by the method of processing a 360-degree audiovisual scene according to one embodiment of the invention; FIG. 3 presents the main steps implemented by the distribution method of a 360-degree audiovisual scene according to one embodiment of the invention; FIG. 4 presents, in the form of a flow diagram, a first exemplary embodiment of the invention; FIG. 5 presents, in the form of a flow diagram, a second exemplary embodiment of the invention; Figure 6 shows respectively the structure of a processing device according to the invention; the invention; and Figure 7 shows respectively the structure of a dispensing device according to the invention.

DESCRIPTION OF AN EMBODIMENT OF THE INVENTION As a reminder, the principle of the invention is based on the real-time distribution to a plurality of user terminals of a 360-degree audiovisual scene, each of which displays the part it wishes and on the rise in the network, by these users, information on the part of the scene viewed and listened to by each, so that they are transmitted to other users and they use them to share their immersive multimedia experience. In relation to FIG. 1, an example of a communication system of a 360-degree multimedia scene according to the invention is presented. An NT telecommunications network is considered, which may be of mobile type, for example 3G or fixed, for example the Internet, and three user terminals Ti, T2, T3 connected to this network. The first user terminal Ti is a 3G mobile terminal, the second user terminal T2 is a personal computer and the third terminal is a multimedia device.

These three types of terminals are also called Pods. Their connection with the telecommunications network is realized for terminals T2 and T3 via a home gateway, via a wired or wireless link (Wifi). T3 are on the other hand connected to a device HMD1, HMD2, HMD3, type HMD allowing the viewing and listening of the scene 360 degrees on a spherical support, for example spherical glasses UT1, UT2 and UT3 users of the Terminals Ti, T2 and T3 place the HMD device on their heads for the purpose of viewing the scene, and a PFS 100 service platform capable of delivering a 360-degree multimedia scene communication service according to the invention is now considered. platform 15 includes several servers, which can be placed in different places of the telecommunications network NT: - a first Web server 120, including the web portal where users register with the s ervice and choose the 360-degree multimedia scene they want to view. If necessary, they can also choose a broadcast schedule; A second content server 130, capable of storing in a memory M the DS audio and video streams constituting the 360 degree multimedia scenes available for the implementation of the service; a third application server 110 adapted to render the communication service, in particular by processing the service access requests sent by user terminals via the first web server 120, establishing the multimedia communication sessions and the case where appropriate voice over IP with the registered user terminals and by triggering the broadcasting of the audio and video streams constituting the 360-degree multimedia scene to the content server 130. At the scheduled time or at the request of a user, the application server 110 triggers the distribution of the audio and video DS streams constituting the 360-degree scene. Advantageously, their distribution is done in streamed mode in the NT telecommunications network to the user terminals Ti, T2 registered in the session. The user terminals Ti, T2 process the streams as and when they receive and control the display of a portion of the 360-degree multimedia scene on the display medium HMD1, HMD2 and HMD3. To do this, the user terminals implement the processing method according to the invention which will now be described in connection with FIG. 2. In a step E1, at least one video stream and at least one audio stream DS constituting the 360-degree multimedia scene are received by the terminal processing device Ti and T2.

In a step E2, they are decoded as and when they are received; In a step E3, data representative of an angular positioning of said user UT1, UT2 with respect to a reference frame of the spherical support are received from the device HMD1, HMD2; In a step E4, coordinates representative of a scene portion to be displayed on the display medium are calculated according to the angular positioning of the user; In a step E5, the portion of the decoded 360-degree multimedia scene is sent on said spherical viewing medium HMD1, HMD2 for display and listening. In a step E6, information UR1 relating to the portion of the scene displayed by the user UT1 is transmitted in the telecommunications network to at least one second user terminal T2 for use. Note that this transmission step can be implemented upon receipt of data at E3, or later. In this example, it is considered that the UT3 user registers for the immersive communication session when it has already started. In this case, the user terminal T3 receives the audio and video streams DS from the moment it joins the session. Its processing device then implements the treatment method according to the invention which has just been described. During a step E7, UR3 information relating to the portion of the scene displayed by a user UT3 is received and processed in E8. They are then superimposed visually or by a sound rendering to the part of the scene viewed and listened to by the user UT1.30. According to the invention, the immersive communication service is deployed by the PFS service platform. To do this, it implements a distribution method which will now be described in relation to FIG. 3. In a first step D1, a request RQ is received from the first user terminal Ti to access the service. Advantageously, this request includes an indication of the 360 degree multimedia scene S1 chosen by the user Ti. It may furthermore include a start time schedule H. In a step D2, a communication is established by the PFS service platform with the user terminal Ti and the distribution of the DS audio and video streams constituting the multimedia scene to the user terminal Ti is initiated at D3. In an exemplary embodiment of the invention according to which the retransmission of the scene starts at a predetermined time H, the DS streams are distributed from the time H to all the terminals T1, T2 of the users UT1, UT2 registered with the service. for this schedule. In a step D4, the service platform PFS receives from the user terminal Ti UR1 information relating to the scene portion that it displays. As mentioned above, such information may be of various types. According to one aspect of the invention, this information takes the form of a voice signal SV1 sent by the user UT1 in response to the scene S1 that it displays. According to this mode, it is understood that it is necessary for a voice type communication channel, for example VoIP, to have been previously established during a step D2 ', between the user terminal Ti and the service platform. PFS. During a step D5, the UR1 information is processed by the PFS. According to a first exemplary embodiment, this information representative of an angular positioning of the user UT1 is processed locally by the service platform PFS and used to adapt the data streams DS transmitted to the first user UT1. By way of example, it can be envisaged that only the angular sector of the sphere corresponding to the angular positioning of the user is transmitted to him, rather than the entire 360-degree scene, in order to save bandwidth. In a second exemplary embodiment, this information represents an angular positioning of the first user UT1. During the step D5, they can be inserted in a message intended for the other users UT2 and UT3, which is transmitted during a step D6. In a third exemplary embodiment, this information takes the form of a voice signal and the processing step D5 consists in routing this signal to the other users UT2, UT3 parties to the session via the voice communication channel. It should be noted that the establishment of voice communication is implemented differently depending on the number of users involved in the immersion session. First of all, the establishment of a voice communication between two users UT1 and UT2 is considered. It is assumed that the first user UT1 is about to launch an immersion session and detects on the service portal that a second user UT2 is connected. He gets in touch with him, for example using a "chat" application, to invite him to participate in the immersion session that is about to start. If the second user UT2 accepts the invitation, the service platform establishes a VoIP connection between the two users UT1 and UT2. They can then discuss together.

The first user then launches the session, which causes the service platform to send the audio and video streams to the terminals of the two users. We now consider the case of a session planned and posted on the portal of the immersion service. Users who have taken note of this appointment will "register" on the portal. Once their registration is validated, the service platform launches their connection on a spatialized conference bridge. VoIP communication is in fact established between users registered for the session. A session administrator launches the immersion, which results in the launch of the streaming of the constituent data streams of the multimedia scene to all users registered for the session. As noted above, latecomers may register for an ongoing session. They receive streamed streams in progress from the playback position of other users.

Of course, these various embodiments can be combined, so that a user can both react to the part of the scene he visualizes by a voice signal and his angular position is transmitted to other users and to the platform of service for use.

Examples of immersive communication services are now considered, which will be detailed in support of the flow diagrams presented in FIGS. 4 and 5. EXAMPLE 1 Immersive Two-Way Communication According to a first embodiment of the invention, in relation to the flow diagram of FIG. 4, a first user Yann and a second user Elodie are considered. Yann and Elodie are each at home. They each have an HMD which can connect to a processing device according to the invention integrated into a pod (eg PC, multimedia player, 3G mobile phone). The pod is itself connected to the Internet either by an ADSL / fiber optic modem, or in 3G. Yann wants to talk with Elodie while remaining immersed in a 360 ° environment consisting for example of video content or spherical photos. In a first step 1, Yann selects the content that he wishes to view in a catalog of contents accessible on the web portal of the service.

In step 2, it gets on its processing device the list of other users connected to the Internet (for example, their name appears in green among the list of potential users). Advantageously, when the terminal is connected to the Internet, it periodically declares the user as connected to the application server of the service platform, which stores this information and sends it to the other connected users.

In step 3, Elodie being declared as connected, Yann sends her an invitation. This invitation is sent to the Elodie terminal via the service platform. If the guest accepts the invitation, the service platform establishes in step 4 a VoIP connection between Yann and Elodie .. The voice communication channel is then established between the two. They can talk together.

When the relationship is established (the two people are talking to each other) and their HMD devices are connected to their respective POD processing devices, Yann initiates the immersion in step 5. This step includes triggering the streaming of the audio and video streams constituting the chosen content, between the content server and the Yann terminal on the one hand, and between the content server and Elodie's terminal on the other hand. Of course, both streaming operations are synchronized temporally. It will be noted that, advantageously, the contents are encoded according to standard compression formats which make it possible to minimize the bandwidth used (for example, according to the H.264 format), which imposes a decoding of this format by the users' terminals. . Each user is free to move in the 360 ° by turning his head. The angular positioning of each user, detected by the gyroscope of his HMD, is raised to the level of the processing device according to the invention, which selects the part of the sphere visualized at 6 and 7 and sends the corresponding image to the micrometers. HMD displays.

For the audio part, streamed audio streams are advantageously encoded according to a spheric audio format. The spheric audio format describes the distribution of sounds on a sphere. A filter matrix calculates the acoustic pressures at the inputs of the auditory canals of a listener generated by the sounds distributed over the sphere. The filter matrix depends on the angular position of each user. Taking into account the geometry of the head and the positions of the ears, these acoustic pressures depend on the morphology of the listener and his orientation. Applying this filter matrix to the spherical audio stream generates a stereo sound stream that is fed back to the HMD's headphones. Thus, Yann and Elodie discover the 360 ° multimedia scene at the same time, but independently. Because of the voice communication between the two, everyone can benefit from the advice of the other ("look right", "look up") and share his emotions. Throughout the experience, a gesture control allows each user to interact with the content (zoom, increase volume, re-centering the scene ...). According to one aspect of the invention, the angular positioning information PA1, PA2, for example the angular sector of the visualized scene part and / or the angular coordinates of the user's head in a reference system of the HMD device, are reassembled. , at 8 and 10 by each terminal to the PFS service platform, which retransmits this information to the other terminal. Upon receipt of this information, the terminals of Yann and Elodie transform the information received into a visual indication that they superimpose at 9 and 11 to the scene portion they view in their HMD device. This visual indication allows them to locate the position of the other user in relation to their own angle of view. In the configuration described, the processing device (player) is located at the POD. As a result, the 360 ° film is "streamed" completely to this POD and the user's angular position information is raised to the level of the POD for the calculation of the angular sector of the sphere to be projected on the spherical support.

According to another aspect of the invention, the angular position information PA1 'of the head is sent back to 12 at the level of the service platform, transmitted at 13 to the content server and used to adapt the data streams DS1 to 14 it remains to transmit in streaming mode to the terminal of the user. Data streams DS1 'adapted to the angular positioning information PA1' of the user Yann are transmitted at 15 to the terminal of this user, then visualized at 16. Such an adaptation consists in calculating the angle of view corresponding to the angular position of the user at the level of the service platform and to transmit to the terminal of this user only the video stream and the stereo audio stream corresponding to this angle of view. One advantage is to reduce the bandwidth needed to use the service. Example 2: The immersive and guided tour of extraordinary places According to a second exemplary embodiment of the invention, which is presented in relation to the flow diagram of FIG. 5, a distribution service of an audiovisual scene is considered 360 degrees relative to the guided tour of the forbidden city in China. This presentation is made on the basis of a spherical content shot in the forbidden city (video and / or 360 ° photos with multichannel sound, Ambisonic or HOA spatialized). The visit is scheduled for Monday, June 26 at 14:00. When the day comes, each visitor, from their home and using their terminal or POD, just register on the service portal (step 1). Visitors are all equipped with an HMD device connected to a POD, which is itself connected to the Internet. The guide G, from his place of work is also registered on the portal of the service. The session registration automatically generates in step 2 the connection of each of the visitors on a conference call bridge (whose administration functions are managed by the guide). Thus the guide and the group of visitors can now interact together. When the guide judges the right moment, he starts the audio-video immersion (step 3). The constituent video and audio streams of the 360-degree audiovisual scene are thus broadcast in streaming mode to the plurality of users or registered visitors (step 4).

The visit begins. Each visitor receives on its pod the 360 ° content and visualizes as it pleases the part of the sphere that interests it in its HMD device. The stereo sound that arrives at the stereo headphones of the HMD device is calculated based on the orientation of the HMD. The guide comments on the visit. It provides explanations of the scenes viewed and invites participants to look at certain details of the scene. In return, each visitor can ask questions to the guide and benefit from his answers live. All these steps are similar to those described in the flow diagram of the previous example, in relation with FIG. 4. In this embodiment, it should be noted that the guide remains master of the control of the content, that is to say say that it can pause it reading content at any time on all terminals, for example to take the time to answer questions.

Advantageously, each user has the opportunity to interact with the content throughout the experience using a gesture remote control. For example, this remote control allows it to perform the following actions: zoom, increase sound volume, refocusing the scene and so on. According to one variant, the positioning information of the guide in the sphere is transmitted to the terminals of the visiting users and their processing devices superimpose a visual indication, for example an arrow to the part of the scene that they visualize to encourage them to go and watch the part of scene commented by the guide.

The operating principle is the same as that of the previous example at

difference that the call VolP is replaced by a connection to an audio bridge. Example 3: The football multiplex In this example, it is again considered a connection service of a group of people (eg group of friends, group of collaborators) previously registered.

Registration for the session is carried out on the service portal. The organizer is the session administrator. When it opens the session, the registered people are automatically connected to a spatial telephone bridge that assigns a positioning of each person in the space and that restores this positioning to others. Each user is equipped with an HMD device connected to a POD, which is itself connected to the Internet. Unlike the previous examples, the nature of the scene is different. It is no longer a single 360-degree multimedia scene in which the user navigates by turning his head. The scene this time consists of several contents or applications spread over a sphere. By turning his head from right to left and from top to bottom, the user moves from one application to another. Applications that can take the form, for example of a ppt file, an xls file, a doc file or a standard TV stream, are positioned in different zones of the sphere (for example angular sectors or according to a mosaic). In an initial position, the client visualizes in full screen in its HMD device a first application (ex: excel file). When he turns his head, the device HMD back the positioning of it, it will gradually arrive in the sphere area where a second application is positioned (eg ppt file). When it reaches this zone, the processing device displays the second application in the HMD of the user. It is thus clear that the processing device according to the invention selects the application to be displayed as a function of the user's positioning in the sphere. If he continues his head movement, he will access other applications. Similarly by gradually tilting the head. Thus, the user can view a multitude of applications in "full screen" size without having to have a multitude of screens (the HMD visualizes a different screen by area assigned to the sphere).

Each of the participants can thus navigate among the applications available independently, without being dependent on that which is viewed and shared by the presenter. Because of the parallel conference call, each user can intervene and encourage others to refer to a specific application by moving in the sphere ... As for the previous examples, it is possible to trace the positioning information of each participant to the service platform and then to the other participants. Thus each participant can have a real-time feedback on the applications viewed by the other participants. The positioning of the applications can be done by administrator profiles defined by the session organizer. According to a variant of this example, consider the case where each application is a TV retransmission of a football match, streamed in standard TV format). This is for example the ten matches of a day of championship that are positioned on the viewing sphere. By turning and tilting the head, the user moves from one match to another. He makes his comments live to his friends connected to the same session as him. In the same way as in the previous examples, the other friends are encouraged to visualize the same match as this user using a visual or sound signal superimposed on the part of scene that they visualize. Advantageously, the spatialized sound feedback gives him an auditory watch of what is happening on the other matches.

As illustrated in FIG. 6, a device for processing a 360-degree multimedia scene according to the invention comprises a memory DV1 comprising a buffer memory, a processing unit DV2, equipped for example with a microprocessor pP, and driven by the program DV3 computer, implementing the treatment method according to the invention. It is further connected to an HMD device comprising a 360 degree viewing medium centered on the user, for example spherical glasses. It finally comprises a communication interface with the network through which it receives the constituent data streams of the 360-degree multimedia scene and feedback information from other users of the service. The processing device according to the invention can be integrated into a user terminal UT1, UT2, UT3, for example a PC ("personal computer", in English), a mobile phone or an intermediate box or Pod insurance. interface between the user's HMD device and the service platform. At initialization, the code instructions of the computer program DV3 are for example loaded into a RAM before being executed by the processor of the processing unit DV2. The processing unit DV2 receives as input the data streams constituting the 360-degree multimedia scene. The microprocessor of the processing unit DV2 implements the steps of the processing method described above, according to the instructions of the computer program DV3. For this purpose, the processing device comprises, in addition to the buffer memory DV1, means for receiving at least one video stream and at least one audio stream constituting the 360-degree multimedia scene originating from the telecommunications network NT. means for decoding said streams as and when they are received, means for detecting an angular orientation of the user with respect to a reference frame of the spherical support, means for calculating the portion of the scene 360 ° displayed and sound reproduced as a function of the detected angular position, means for displaying the part of the computed 360 degree multimedia scene and rendering of the associated sound, and means for transmitting information relating to the part of the scene displayed and listened to by the user for at least one second user terminal for use. It further comprises means for processing information relating to a scene portion displayed by another user of the service.

As illustrated in FIG. 7, the device for distributing a 360-degree multimedia scene to a plurality of user terminals comprises a memory DD1 comprising a buffer memory, a processing unit DD2, equipped for example with a microprocessor pP, and driven by the computer program DD3, implementing the diffusion method according to the invention.

At initialization, the code instructions of the computer program DD3 are for example loaded into a RAM memory before being executed by the processor of the processing unit DD2. The processing unit DD2 receives as input an access request to the immersive communication service from a user terminal. The microprocessor of the processing unit DD2 implements the steps of the broadcasting method described above, according to the instructions of the computer program DD3. For this, the broadcasting device comprises, in addition to the buffer memory DD1, means for distributing said streams to the terminals, a terminal being able to decode the video and audio streams as and when they are received and to display at least a part of the decoded multimedia scene on a user-centered 360-degree viewing medium, means for receiving, from at least a first terminal of said plurality of terminals, information relating to the displayed and listened scene portion by the user of this first terminal and means for retransmission of said information to at least a second user terminal. According to one aspect of the invention, it further comprises means for processing information relating to the part of the scene viewed and listened to by the user of the first terminal and of adapting data streams transmitted to the first user to said information.

Claims (12)

REVENDICATIONS1. A method of processing a multimedia scene distributed in a telecommunications network by a server to a plurality of user terminals connected to said network, characterized in that said method comprises the following steps, intended to be implemented by a terminal said plurality of terminals, said terminal being connected to a 360 degree viewing medium centered on a first user; receiving at least one video stream and at least one audio stream constituting the 360 degree multimedia scene; decoding of said streams as and when they are received; detecting an angular positioning of said user with respect to a repository of the visualization medium; displaying at least a portion of the 360-degree multimedia scene decoded on said 360-degree viewing support and rendering of the associated sound, as a function of said detected angular positioning; transmitting information relating to the part of the scene displayed by the user to at least one second user terminal for use. 2. Processing method according to claim 1, characterized in that the information relating to the part of the scene viewed and listened to by the first user comprises at least one voice signal representative of a reaction of the user to the scene part that he visualizes. 3. Treatment method according to claim 1, characterized in that it comprises a step of receiving information relating to a scene portion displayed and listened to by a third user and a step of adapting the displayed scene part and listened to using the information received. 4. Processing method according to claims 3 and 4, characterized in that the voice signal of reaction issued by the third user is transmitted in the processing device of other users with the angular orientation returned by the terminal of the third user for information about the part of the scene he is viewing and listening. 5. Processing method according to claim 1, characterized in that the multimedia scene consists of a plurality of contents or applications, and metadata describing a spatial organization of said contents relative to each other on the 360-degree support. 6. Processing method according to claim 5, characterized in that the information received from a third user during the reception step comprises an identifier of the content of the plurality of contents displayed by the third user and the step of adaptation comprises an overlay to the projected scene to the first user of a visual or audible indication of the content viewed and listened to by the third user and identified from said received identifier. A method of distributing at least one video stream and one audio stream constituting a 360-degree multimedia scene to a plurality of user terminals connected to a telecommunications network, characterized in that said method comprises the following steps: distribution of said streams to the terminals, a terminal being able to decode the video and audio streams as and when they are received and to display at least a portion of the decoded multimedia scene on a 360 degree support centered on the user; receiving, from at least one first terminal of said plurality of terminals, information relating to the scene portion displayed by the user of said first terminal; retransmission of said information to at least one second user terminal. 8. Device for distributing at least one video stream and a constituent audio stream of a 360-degree multimedia scene to a plurality of user terminals connected to a telecommunications network, characterized in that said device is able to implement the following means: distribution of said streams to the terminals, a terminal being able to decode the video and audio streams as and when they are received and to display at least a part of the decoded multimedia scene on a 360 degree support centered on the user; receiving, from at least a first terminal of said plurality of terminals, information relating to the part of the scene viewed and listened to by the user of this first terminal; retransmission of said information to at least one second user terminal. 9. Device for processing a 360-degree multimedia scene capable of being connected to a user terminal of said plurality of terminals and connected to a viewing medium 360 centered on a first user, characterized in that it comprises the means following: reception of at least one video stream and at least one audio stream constituting the 360-degree multimedia scene; decoding of said streams as and when they are received; detecting an angular orientation of said user with respect to a repository of the visualization medium; displaying at least a portion of the decoded 360 degree multimedia scene on said display medium and rendering the associated sound, according to the detected angular position; transmission of information relating to the part of the scene viewed and listened to by the user for at least one second user terminal for use. 10. System for distributing a 360-degree multimedia scene to a plurality of user terminals connected to a telecommunications network, characterized in that it comprises at least one device for distributing a multimedia scene according to claim 8 and at least one multimedia scene processing device according to claim 9. 11. Computer program characterized in that it comprises instructions for the implementation of a processing method according to claim 1, when the program is executed by a processor. 12. Computer program characterized in that it comprises instructions for the implementation of a distribution method according to claim 7, when the program is executed by a processor.