WO2025012524A1 - Method and device for the personalized displaying of a scene on a display medium on the basis of one or more images of said scene - Google Patents

Abstract

The invention relates to a method (100) for representing a scene on the basis of at least one image of said scene, the method (100) comprising at least one iteration of a displaying phase (102) comprising the following steps: - displaying (104) an image of the scene (IC), referred to as current image, on a first display medium, - detecting (106), a position, referred to as target position, of interest to a user in the current image (IC), - obtaining (108) an image (IN), referred to as new image, sharply displaying the scene in the target position, - displaying (110) the new image on at least a second display medium viewed by at least a second user. The invention also relates to a computer program and a system implementing such a method.

Description

DESCRIPTION

Title: Method and device for personalized display of a scene from image(s) of said scene, on a display medium

[0001] The present invention relates to a method for representing a scene allowing a personalized display of a scene on a display medium. It also relates to a computer program and a system implementing such a method.

[0002] The field of the invention is the field of displaying a digital image on a display medium, such as a display screen or a projection surface.

State of the art

[0003] When the human eye scans a scene, it observes said scene according to different focalizations to perceive the entirety of a scene. The focalizations used to scan a scene depend directly on the depth of the objects in said scene. Generally, to observe a part of a scene, the human eye adapts the focalization to the depth of the scene in said part, to obtain a clear perception of the scene in said area.

[0004] However, the methods for representing a scene currently known perform the display of an image of said scene according to a single focus, namely the focus used to capture said image. When the scene comprises parts located at different depths, the focus corresponds to one of said depths so that only one of the parts of the scene is clear in the displayed image.

[0005] Such a display gives a very limited and very static representation of the scene. It does not adapt to the interest that a user may have in this or that part of the scene represented on a display medium. In the case of a speaker observing the scene, it does not allow the display of said scene to the participants, depending on the area the speaker is interested in.

[0006] An aim of the present invention is to remedy at least one of the drawbacks of the state of the art.

[0007] Another aim of the invention is to propose a solution allowing a representation of a scene on a display medium which is more suited to a person observing said scene on said medium.

Disclosure of the invention

[0008] The invention proposes to achieve at least one of the aforementioned aims by a method of representing a scene from at least one image of said scene, said method comprising at least one iteration of a display phase comprising the following steps:

- display, on a first display medium, of an image of the scene, called the current image;

- detection, on said current image, of a position, called target position in which a first user, called speaker, is interested;

- obtaining an image, called a new image, clearly displaying said scene at said target position; and

- displaying said new image on at least one second display medium observed by at least one second user, called a participant.

[0009] Thus, in a conventional manner, the method proposes to display an image, called the current image, of the scene on a first display medium. This current image comprises a sharpness zone corresponding to a given depth of field: all the other zones of the current image are blurred. The method according to the invention also proposes to determine a target position in which a speaker, observing the current image displayed on the first display medium, is interested. Then, this target position is used to obtain another image of the scene in which the target position is in a sharp zone, this new image is in a way an adjusted or refocused image. This new image is then displayed on at least one second medium observed by one or more participants, different from the speaker.

[OO1O] Thus, the display of the scene for each participant is personalized and adapts to the position in which the speaker is interested, so that the area observed by the speaker is always displayed clearly for the participant. In other words, according to a non-limiting exemplary embodiment, the clear area of the scene displayed on the second display medium for the participant follows the gaze of the speaker. Unlike the prior art, the focusing distance and optionally the depth of field of an image of a scene is not static, adapts and adjusts according to the speaker.

[0011] Thus, the or each speaker can follow the gaze of the speaker who navigates in the scene.

[0012] According to the invention, each display medium can be any type of display medium.

[0013] For example, the first display medium, and/or the second display medium, may be an electronic display screen.

[0014] For example, the first display medium, and/or the second display medium, may be a surface onto which an image of the scene is projected, such as for example a display surface associated with a projector, etc.

[0015] The first display medium, and/or the second display medium, may be a display medium equipping a device participating in carrying out the display phase, such as:

- a display screen equipping said device; or

- a projection surface equipping said device.

[0016] The display medium may be a medium independent of the device participating in carrying out the display phase, for example in the context of a projector projecting the images onto a display surface.

[0017] According to embodiments, the target position may be a point position on the current image. [0018] According to embodiments, the target position may be an area of the current image.

[0019] Advantageously, the method according to the invention can further comprise a step of displaying the new image on the first display medium.

[0020] Thus, the representation of the scene on the first display medium also adapts to the target position observed by the speaker.

[0021] According to embodiments, the step of obtaining the new image can be carried out on the speaker's side, and in particular on the same site as the speaker, said new image being transmitted to the participant.

[0022] In particular, when the invention is implemented by a system comprising a first apparatus on the speaker side and a second apparatus on the participant side, then the step of obtaining the new image can be carried out at, and in particular by, the first apparatus.

[0023] According to embodiments, the step of obtaining the new image can alternatively, or in addition, be carried out on the participant's side, and in particular on the same site as the participant, said method comprising a transmission of the target position to the participant.

[0024] In particular, when the invention is implemented by a system comprising a first apparatus on the speaker side and a second apparatus on the participant side, then the step of obtaining the new image can be carried out at, and in particular by, the second apparatus.

[0025] According to embodiments, the step of obtaining the new image can be carried out elsewhere, for example at a device which is remote from the speaker and the participant.

[0026] In particular, when the invention is implemented by a system comprising a server located at a distance from the speaker and the participant, the obtaining step can be carried out at said server. In this case, the target position is transmitted to said server, and the new image obtained at level of said server is transmitted to the participant for display, and optionally to the speaker for display.

[0027] According to embodiments, the step of obtaining the new image can comprise a generation of the new image, by calculation.

[0028] Indeed, the new image can be generated on the fly after the target position is determined. In this case, the new image does not exist before the target position is known. Once the target position is known, then the new image is generated.

[0029] The new image can be generated from the current image.

[0030] For example, the new image may be generated by blurring the currently sharp area of the current image and sharpening the area comprising, or corresponding to, the target position.

[0031] The new image may be generated from a source image, which is not the current image.

[0032] For example, the source image may be a previously obtained image representing the scene in a clear manner in all areas. In this case, a copy of this source image is made with modification of the sharpness: the target area is kept clear and all the other areas are blurred, to obtain the new image.

[0033] Such a source image can be obtained by taking several images of the scene with different depths of field, for example by the focus bracketing technique. Then, the sharp area of the scene is selected in each image. All the sharp areas obtained from all the images can be concatenated to obtain an image of the scene sharp in all areas.

[0034] Such a source image can also be obtained, by simulation or by calculation, from a 3D modeling of the scene (geometry + textures). In this case, the calculation consists of synthesizing an image of the scene, which in conventional methods would be intrinsically sharp everywhere, and to "reblur" it on the fly according to the target area, by applying a blur function linked to the difference in depth between each point of the scene to be represented, and the depth of the target area. [0035] According to embodiments, the step of obtaining the new image can comprise a step of selecting the new image from a stack of images of said scene, each having a different sharpness zone.

[0036] In this case, the new image is not generated on the fly. It exists prior to the identification of the target position. This embodiment allows greater responsiveness of the method according to the invention.

[0037] In this case, the obtaining step may comprise an identification, in the image stack, of the image on which the target position is clearly represented. This image is selected as a new image to be displayed during the displaying step.

[0038] The identification of the new image in the image stack can be carried out in different ways. For example, each image in the stack can be associated with information indicating the sharp area on said image. In this case, the identification step consists in identifying which of the sharp areas comprises the target position and selecting the image associated with this sharp area.

[0039] According to embodiments, the image stack may comprise at least one image captured by an imaging device, such as a camera.

[0040] In particular, the image stack may have been captured by one or more cameras, by implementing the focus bracketing technique providing several images of a scene, each with a different depth of field and therefore a different sharp area.

[0041] According to embodiments, the image stack can comprise at least one image, obtained by calculation or by simulation, from at least one other image.

[0042] This other image may be an image comprising a sharp area and blurred areas. In this case, the calculation to obtain an image of the stack consists in generating an image comprising a depth of field different from that of the other image. The sharp area of the other image is made blurred and another area of the scene is made sharp. To do this, the calculation consists in determining as best as possible the transfer function of the original optics as a function of the defocusing, and in determining the transfer function that we wishes to simulate given the distance of the points to be represented, i.e. as a function of the simulated focusing distance and the distance of the target area. Traditionally, one might be tempted to find total sharpness at all points by inverting the optical transfer function (deconvolution) before applying parametric deblurring. To minimize artifacts related to noise and modeling errors, and the amount of calculations that controlling these effects involves, we focus here on calculating only an incomplete deconvolution, which reduces the transfer function of the original optics to the transfer function that we wish to simulate, which is also incompletely sharp over most of the image.

[0043] This other image can also be an image of the scene that is sharp in all areas, such as the source image described above.

[0044] The new image can be selected in several ways.

[0045] According to embodiments, the display phase may further comprise, after identification of the target position, a step of identifying an object located at the target position. In this case, obtaining the new image may be carried out as a function of said object.

[0046] In particular, the selection of the new image may comprise the identification of the image on which said object is clearly represented. To do this, each image in the stack may be associated with information indicating the object(s) clearly represented on said image.

[0047] According to embodiments, the display phase may comprise, after identification of the target position, a step of identifying a zone, called target, of the scene comprising the target position. In this case, obtaining the new image may be carried out as a function of said target zone.

[0048] In particular, the selection of the new image may comprise the identification of the image on which said target area is in focus. To do this, each image in the stack may be associated with information indicating the area represented in focus on said image. [0049] According to embodiments, the display phase may comprise a step of determining depth data of the scene at said target position, the new image being obtained as a function of said depth data.

[0050] Indeed, once the target position has been identified, the depth of the scene at said position can be determined. The depth information of the scene at the target position can be calculated on the fly, for example by triangulation with another image of the scene. Alternatively, the depth information of the scene at the target position may have been measured when the scene was imaged, by suitable measurement means, such as a time-of-flight camera, lidar, etc.

[0051] This depth information can be used, during the obtaining step, to select the new image in a stack of images: in this case, the image for which the depth of field corresponds to or includes the depth of the target position is identified. To do this, each image in the image stack can be associated with information indicating for said image the depth of field of said image.

[0052] The target position in which the user is interested can be detected in different ways.

[0053] According to embodiments, the target position may be manually entered by the speaker.

[0054] In this case, it is the speaker who manually selects the target position on the current image, for example using a pointer, such as a mouse, a trackball, or a directional pad moving a cursor, or any other position input device. The speaker can also enter the target position by touch selection on a touch surface arranged on the display medium in the case, for example, of a touch display screen, or remote from the display medium.

[0055] The step of detecting the target position may comprise a detection of a zoomed region of the current image by the speaker. [0056] Indeed, when the speaker applies a zoom to a region of the current image displayed on the first display medium, this probably indicates that he is paying attention to the part of the scene located in said zoomed region of the current image. In this case, the target position may be the zoomed region, or may be a position in said zoomed region, such as for example a central position of said zoomed region.

[0057] The zoom can be applied in a known manner:

- tactilely by applying a movement, for example with two fingers. In this case, the target position can be the position equidistant from the positions of the fingers applying the zoom;

- using a pointer surrounding the region to zoom;

- etc.

[0058] The step of detecting the target position may comprise, or be carried out, by a detection, by at least one sensor, of a position targeted by a part of the speaker's body, on the current image.

[0059] In this case, in particular, said body part may be at a distance from the first display support, and does not come into contact with said first display support or an associated control surface.

[0060] This part of the body can be a finger, or a hand, pointing the target position on the current image. In this case, it is the position aimed at by the finger or the hand located at a distance from the display support which is detected.

[0061] This part of the body can be an eye, or the eyes. In this case, it is the position targeted by the eye or eyes that is detected.

[0062] This part of the body may be a face. In this case, the orientation of the face is detected and a position aimed at by a point of the face, for example by the center of the face, is detected.

[0063] The position targeted by the body part may be detected for example by at least one sensor equipping the first display support, or a device integrating the display support. The sensor may alternatively be arranged at a distance from said first display support.

[0064] The at least one sensor for detecting the position may be an optical sensor, an acoustic sensor, a capacitive sensor, etc. For example, in the case where the first display medium is a touch screen, then the sensor may be a capacitive sensor equipping said touch screen.

[0065] The at least one sensor, for detecting the position of attention, or at least the movement of this position, may be an orientation sensor such as an accelerometer. For example, in the case where the first display medium is a screen held in the hands of the speaker or participant, an accelerometer detecting its inclination or its changes in inclination relative to an initial inclination, or even translational movements of said medium, for example by double integration of the acceleration signal, may be used to detect a change in attention, assuming for example that the user's gaze would represent a fixed axis, and that the medium would be moved relative to this axis of attention, or even that the attention would come from an axis remaining perpendicular to said medium and passing through the user's eye, which amounts to associating with the variations in angle of the medium also a movement of the attention zone. This detection mode can also be used in addition to other detection means such as the distance between the support and the eye of the observer of the support or in addition to the estimation of the position targeted by the eye. [0066] Of course, other types of sensors can be used and the invention is not limited to a specific type of sensor.

[0067] The display phase can be repeated as many times as desired with a different target position, from one iteration to another. Thus, the speaker can move through an image to observe different parts of the image, each part being displayed in a sharp manner for the participant when it is observed by the speaker. Thus, the sharpness area of the image is adjusted, for the participant, as the scene is moved through by the speaker.

[0068] This allows for more ergonomic and more personalized navigation in an image. Above all, it allows a scene displayed on a display medium to be observed more realistically, as if said scene were actually in front of the participant and/or the speaker. [0069] The invention has just been described with reference to a representation of the scene, at a given instant. In other words, preferably, the current image and the new image represent the scene at the same instant.

[0070] Of course, it can be applied to a representation of a scene at several times. For this, the display phase can be repeated for several images, or series of images, each captured at a different time, for example part of a video. In this case, each image or series of images represents the scene at a different time.

[0071] According to another aspect of the invention, there is provided a computer program comprising executable instructions which, when executed by a computing device, implement all the steps of the method according to the invention.

[0072] The computer program may be in any computer language, such as for example machine language, C, C++, JAVA, Python, etc.

[0073] According to another aspect of the invention, a system is proposed comprising means configured to implement all the steps of the method according to the invention.

[0074] In particular, the system according to the invention may comprise:

- a first display support,

- at least one second display support, and

- at least one computing unit; configured to implement the method according to the invention

[0075] According to one embodiment, the system may comprise:

- a first device equipped with a first display support, located on the speaker's side, and

- a second device equipped with a second display support, located on the participant's side. In this case, the step of obtaining the new image can be implemented in the first device or in the second device, as described above with reference to the method according to the invention.

[0076] According to one embodiment, the system may comprise:

- a first device equipped with a first display support, located on the speaker's side,

- a second device equipped with a second display support, located on the participant's side, and

- a server located between the first device and the second device.

In this case, the step of obtaining the new image can be implemented in the first device or in the second device, or even in the server, as described above with reference to the method according to the invention.

[0077] According to one embodiment, the system may comprise:

- a first display support, located on the speaker's side,

- a second display stand, located on the participant's side, and

- a common device, connected to said display media.

In this case, the step of obtaining the new image is implemented in the device and displayed on the second device, and optionally on the first display medium.

[0078] The common device, or at least one or more first and second devices, may be a user device of the Smartphone, tablet, etc. type comprising a display screen. In this case, the detection means may be or may comprise a touch surface, in particular integrated into, or associated with, the display screen of said device.

[0079] The common apparatus, or at least one or more first and second apparatuses, may be a computer-type user apparatus, comprising a display screen. In this case, the detection means may be or may comprise a touch surface, in particular integrated in, or associated with, the display screen of said computer, or a pointer moved for example by a mouse, or a directional pad of said computer.

[0080] The common device, or at least one or more first and second devices, may be a television. In this case, the detection means may be a camera integrated into said television, detecting the gaze and the position of the head of the observer, or a pointer moved for example by a remote control of said television.

[0081] The common device, or at least one or more first and second devices, may be a video headset, a virtual reality or augmented reality headset comprising a display screen or a projector associated with a projection surface on which each image is projected. In this case, the detection means may be or may comprise a sensor, in particular an optical sensor, equipping said headset.

[0082] The common apparatus, or at least one or more first and second apparatuses, may be a medical imaging apparatus. In particular, the apparatus may be an endoscope, an ultrasound apparatus, etc.

[0083] The common apparatus, or at least one or more first and second apparatuses, or part of the image capture of the system implementing the method, may be an inspection apparatus in a difficult-to-access environment such as a pipeline, or a location presenting a particular danger (such as exposure to radiation or an area threatening to collapse). In particular, the apparatus may be an endoscope, an ultrasound apparatus, or a mobile camera on a robot, etc.

[0084] The common apparatus, or at least one or more first and second apparatuses, may be arranged, or integrated, in a vehicle or several vehicles comprising:

- a means of displaying an image, and

- at least detection of a target position,

- at least one calculation means; configured to implement all the steps of the method according to the invention. [0085] The vehicle may be common to the first and second devices, or said first and second devices may be in different vehicles.

[0086] The common device, or at least the first device, may be arranged, or integrated, in a device for taking at least one or more video surveillance images, such as a video camera monitoring, for example in a non-exhaustive manner, road traffic or an urban environment, or premises to be protected, or even a place receiving the public or a gathering of people, or a conference room.

[0087] The second device may be integrated into a vehicle, or be a smartphone or even a computer, or a display screen.

[0088] Thus, for all of these implementation examples, said first and second devices may be of different types, or integrated into different objects, all combinations are possible such as, in a non-limiting manner, the first device integrated into a smartphone, or into an endoscope, and the second device integrated into a computer or into a vehicle, for example.

[0089] According to embodiments, at least one image of the scene can be a 2D image. In particular, the current image is a 2D image. In particular, the new image is a 2D image.

[0090] Where applicable, the image stack comprises at least one 2D image. In particular, each image in the image stack is a 2D image.

[0091] If applicable the clear image everywhere is a 2D image.

[0092] According to embodiments, at least one image of the scene is a 3D image. In particular, the current image is a 3D image. In particular, the new image is a 3D image (therefore comprising a depth field).

[0093] Where applicable, the image stack comprises at least one 3D image. In particular, each image in the image stack is a 3D image.

[0094] If applicable the clear image everywhere is a 3D image. Description of figures and embodiments

[0095] Other advantages and characteristics will appear on examining the detailed description of non-limiting embodiments, and the attached drawings in which:

- FIGURE 1 is a schematic representation of a non-limiting exemplary embodiment of a method according to the invention;

- FIGURES 2 and 3 are schematic representations of non-limiting exemplary embodiments of a step of obtaining a new image that can be implemented in the present invention;

- FIGURE 4 is a schematic representation of a non-limiting exemplary embodiment of obtaining a clear image everywhere;

- FIGURES 5-7 are schematic representations of other non-limiting exemplary embodiments of a method according to the invention; and

- FIGURES 8-9 are schematic representations of non-limiting exemplary embodiments of a system according to the invention; and

[0096] It is understood that the embodiments that will be described below are in no way limiting. In particular, it will be possible to imagine variants of the invention comprising only a selection of features described below isolated from the other features described, if this selection of features is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. This selection comprises at least one preferably functional feature without structural details, or with only part of the structural details if it is this part that is only sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art.

[0097] In particular, all the variants and all the embodiments described can be combined with each other if nothing prevents this combination from a technical point of view. [0098] In the figures and in the remainder of the description, the elements common to several figures retain the same reference.

[0099] FIGURE 1 is a schematic representation of a non-limiting exemplary embodiment of a method according to the present invention.

[0100] The method 100 of FIGURE 1 can be used to produce a personalized display of a scene from image(s) of said scene.

[0101] The method 100 comprises a display phase 102 allowing the clear display, on the fly, of an area of a scene from at least one image of said scene.

[0102] The display phase 102 comprises a step 104 during which an image IC of the scene is displayed on a first display medium. This current image IC is called current image in the following.

[0103] The first display medium may be an electronic screen, and in particular a touch screen. The first display medium may alternatively be a display surface, such as a wall, a board, a panel or a canvas, onto which the image is projected, for example by a projector.

[0104] This first display medium is observed by a first user. In the following, and without loss of generality, this first user is called a “speaker”.

[0105] The current image IC may be one captured by an imaging device, such as a camera, or a camera module. The current image may also be an image generated by calculation. The current image displayed on the first display medium has a given depth of field such that a region of the scene is displayed sharply and the rest of the scene is displayed blurred.

[0106] The display phase 102 comprises a step 106 of detecting, on the current image IC, a target position POS of the scene on which the speaker is focusing his attention. [0107] This target position POS can be a point position, or an area not limited to a point position.

[0108] The target position POS can be detected in different ways, some non-limiting examples of which are given above.

[0109] In particular, the target position POS can be selected with a pointer, such as a mouse or the like. In this case, the speaker moves the pointer to the desired target position.

[0110] The target position POS may be a position indicated by a part of the speaker's body, such as a finger or a hand, by contact on a touch-sensitive surface, integrated or not in the first display medium. For example, the target position POS may be indicated by contact on a touch screen forming the first display medium.

[YES] The target position POS may be a position aimed at by a part of the speaker's body, such as a finger, a hand, an eye, the eyes or the face of the speaker, which does not come into contact with the first display medium. In this case, the position aimed at by said part of the speaker's body on the current image IC, and therefore on the first display medium, may be detected in any known manner, for example by at least one sensor, integrated or not in the first display medium, such as a capacitive sensor, a camera, etc.

[0112] Once the target position POS is known, we then know which part of the scene should be displayed sharply.

[0113] The display phase 102 comprises a step 108 of obtaining a new image clearly displaying the scene at the target position POS, or a target area of the scene comprising said target position POS. Preferably, this new image represents the scene at the same time as the current image. Alternatively, this new image may represent the scene at a time other than the current image, such as for example at a following time or at a later time.

[0114] Non-limiting examples of obtaining such an image are described below, for example with reference to FIGURES 2 and 3. [0115] The display phase 102 comprises a step 110 of displaying the new image on at least one second medium.

[0116] The, or each, second display medium may be an electronic screen, and in particular a touch screen. The, or each, second display medium may alternatively be a display surface, such as a wall, a board, a panel or a canvas, onto which the image is projected, for example by a projector.

[0117] The, or each, second display medium is observed by at least one second user. In the following, and without loss of generality, this second user is called a “participant”.

[0118] Optionally, but advantageously, the display phase 102 may further comprise a step 112 of displaying the new image on the first display medium. Steps 110 and 112 may be performed in turn, in any order, or at the same time.

[0119] According to embodiments, the step 108 of obtaining the new image can be carried out on the speaker's side, and in particular on the same site as the speaker, said new image being transmitted to the participant. In particular, when the invention is implemented by a system comprising a first apparatus on the speaker's side and a second apparatus on the participant's side, then the step 108 of obtaining the new image can be carried out at, and in particular by, the first apparatus. [0120] Alternatively, or in addition, the step 108 of obtaining the new image can be carried out on the participant's side, and in particular on the same site as the participant. In this case, the target position POS is transmitted to the participant for carrying out step 108. In particular, when the invention is implemented by a system comprising a first device on the speaker side and a second device on the participant side, then step 108 of obtaining the new image can be carried out at, and in particular by, the second device.

[0121] According to embodiments, the step 108 of obtaining the new image can be carried out elsewhere, for example at a device that is remote from the speaker and the participant. In particular, when the invention is implemented by a system comprising a server located at a distance from the speaker and the participant, the obtaining step 108 can be carried out at said server. In this case, the target position POS is transmitted to said server, and the new image obtained at said server is transmitted to the speaker, and optionally to the participant.

[0122] The exemplary embodiment of the method has just been described with reference to a single participant. Of course, the display phase can be carried out for several participants, for the same speaker. In this case, the display step 110 is carried out in a common manner for several participants when the second display medium is common to several participants, or individually for each participant when a second display medium is associated with each participant.

[0123] The display phase 102 can be repeated as many times as desired to change the display of the scene, and in particular to change the area of the scene displayed clearly.

[0124] The display phase 102 can be carried out each time the speaker chooses, or validates, a new target position.

[0125] Alternatively, the display phase 102 can be carried out continuously, for example:

- as soon as the position of a pointer moved by the speaker changes,

- as soon as the position aimed at on the current image by a finger, and more generally by a part of the body, of the speaker changes; or again

- as soon as the contact position of a finger or hand of the speaker changes.

[0126] In some embodiments, the new image displayed during step 110, and optionally during step 112, can be used as the current image for the next iteration of the display phase 102. In this case, step 112 can be confused with step 104, in particular when the display phase 102 is iterated several times for the same scene. Indeed, in this case, the new image IN can be the current image IC for the next iteration of the display phase 102. [0127] FIGURE 2 is a schematic representation of a first non-limiting exemplary embodiment of a step of obtaining a new image, which can be implemented in the present invention.

[0128] Step 200, shown in FIGURE 2, can be implemented to obtain a new image to be displayed, from a target position POS, in the method according to the invention, and in particular in the method 100 of FIGURE 1.

[0129] Step 200 may be, or may be included in, step 108 of method 100 of FIGURE 1.

[0130] In the example shown, the new image is selected from a stack of PIL images of the scene. Each of the images in the stack of images represents the scene with a different depth of field. Thus, each image in the PIL stack represents the scene but on each image a different part of the scene is sharp and all other parts of the scene are blurred. Each image can be associated with at least one of the following data:

- data indicating the area of the image that is in focus, for example with coordinates delimiting said area,

- one or more objects that are displayed clearly in the image, for example by indicating an identifier for these objects, or the position of these objects, and/or

- the depth of field of the image.

In the following, and without loss of generality, we consider that each image is associated with data indicating the depth of field of the image, said depth of field thus indicating the part of the scene which is clearly represented on the image.

[0131] The PIL image stack can be acquired by known focus bracketing techniques, also called "focus stacking" in French, with an imaging device. In summary, the imaging device is controlled to obtain several images of the same scene one after the other. Between each image, the focus of the device is modified so that each image is acquired with a different focus, and therefore with a different depth of field.

[0132] Of course, the PIL image stack can be obtained by calculation, from one or more images of the scene.

[0133] During a step 204, the depth associated with the target position POS, in the scene, is determined. This target position POS has been previously determined, for example during step 106.

[0134] This depth may have been measured, by one or more sensors, when the scene is imaged and stored. Such a measurement may for example be carried out by sensors such as lidar sensors, or time-of-flight cameras, etc. In this case, the depth data is read during step 204. [0135] Alternatively, the depth may be calculated, for example by triangulation techniques, from at least two images of the scene. These triangulation calculation techniques are well known and will therefore not be detailed here. In this case, the depth data is calculated in step 204, or read when the calculation has been carried out previously.

[0136] During a step 206, the depth of the target position POS is compared to the data associated with each image of the stack 202, and indicating the depth of field, that is to say an interval of distances at which a point of the scene is obtained in a sufficiently clear manner on the image, in said image.

[0137] When the depth of the target position POS is included in the depth of field associated with an image of the stack of images PIL, this image of the stack is selected, during a step 206, as a new image. Indeed, as the depth of the target position is in the depth of field of this image of the stack, this means that a part of the scene, including the target position, is represented clearly on this image.

[0138] The image of the stack can thus be selected as a new image. [0139] Optionally, it can be processed during an optional image processing step 208. [0140] In the example described with reference to FIGURE 2, the new image is obtained from the depth associated with the target position.

[0141] Of course, other embodiments are possible. For example, it is possible to identify an object located at the target position, or it is possible to identify a target area comprising the target position. Obtaining the new image can then be carried out as a function of said identified object, or of said identified target area. In this case, in the PIL image stack, the new image is identified and selected during steps 204 and 206, as a function of an identifier of said object, or of said target area.

[0142] According to other modes, it is possible to compare the coordinates of the target position POS to an area displayed in sharpness in each image. If the target position is in the sharp area of an image in the image stack PIL, then said image is selected as the new image to be displayed.

[0143] FIGURE 3 is a schematic representation of a second non-limiting exemplary embodiment of a step of obtaining a new image, which can be implemented in the present invention.

[0144] Step 300, shown in FIGURE 3, can be implemented to obtain a new image to be displayed, from a target position, in the method according to the invention, and in particular in the method 100 of FIGURE 1.

[0001] Step 300 may be, or may be included in, step 108 of method 100 of FIGURE 1.

[0002] In the example shown, the new IN image is generated, on the fly, from an INP image of the scene, sharp everywhere. In other words, in the sharp everywhere INP image, the depth of field potentially covers the entire depth of the scene.

[0003] Such a sharp INP image across the entire scene can be obtained in various ways.

[0004] According to an exemplary embodiment, the everywhere clear image INP can be obtained by assembling images from a stack of images, each having a limited area of sharpness in the scene, such as the image stack

PIL.

[0005] According to another exemplary embodiment, the sharp image everywhere INP can be obtained by calculation, from a single image of the scene.

[0006] A non-limiting example of obtaining a sharp image everywhere will be described later, in particular with reference to FIGURE 4.

[0007] Step 300 may optionally include step 202 determining the depth of the scene at the target position POS, as described above.

[0008] In a step 302, an area of the sharp image everywhere IM is selected, this area comprising the target position POS. This area can be selected arbitrarily, as long as it contains the target position. Alternatively, this area can be selected, according to a predetermined rule: for example, it can correspond to the center of an area of predetermined size and shape. According to yet another alternative, this area can be selected, according to one or more objects located at the target position. According to yet another alternative, this area can be selected according to a depth map of the scene around the position, and a predetermined depth interval: for example it may be desirable for the area to include the objects located around the target position and whose depth, in the scene, is included in an interval centered on the depth of the target position.

[0009] In a step 304, the everywhere clear image IM is blurred everywhere, except in the area selected in step 302. This blurring can be carried out by any known technique. The image thus obtained is used as a new image, possibly after an optional image processing step 306. [0010] FIGURE 4 is a schematic representation of a non-limiting exemplary embodiment of obtaining an image of a scene that is sharp throughout, which can be implemented in the present invention.

[0011] The method 400, shown in FIGURE 4, can be implemented to obtain an image of a scene, clear everywhere, and can be used, in the method according to the invention, and in particular in the method 100 of FIGURE 1, to generate a new image of the scene.

[0012] More particularly, method 400, shown in FIGURE 4, may be implemented to obtain the sharp everywhere INP image used in step 300 of FIGURE 3.

[0013] In the example shown in FIGURE 4, the sharp image throughout is obtained from a stack of images of the scene, comprising several images, each representing the scene with a different depth of field.

[0014] The PIL image stack can be obtained either by an imaging device implementing a focus bracketing technique, or by calculation, as described above.

[0015] The method 400 comprises a step 402 which extracts, for each image from the stack of PIL images, the sharpness zone of said image.

[0016] Extracting the sharpness zone of an image can be done in different ways. For example, we construct a local focus estimator (one possible implementation is to assimilate it to an energy/variance/entropy operator) that we calculate over the entire surface of all the images. We use the hypothesis that for a given image zone, the estimator is maximal when the image is sharpest. Therefore, we determine for each image in the stack, the region(s) in which the value given by the local estimator is the largest of the entire stack of images. The content of these regions is directly copied into the desired result. By repeating the process for all the zones of all the images, we construct a result that at each point contains the information of the image that was sharpest at that location. [0017] The method 400 comprises a step 404, constructing a sharp image everywhere from the areas extracted during step 402. Indeed, once the sharp area of each image of the stack has been extracted, all the sharp areas thus obtained can be concatenated together.

[0018] The concatenation may be performed using any known technique. For example, the concatenation may rely on overlapping areas between the sharp areas, and possibly on objects of interest located in said sharp areas.

[0019] Alternatively, the sharp areas can be concatenated pixel by pixel so as to construct a sharp image throughout.

[0020] In an optional step 406, the clear image everywhere is stored.

[0021] Preferably, but in no way limiting, the sharp image everywhere can be enriched by depth data of the scene. In particular, depth data can be stored in association with at least one, in particular each, pixel or object of the scene. This, or these depth data, can be obtained in each of the images of the PIL image stack for example. Alternatively, the image stack can comprise an image, or a map, giving the depth of the scene for different points, and in particular each point of the scene.

[0022] FIGURE 5 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0023] The method 500 of FIGURE 5 can be used to produce a personalized display of a scene from at least one image of said scene.

[0024] The method 500 of FIGURE 5 includes all of the steps of the method 100 of FIGURE 1.

[0025] The method 500 of FIGURE 5 may further comprise an optional step 502 of obtaining a stack of images of the scene, each image representing the scene over a limited depth of field such that only a portion of the scene is clearly represented in said image. [0026] The image stack may be acquired by known focus bracketing techniques. Alternatively, the image stack may be obtained computationally, from one or more images of the scene.

[0027] For example, the image stack may be the image stack PIL of FIGURES 2 and 4.

[0028] In this case, and without loss of generality, step 108 of obtaining the new image can be carried out in accordance with the example described with reference to FIGURE 2.

[0029] FIGURE 6 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0030] The method 600 of FIGURE 6 can be used to produce a personalized display of a scene on a display medium from at least one image of said scene.

[0031] The method 600 of FIGURE 6 includes all of the steps of the method 100 of FIGURE 1.

[0032] The method 600 of FIGURE 6 may further comprise an optional step 602 of obtaining a clear image throughout.

[0033] The everywhere sharp image can be obtained by computation from a stack of images. For example, the everywhere sharp image can be obtained by the method 400 of FIGURE 4.

[0034] The sharp image everywhere can be obtained by calculation from a single image of the scene.

[0035] In this case, and without loss of generality, step 108 of obtaining the new image can be carried out in accordance with the example described with reference to FIGURE 3.

[0036] FIGURE 7 is a schematic representation of another non-limiting exemplary embodiment of a method according to the present invention.

[0037] The method 700 of FIGURE 7 can be used to produce a personalized display of a scene from at least one image of said scene. [0038] The method 700 of FIGURE 7 includes all of the steps of the method 600 of FIGURE 6.

[0039] The method 700 of FIGURE 7 further comprises the optional step 502 of obtaining a stack of images of the scene.

[0040] FIGURE 8 is a schematic representation of a non-limiting exemplary embodiment of a system according to the present invention.

[0041] The system 800 of FIGURE 8 may be used to implement the method according to the invention, and in particular any one of the methods 100, 500, 600, or 700 described above.

[0042] The system 800 includes a display medium 802 used by a first user 804, called a speaker. The first display medium 802 can be of any type, such as an electric display screen or a display surface associated with a projector. In the example described, and without any generality, it is considered that the first display medium 802 is an electronic display screen.

[0043] The first display support 802 can be associated with at least one sensor 806 to detect:

- the position of a pointer manipulated by speaker 804,

- the contact position of a part of the speaker's body 804 such as a hand or finger, or

- the position targeted by a part of the body of the speaker 804 such as a hand or a finger, an eye, eyes or the face of the speaker; to provide, or with a view to determining, a target position on a current image displayed on the first display medium 802. In the following and without loss of generality, the at least one sensor 806 is a capacitive surface integrated into the display screen 802.

[0044] The system 800 further comprises a second display medium 810 used by a second user 812, called a participant. In the example shown, only one second display medium 810 and only one participant 812 are shown. Of course, the system 800 may comprise one or more second display mediums, each associated to one or more participants. The, or each, second display medium 810 may be of any type, such as an electric display screen or a display surface associated with a projector. In the example described, and without any generality, it is considered that the, or each, second display medium 810 is an electronic display screen.

[0045] In the example shown, the system 800 comprises an apparatus 820 connected to the display media 802 and 810 and making it possible to generate a personalized display of a scene on the display media 810, and optionally on the display media 802.

[0046] In particular, the apparatus 800 comprises a module 822 carrying out a display of an image of the scene given to it as input. This module 822 is in particular configured/programmed to carry out the display steps 104 and 110 and 112 of the display phase 102.

[0047] The apparatus 820 further comprises a module 824 for detecting a target position on a current image displayed on the first support 802. This module 822 takes as input a position of a pointer or of a part of the user's body, and determines the target position on the current image displayed on the first display support 802. This module 824 is in particular configured/programmed to carry out step 106.

[0048] The apparatus 820 further comprises a module 826 for obtaining the new image to be displayed, for example according to any of the techniques above. This module 826 is in particular configured/programmed to carry out step 108.

[0049] At least one of these modules may be a module independent of the others.

[0050] At least two of these modules can be integrated within the same module.

[0051] At least one of these modules may be a hardware module.

[0052] At least one of these modules may be a software module, such as a computer program.

[0053] At least one of these modules may be a combination of at least one software module, such as a computer program, and at least one hardware module. [0054] In particular, at least one of the modules 822-826 can be integrated into an electronic chip, or even into a software application.

[0055] The device 820 can be any type of device such as a smartphone, a tablet, a computer, a server, a processing unit, a vehicle, etc.

[0056] The apparatus 820 may be located on the same site as the first display medium 802, or on the same site as the second display medium 810, or on a site remote from the display mediums 802 and 810.

[0057] The first display medium 802 and the second display medium may be located on the same site or on remote sites.

[0058] In the example shown, the system 800 comprises a single device 820 common to the display supports 802 and 810. Of course, the invention is not limited to this exemplary embodiment.

[0059] FIGURE 9 is a schematic representation of another non-limiting exemplary embodiment of a system according to the present invention.

[0060] The system 900 of FIGURE 9 can be used to implement the method according to the invention, and in particular any one of the methods 100, 500, 600, or 700 described above.

[0061] The system 900 comprises a first apparatus 902 associated with the first display medium 802. In particular, the first apparatus 902 comprises the first display medium 902. This first apparatus 902 comprises the modules 822-826 described with reference to the system 800 of FIGURE 8.

[0062] The system 900 comprises a second apparatus 904 associated with the second display medium 810. In particular, the second apparatus 904 comprises the second display medium 810. This second apparatus 904 comprises a display module 906, for example similar or identical to the display module 822, for displaying an image of the scene on the display medium 810. [0063] In the system 900, the module 822 located in the first apparatus 902 performs a display of a current image of the scene on the first display medium 802. The target position POS is determined by the module 824 of said first apparatus 902 and the new image is obtained by the module 826 of said first apparatus 902. The new image is transmitted by the first apparatus 902 to the second apparatus 904, wired or wirelessly. The new image received by the second apparatus 904 is displayed by the display module 906 on the second display medium 810.

[0064] In the system 900, the first and second devices 902 and 904 are in communication, directly or via a communication network.

[0065] In system 900, the new image is obtained in the first device and communicated to the second device.

[0066] According to an alternative not shown, the module 824 for obtaining the new image can be integrated into the second device 904. In this case, the first device 902 transmits the target position POS to the second device and the new image is obtained in the second device.

[0067] According to yet another alternative not shown, the first apparatus 902 can comprise the module 824 for obtaining the new image, and a module for obtaining a new image, similar or identical to said module 824, can be integrated in the second apparatus 904. In this case, the first apparatus 902 transmits to the second apparatus the target position POS and the new image is obtained both in the first apparatus 902 and in the second apparatus 904.

[0068] The first device may be any type of device, such as a smartphone, a tablet, a computer, a television, a video headset, a virtual reality headset, an augmented reality headset, etc.

[0069] The first device may be a medical imaging device.

[0070] The first device may be integrated into a vehicle, such as a land vehicle, for example of the car type, or a flying vehicle, for example of the drone, airplane type, etc. [0071] The second device can be any type of device, such as a smartphone, a tablet, a computer, a television, a video headset, a virtual reality headset, an augmented reality headset, etc.

[0072] The second device may be a medical imaging device.

[0073] The second device can be integrated into a vehicle, such as a land vehicle, for example of the car type, or a flying vehicle, for example of the drone, airplane type, etc.

[0074] According to non-limiting exemplary embodiments, the system according to the invention may be a conference system, or a teleconference system, or a videoconference system, comprising several display media located on the same site or on different sites. Each user may be a speaker or a participant.

[0075] According to other exemplary embodiments, the system according to the invention can be integrated into a communication system between a person carrying out an observation, monitoring, medical diagnosis, etc., and one or more other persons - the participants - carrying out operations or interventions linked to these observations (in the field of personal safety in urban areas, the signaling of a danger on a road network to a team that must neutralize it, or to one or more road users who must avoid the danger, or a medical worker carrying out a remotely guided intervention).

[0076] Of course, the invention is not limited to the examples which have just been described.

Claims

1. Method (100; 500; 600; 700) of representing a scene from at least one image of said scene, said method (100; 500; 600; 700) comprising at least one iteration of a display phase (102) comprising the following steps: - display (104), on a first display medium (802), of an image of the scene (IC), called the current image; - detection (106), on said current image (IC), of a position, called target position in which a first user (804), called speaker, is interested; - obtaining (108) an image (IN), called a new image, clearly displaying said scene at said target position; and - displaying (110) said new image on at least one second display medium (810) observed by at least one second user (812), called participant. 2. Method (100; 500; 600; 700) according to the preceding claim, characterized in that it comprises a step (112) of displaying the new image (IN) on the first display medium (802). 3. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the step (108) of obtaining the new image (IN) is carried out on the speaker side (804), said new image being transmitted to the participant (812). 4. Method (100; 500; 600; 700) according to any one of claims 1 or 2, characterized in that the step (108) of obtaining the new image (IN) is carried out on the side of the participant (812), said method (100; 500; 600; 700) comprising a transmission of the target position to the participant (812). 5. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the step (108) of obtaining the new image comprises an on-the-fly generation of the new image, by calculation. 6. Method (100;500;600;700) according to the preceding claim, characterized in that the source image is an image (INP), previously obtained, and representing the scene clearly in all areas. 7. Method (100; 500; 600; 700) according to any one of claims 1 to 3, characterized in that the step (108) of obtaining the new image (IN) comprises a step (206) of selecting the new image (IN) in a stack of images (PIL) of said scene, each having a different sharpness zone. 8. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the display phase (102) further comprises, after identification of the target position, a step of identifying an object located at the target position, the obtaining of the new image being carried out as a function of said object. 9. Method according to any one of the preceding claims, characterized in that the display phase (102) further comprises a step of identifying a zone, called target, of the scene comprising the target position, the obtaining of the new image being carried out as a function of said target zone. 10. Method according to any one of the preceding claims, characterized in that the display phase (102) comprises a step of determining depth data of the scene at said target position, the new image being obtained as a function of said depth data. 11. Method (100;500;600;700) according to any one of the preceding claims, characterized in that the target position is entered manually by the speaker (804). 12. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the step (106) of detecting the target position comprises a detection, by at least one sensor, of a position targeted by a part of the body of the speaker (804). 13. Method (100; 500; 600; 700) according to any one of the preceding claims, characterized in that the display phase (102) is repeated for several images, or series of images, each captured at a different time, for example forming part of a video. 14. Computer program comprising executable instructions which, when executed by a computing device, implement all the steps of the method (100; 500; 600; 700) according to any one of the preceding claims. 15. System (800;900) comprising: - a first display support (802), - at least one second display medium (810), and - at least one calculation unit; configured to implement the method (100; 500; 600; 700) according to any one of claims 1 to 13.