[go: up one dir, main page]

WO2011135390A1 - Procédé et appareil de réduction de la répétition des trames en imagerie 3d stéréoscopique - Google Patents

Procédé et appareil de réduction de la répétition des trames en imagerie 3d stéréoscopique Download PDF

Info

Publication number
WO2011135390A1
WO2011135390A1 PCT/IB2010/000953 IB2010000953W WO2011135390A1 WO 2011135390 A1 WO2011135390 A1 WO 2011135390A1 IB 2010000953 W IB2010000953 W IB 2010000953W WO 2011135390 A1 WO2011135390 A1 WO 2011135390A1
Authority
WO
WIPO (PCT)
Prior art keywords
motion blur
frame
deriving
total
eye
Prior art date
Application number
PCT/IB2010/000953
Other languages
English (en)
Inventor
Emil Tchoukaleysky
Original Assignee
Thomson Licensing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing filed Critical Thomson Licensing
Priority to PCT/IB2010/000953 priority Critical patent/WO2011135390A1/fr
Priority to US13/642,658 priority patent/US20130038693A1/en
Publication of WO2011135390A1 publication Critical patent/WO2011135390A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/349—Multi-view displays for displaying three or more geometrical viewpoints without viewer tracking
    • H04N13/354—Multi-view displays for displaying three or more geometrical viewpoints without viewer tracking for displaying sequentially

Definitions

  • the present invention relates to 3-dimensional imaging. More specifically, it relates to a method and apparatus for reducing frame repetition in stereoscopic 3D (S3D) imaging.
  • S3D stereoscopic 3D
  • the present invention addresses, first and foremost, the problem of convolution between the multiplexed eye-image sequence, and the frame sequence reproducing dynam ic objects. Secondly, it explores the possibilities for optimizing the popular method of multiplexing the LE-RE images for one frame in a sequence:
  • the method for reducing frame repetition in stereoscopic 3D imaging includes deriving a left eye (LE) motion blur for an input frame, deriving a right eye (RE) motion blur for the same input frame, deriving a coincidental motion blur (Cmb) for the input frame, and adding the coincidental motion blur (Cmb) to both LE and RE images.
  • LE left eye
  • RE right eye
  • Cmb coincidental motion blur
  • the apparatus for reducing frame repetition in stereoscopic 3D imaging includes at least one motion blur extraction circuit configured to derive motion blur for an input video frame for each of a left eye (LE) image and a right eye (RE), at least one total motion blur extraction circuit configured to derive a total motion blur for each of the LE image and RE image, a circuit for deriving a coincidental motion blur using the total motion blur extracted for each of the LE image and the RE image, and at least one adder circuit configured to add the input video frame with the coincidental motion blur and a processed version of the total motion blur for each the LE and RE image.
  • at least one motion blur extraction circuit configured to derive motion blur for an input video frame for each of a left eye (LE) image and a right eye (RE)
  • at least one total motion blur extraction circuit configured to derive a total motion blur for each of the LE image and RE image
  • a circuit for deriving a coincidental motion blur using the total motion blur extracted for each of the LE image and the RE image and at least one add
  • Figures l a-lc show a graphical representation of the frame sequences for left eye
  • Figure 2 is a visualization of the coincidental motion blur (Cmb) for LE and RE images according to an implementation of the present invention
  • Figure 3 is a graphical representation of the coincidental blur for the covered eye with a dark image frame present, according to an implementation of the present invention
  • Figures 4a and 4b are graphical representations highlighting the non-linear relation between motion blur and object speed
  • Figure 5 is a flow diagram of the method for reducing frame repetition according to an implementation of the present invention.
  • Figure 6 is a block diagram of an apparatus within which the method of the present invention is implemented.
  • the present invention is directed towards enhancing the reproduction of three- dimensional dynamic scenes on digital light processing (DLP) and (liquid crystal display) LCD projectors and displays by adding optimal amount of motion blur to stimulate the covered eye to continue perceiving scene picture changes. Too much blur would bring smearing, but a lack of blur induces motion breaking.
  • DLP digital light processing
  • LCD liquid crystal display
  • processor or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage.
  • DSP digital signal processor
  • ROM read-only memory
  • RAM random access memory
  • any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
  • any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function.
  • the present principles as defined by such claims reside in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
  • the method of the present invention introduces a small increase in a type of motion blur, referred to herein as "coincidental blur", which is specific for stereography, and relies on some particularities of the Human Visual System in perceiving this blur.
  • the present invention proposes to increase the amount of the coincidental blur which reaches directly the active eye for a given frame, and to utilize the fact that it is perceived by the other eye indirectly, through brain processing, as a reduced amount.
  • the Method and Apparatus for Reducing the Frame Repetition in Stereoscopic 3D Imaging analyzes in detail the motion scene through the frame sequence, and extracts the scene-object displacement data.
  • the coincidental blur information is valid for both eyes.
  • the increased amount of coincidental blur will contribute for the covered eye to better handle the missing pictorial data for this frame, via image processing in the brain.
  • the enhancement process can be performed efficiently when an experienced operator selects the amount of coincidental blur and establishes its optimal amount in several viewing iterations.
  • This correction can be implemented by electronic video-mixing equipment at the post-production facility.
  • the number of the reproduced frames per second is brought down to a standard amount.
  • the method and apparatus of the present invention are designed to improve the perceivable quality of S3D images that represent volumetric dynamic/motion scenes on digital cinema screens.
  • S3D images that represent volumetric dynamic/motion scenes on digital cinema screens.
  • Those of skill in the art will recognize that there are two (2) categories of methods for quality enhancement of dynamic images in stereoscopic digital cinema theatres, to which the present invention could be compared:
  • FPS Frame Per Second
  • the FPS increase for S3D usually is three times per eye-image, compared to the standard 24 FPS rate. Sometimes it is called triple flash, or triple flashing.
  • Frame repetition has been employed for a long time in non-stereoscopic cinema theatre, at the standard 48 FPS for double projection of every frame.
  • the introduction of stereoscopic imagery brought about the need to triple the frame repetition to 72 FPS, or 3x24 FPS per eye- image, in order to avoid the motion breaking, or judder.
  • the total FPS per both eyes is 144 FPS.
  • the motion blur is not applied selectively. Rather, the enhancement for one of the eye- images highlights the general blur in the displayed frame. Since the fundamental problem to be solved ensues from the insufficient picture elements for the covered eye, this category doesn't offer adaptive improvement in the desired direction; and
  • the employed motion blur is not categorized for the particularities of the sequential LB- RE stereoscopic projection.
  • the present invention employs methods in line to solve the same problems as identified in category 2, while overcoming the disadvantages of the same.
  • a main goal and advantage of the present invention is to use intra-frame and inter-frame motion blur to achieve smooth perception of dynamic S3D images, while using the classic cinematic dual frame flashing, rather than the triple frame flashing currently utilized in stereoscopy.
  • the method counts on the natural leaking of one eye-image to both optical receiving hemispheres of the human brain, and proposes to utilize this phenomenon by modifying the projected pictures during the phase of image processing in the video domain.
  • the present invention widens the applicabil ity of the method by including video monitors, displays, and TV sets, in the list of possible S3D reproducing devices.
  • the present invention proposes to perform the following:
  • the Method and Apparatus for Reducing the Frame Repetition in Stereoscopic 3D Imaging of the present invention analyzes, in detai l, the motion scene presented by the frame sequence, and extracts the directional scene object data, which also constitutes the object displacement data.
  • the coincidental blur information is valid for both eyes. The increased amount of coincidental blur will help the covered eye processing part of the brain to better handle the "dark." frame for this eye.
  • the object displacement could be the result of scene motion, of camera zoom and pan, and also of animation in synthesized scenes. These methods require frame buffers to store the LE and RE neighboring frames for comparison. Those of skill in the art will appreciate that the following image processing concepts are the building blocks of the proposed invention. Deriving Coincidental Motion Blur
  • the LE motion blur is a collection of object trails in the left-eye frame sequence, which appears only in the left camera during capture; its sources are the object edges invisible to the right camera.
  • the LEmb is an image, whose pixels are situated around the pixels of solid objects and mainly in a direction opposite to the direction of the object movement. The LEmb pixels are not found in the RE image;
  • the RE motion blur is a collection of object trails in the right-eye frame sequence, which appears only in the right camera during capture; its sources are the object edges invisible to the left camera.
  • the REmb is an image, whose pixels are situated around the pixels of solid objects and mainly in a direction opposite to the direction of object movement. The REmb pixels are not found in the LE image;
  • the total LE motion blur (TLEmb) image is a sum of the object trails in the LE frame sequences, visible by both cameras during capture, plus the LEmb motion blur specific for the left eye;
  • the total RE motion blur (TREmb) image is a sum of the object trails in the RE frame sequences, visible by both cameras during capture, plus the REmb motion blur specific for the right eye.
  • the Coincidental motion blur (Cmb) created by the present invention is used to convey extra image information to the eye system which does not receive light during the current video frame.
  • the coincidental motion blur (Cmb) is derived from a pair of LE and RE images, and then a small amount of it is added to any of the total blur to form a corrected value:
  • a frame buffer of one frame is required to conduct the addition, which shapes the final result of the image process algorithm.
  • the projected frame is sti ll one per eye.
  • the relation between the inter-frame difference and the amount of the coincidental motion blur Cmb is not necessarily straightforward. Rather, the relation is non-linear and reflects the perceiving characteristics of the HVS.
  • the method and process to implement this non-linear relation is an important aspect of the present invention. Human eyes take an amount of motion blur proportionally to the object speed, under a logarithmic law. This is valid for S3D imagery as well.
  • FIG. 3 there is shown a graphical representation of the position of the coincidental motion blur (Cmb) during a dark frame.
  • Cmb coincidental motion blur
  • inter-frame difference there are a number of known methods for motion analysis, which could be used to define the inter-frame difference, and therefore the total blur. Those of skill in the art will recognize that the principles of the present invention are not restricted to a specific motion analysis method. As the simplest computation of the inter-frame difference, the pixel by pixel inter-frame comparison could be used.
  • Figures 4a and 4b show a graphical representation the non-linear relation between the object speed and the amount of introduced motion blur.
  • Figure 4a presents the linear correspondence between the scene object speed and the inter- frame object displacement in the image sequence.
  • Figure 4b shows the non-linear relation between the object speed and the amount of introduced motion blur, as discussed above in this section.
  • the curve is of logarithmic nature, the way the perceiving characteristics of the HVS are.
  • Figure 5 shows the flow-diagram of the method 50 for reducing frame rate repetition according to an implementation of the present invention.
  • the input video is accepted (52).
  • the LE and RE motion blur are derived (54) and the total LE motion blur and total RE motion blur are derived (56).
  • the coincidental motion blur (Cmb) is derived (58).
  • the Cmb derived at step 58 is added to both the LE and RE images of the input video.
  • This addition at step 60 operates to add the derived Cmb to the input video.
  • a non-linear motion blur is applied (62) which, as described above, is a function (F) of the object speed.
  • a determination is made as to whether or not this is the last frame (64), and if yes, the process ends (66). If this is not the last frame (64), then the process begins again at step 52 for the next frame.
  • Figure 6 shows a block diagram of an apparatus 70 according to an implementation of the present invention.
  • the input video for the LE (72) and RE (82) is applied to modules 74 and 84, respectively, for extracting the specific motion (i.e., motion blur) for each eye-image.
  • the resulting extracted motion blur is, together with the input video, is passed to a circuit for extracting the total motion blur for each eye (76, 86). This amount is weighted using a reducer (78, 88) and applied to the corresponding adder (79, 89).
  • outputs of the Total motion extraction modules 76, 86 is input into the logical AND circuit 80 to generate the coincidental motion blur (Cmb).
  • the Cmb is also input to each adder 79, 89, which adds the input video, weighted outputs of reducers 78, 88 and the determined Cmb to provide the resulting left eye (LE) output and right eye (RE) output.
  • the adders 79, 89 function to apply the non linear motion blur (step 60 in Figure5) to provide the respective output.
  • the CPU 90 is in signal communication with all modules shown and controls the image processing throughout the system.
  • Figure 6 is only one example of an implementation of an apparatus according to the present invention. This figure shows separate circuits for left eye (LE) and right eye (RE) image processing. In other words, L and RE image processing.
  • the apparatus may include motion blur and total motion blur extraction circuits that are integrated into the same circuit and remain capable of processing the LE and RE images independent of each other.
  • the present invention would be implemented as a combination of hardware and software.
  • the software is preferably implemented as an application program tangibly embodied on a program storage device.
  • the application program can be uploaded to, and executed by, a machine comprising any suitable architecture.
  • the machine would be implemented on a computer platform having hardware such as one or more central processing units (CPU), a random access memory (RAM), and input/output ( 1 10) interface(s).
  • the computer platform also includes an operating system and microinstruction code.
  • the various processes and functions described herein may either be part of the microinstruction code, or part of the application program (or a combination thereof), which is executed via the operating system.
  • various other peripheral devices may be connected to the computer platform, such as an additional data storage device, and a printing device.
  • teachings of the present principles are implemented as a combination of hardware and software.
  • the software can be implemented as an application program tangibly embodied on a program storage unit.
  • the application program can be uploaded to, and executed by, a machine comprising any suitable architecture.
  • the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPU"), a random access memory (“RAM”), and input/output (“I/O") interfaces.
  • the computer platform can also include an operating system and microinstruction code.
  • the various processes and functions described herein may be either part of the m icroinstruction code or part of the appl ication program, or any combination thereof.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

La présente invention a pour objet l'amélioration de la reproduction de scènes dynamiques tridimensionnelles sur des projecteurs et afficheurs de traitement de lumière numérique (DLP) et à LCD (affichage à cristaux liquides) en ajoutant un niveau optimal de flou de mouvement afin de stimuler l'œil couvert pour qu'il continue de percevoir les changements d'images de la scène. Un flou excessif provoquerait un effet de rémanence, mais un manque de flou entraînerait une discontinuité du mouvement.
PCT/IB2010/000953 2010-04-27 2010-04-27 Procédé et appareil de réduction de la répétition des trames en imagerie 3d stéréoscopique WO2011135390A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/IB2010/000953 WO2011135390A1 (fr) 2010-04-27 2010-04-27 Procédé et appareil de réduction de la répétition des trames en imagerie 3d stéréoscopique
US13/642,658 US20130038693A1 (en) 2010-04-27 2010-04-27 Method and apparatus for reducing frame repetition in stereoscopic 3d imaging

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/IB2010/000953 WO2011135390A1 (fr) 2010-04-27 2010-04-27 Procédé et appareil de réduction de la répétition des trames en imagerie 3d stéréoscopique

Publications (1)

Publication Number Publication Date
WO2011135390A1 true WO2011135390A1 (fr) 2011-11-03

Family

ID=43085671

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2010/000953 WO2011135390A1 (fr) 2010-04-27 2010-04-27 Procédé et appareil de réduction de la répétition des trames en imagerie 3d stéréoscopique

Country Status (2)

Country Link
US (1) US20130038693A1 (fr)
WO (1) WO2011135390A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9274403B2 (en) * 2013-03-13 2016-03-01 Robert C. Weisgerber Method for selectively imparting a cinematic appearance to motion pictures photographed and exhibited at high frame rates

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5805205A (en) * 1994-03-28 1998-09-08 Magma, Inc. Simulataneous two-dimensional and three-dimensional imaging system using a bifurcated, dual-aperture light valve
WO2006110646A2 (fr) * 2005-04-08 2006-10-19 Real D Ecran autostereoscopique dote d'un systeme de passage plan
US20090009590A1 (en) * 2007-07-06 2009-01-08 Texas Instruments Incorporated Method for reducing stereoscopic phase-lag distortion under motion in a 3-dimensional video display
US20100060720A1 (en) * 2008-09-09 2010-03-11 Yasutaka Hirasawa Apparatus, method, and computer program for analyzing image data

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1186014A (ja) * 1997-09-08 1999-03-30 Fujitsu Ltd 文書画像表示方法および表示装置
JP4729812B2 (ja) * 2001-06-27 2011-07-20 ソニー株式会社 画像処理装置および方法、記録媒体、並びにプログラム
US7162101B2 (en) * 2001-11-15 2007-01-09 Canon Kabushiki Kaisha Image processing apparatus and method
EP1875732B1 (fr) * 2005-04-26 2016-12-28 Imax Corporation Systemes de projection electronique et procedes associes
KR100860967B1 (ko) * 2007-04-16 2008-09-30 삼성전자주식회사 영상의 모션 블러를 제거하기 위한 장치 및 방법
JP2012523788A (ja) * 2009-04-13 2012-10-04 ショースキャン デジタル エルエルシー 動画の撮影及び投影方法及びその装置
US20110080466A1 (en) * 2009-10-07 2011-04-07 Spatial View Inc. Automated processing of aligned and non-aligned images for creating two-view and multi-view stereoscopic 3d images

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5805205A (en) * 1994-03-28 1998-09-08 Magma, Inc. Simulataneous two-dimensional and three-dimensional imaging system using a bifurcated, dual-aperture light valve
WO2006110646A2 (fr) * 2005-04-08 2006-10-19 Real D Ecran autostereoscopique dote d'un systeme de passage plan
US20090009590A1 (en) * 2007-07-06 2009-01-08 Texas Instruments Incorporated Method for reducing stereoscopic phase-lag distortion under motion in a 3-dimensional video display
US20100060720A1 (en) * 2008-09-09 2010-03-11 Yasutaka Hirasawa Apparatus, method, and computer program for analyzing image data

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
RAV-ACHA A ET AL: "Two motion-blurred images are better than one", PATTERN RECOGNITION LETTERS, ELSEVIER, AMSTERDAM, NL, vol. 26, no. 3, 1 February 2005 (2005-02-01), pages 311 - 317, XP025292487, ISSN: 0167-8655, [retrieved on 20050201], DOI: DOI:10.1016/J.PATREC.2004.10.017 *

Also Published As

Publication number Publication date
US20130038693A1 (en) 2013-02-14

Similar Documents

Publication Publication Date Title
US9961273B2 (en) Mobile terminal and shooting method thereof
US8633968B2 (en) Three-dimensional recording and display system using near- and distal-focused images
JP2014502818A (ja) 画像処理のための一次像および二次像画像取込装置およびその方法
EP2362668A2 (fr) Procédé et appareil pour le traitement d'images vidéo stéréoscopiques
EP3281400A1 (fr) Génération automatisée de prises de vue panoramiques
EP2569950B1 (fr) Traitement de bruit de confort et de grain de film pour une vidéo en 3 dimensions
JP2012044407A (ja) 画像処理装置および方法、並びにプログラム
US20150304640A1 (en) Managing 3D Edge Effects On Autostereoscopic Displays
JP2007110360A (ja) 立体画像処理装置およびプログラム
EP2517472B1 (fr) Procédé et appareil pour une reproduction optimale de mouvement dans un cinéma numérique stéréoscopique
EP3238212A1 (fr) Appareil et procédé pour générer une image extrapolée à l'aide d'un processus hiérarchique récursif
JP4989788B2 (ja) 立体画像表示制御装置およびその動作制御方法
US20130038693A1 (en) Method and apparatus for reducing frame repetition in stereoscopic 3d imaging
Curti et al. 3D effect generation from monocular view
EP2822279A1 (fr) Représentation de tapisserie autostéréoscopique
CN110809147A (zh) 图像处理方法及装置、计算机存储介质、电子设备
US20240155095A1 (en) Systems and methods for processing volumetric images
CN119011998A (zh) 一种虚拟拍摄方法、视频处理方法及装置
CN116866540A (zh) 图像渲染方法、系统、装置及存储介质
CN119071463A (zh) 一种虚拟拍摄方法及装置
CN119342336A (zh) 图像处理方法、图像处理装置和电子设备
CN119788925A (zh) 一种vr场景中的单目投屏方法及虚拟现实设备
CA2982015A1 (fr) Methode et appareil d'imagerie amelioree en profondeur
Mangiat High Dynamic Range and 3D Video Communications for Handheld Devices
JP2017126878A (ja) 映像切替装置及びそのプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10725269

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 13642658

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10725269

Country of ref document: EP

Kind code of ref document: A1