CN101189643A - 3D image generation and display system - Google Patents

Abstract

A 3D image generation and display system for easily displaying high-quality images on a web browser, comprising: for creating 3D images from a plurality of different images and computer graphics modeling and generating 3D from these images with texture and attribute data An object; an apparatus for converting and outputting a 3D object as a 3D description file using a 3D graphics description language; an apparatus for extracting a 3D object and a texture from a 3D description file, setting various attribute data, and editing and processing a 3D object to Means for introducing animation, etc., and assigning various effects; for generating various web-based 3D objects from the 3D data files generated above that are compressed so as to be displayed on a web browser, and for generating behavior data for browsing on the web through animation and means for generating executable files including web pages and web-based programs such as scripts, plug-ins, and applets for drawing and displaying 3D scenes on the web browser.

Description

3D image generation and display system

technical field

The present invention relates to a 3D image generation and display system that generates 3D objects for displaying various photographic images and computer graphics models in three dimensions (3D), and for editing and processing for rendering and displaying on web browsers A 3D object for a 3D scene.

Background technique

There are various known systems in the field for creating 3D objects for use in 3D displays. One such technique using a 3D scanner for modeling and displaying 3D objects is known in the art as light sectioning (implemented by projecting a slit of light) and the like. This method uses a CCD camera to capture points or lines of light projected onto an object by a laser beam or other light source and uses the principle of triangulation to measure the distance from the camera to perform 3D modeling.

FIG. 13( a ) is a schematic diagram showing a conventional 3D modeling apparatus using a light section method.

A CCD camera captures an image when a slit of light is projected from a light source onto an object. An image such as that shown in Fig. 13(b) is obtained by scanning the entire object to be measured while gradually changing the direction in which the light source projects the light slit. According to the triangulation method, 3D shape data is measured from the known positions of the light source and the camera. However, since the entire periphery of an object cannot be three-dimensionally rendered using the light section method, it is necessary to provide multiple cameras to collect images around the entire periphery of the object as shown in FIG. 14 so that the object is imaged without hidden areas.

Furthermore, 3D objects created by these methods must thereafter undergo various effect applications and animation processing for displaying 3D images according to intended uses and various data processing required to three-dimensionally display the objects on a web browser. For example, the image needs to be optimized by reducing the file size etc. to fit the quality of the communication line.

One type of 3D image display is a liquid crystal panel or display used in a game console or the like to display a 3D image in which objects appear to jump off the screen. This technique utilizes special glasses such as polarizers with different polarization directions on the left and right lenses. In such a 3D image display device, left and right images are captured from the same position when viewed with left and right eyes, and polarization is used so that only the left image is viewed with the left eye and the right image is viewed with only the right eye. Other examples include devices using mirrors or prisms. However, these 3D image displays have a complication that requires a viewer to wear glasses or the like. Accordingly, 3D image display systems using cylindrical mirrors, parallax barriers, or other devices that allow 3D images to be viewed without glasses have been developed and commercialized. One such device is "3Diamge signal generator" disclosed in Patent Document 1 (Japanese Unexamined Patent Application Publication No. H10-271533). This device improves on the 3D image display disclosed in US Patent 5,410,345 (April 25, 1995) by displaying a 3D image on an ordinary LCD system used to display two-dimensional images.

FIG. 15 is a schematic diagram showing the generator of the 3D image signal. The 3D image signal generator includes: a backlight 1, including a light source 12 arranged on the side by a side lighting method; a cylindrical mirror 15, which can move in the front and rear directions; a diffuser 5, which is used to slightly diffuse incident light; and LCD 6 for displaying images. As shown in a stereoscopic display image 20 in FIG. 16, the LCD 6 has a structure well known in the art in which pixels P displaying each of the colors R, G, and B are arranged in a stripe pattern. A single pixel Pk (k=0-n) is composed of three sub-pixels of RGB arranged horizontally. The color of the pixel is displayed by mixing the three primary colors displayed by each sub-pixel in additive color processing.

When a 3D image is displayed through the backlight 1 shown in FIG. In order to describe this phenomenon based on the stereoscopic display image 20 of FIG. , 5... of sub-pixels. Therefore, in order to display a 3D image, the 3D image signal generator generates 3D image signals from the image signals of the left image and the right image captured at the positions of the left and right eyes, and supplies these signals to the LCD 6.

As shown in FIG. 16 , a stereoscopic display image 20 is generated by interleaving RGB signals from a left image 21 and a right image 22 . In this way, the 3D image signal generator composes the rgb component of the pixel P0 in the 3D image signal according to the r and b components of the pixel P0 in the left image signal and the g component of the pixel P0 in the right image signal, and The g component of P1 and the r and b components of pixel P1 in the right image signal constitute the rgb component (middle column) of pixel P1 in the 3D image signal. Through this interleaving process, the kth pixel in the 3D image signal is usually composed of the r and b components of the kth pixel in the left image signal and the g component of the kth pixel in the right image signal (k is 1, 2, . ..) rgb component, and the k+1th pixel in the 3D image signal composed of the g component of the k+1th pixel in the left image signal and the r and b components of the k+1th pixel in the right image signal rgb component.

The 3D image signal generated in this method can display a 3D image compressed to the same number of pixels as in the original image. Since the left eye can only see the sub-pixels displayed in the LCD 6 in the even columns, and the right eye can only see the sub-pixels displayed in the odd columns, as shown in FIG. 18 , a 3D image can be displayed. In addition, by adjusting the position of the cylindrical mirror 15, the display can be switched between 3D and 2D display.

Although the above example in FIG. 15 has the cylindrical mirror 15 arranged on the rear surface of the LCD 6, the "stereoscopic image display device" disclosed in Patent Document 2 (Japanese Unexamined Patent Application Publication No. H11-72745) gives An example of a cylindrical mirror arranged on the front surface of the LCD is shown. As shown in FIG. 19, the stereoscopic image display device has a parallax barrier (also a cylindrical lens) 26 disposed on the front surface of the LCD 25. In this device, a pixel group 27R is formed by a pixel pair of right-eye pixels (Rr, Gr, and Br) driven by an image signal for the right eye and left-eye pixels (RL, GL, and BL) driven by an image signal for the left eye. , 27G and 27B. Two disparity signals are generated by configuring both left and right cameras to photograph objects at left and right observation points corresponding to the observer's left and right eyes. The examples in Figures 20(a) and 20(b) show R and L signals generated for the same color. As shown in Figure 20(c), the means for compressing and combining these signals is used to reconfigure these R and L signals in an alternating pattern (R, L, R, L...) to form a single stereoscopic image. Since the synthesized left and right signals must be compressed by half, the actual signal for forming a single stereoscopic image is composed of different color image data pairs for left and right eyes as shown in FIG. 20(d). In this example, the display is switched between 2D and 3D by switching the position of the slits in the parallax barrier.

Patent Document 1: Japanese Unexamined Patent Application Publication No. H10-271533

Patent Document 2: Japanese Unexamined Patent Application Publication No. H11-72745

Contents of the invention

However, the 3D scanning method shown in FIGS. 13 and 14 uses a large amount of data and must perform many calculations, requiring a long time to generate a 3D object. In addition, the device is complex and expensive. The device also requires particularly expensive software for applying various effects and animations to 3D objects.

Therefore, an object of the present invention is to provide a 3D image generation and display system in which, instead of a method of collecting photo data by a plurality of cameras arranged around the periphery of an object, a 3D scanner employing a scanning stage method for rotating an object is used, Accurate 3D objects are thereby generated from multiple different images in a short time and with a simple structure. In order to quickly draw and display a 3D scene on a web browser, the 3D generation and display system uses commercial software to edit and process the main part of the 3D object to generate a web-specific 3D object.

In the stereoscopic image devices shown in FIGS. 15 to 20, the formats of the left and right parallax signals are different when the formats of the display devices are different, for example, when the same liquid crystal panel is used by moving the cylindrical mirror as shown in FIG. A system for switching between 2D and 3D displays and a system for fixing the parallax barrier shown in FIG. 19 . In the same way, for all display devices having different formats (for example, various display panels, CRT screens, 3D, and projectors), the formats of the left and right disparity signals are different.

The formats of the left and right parallax signals are also different when different image signal formats are used, such as the VGA method or the method of interlaced video signals.

Furthermore, in the conventional art as shown in FIGS. 15 to 20 , left and right disparity signals are generated by two captured images captured by two digital cameras positioned correspondingly to left and right eyes. However, when the formats of raw image data are different, for example, when left and right disparity signals are created directly using left and right disparity data generated by photographing a subject and characteristic images generated by computer graphics modeling or the like, left and right The format and method of disparity data vary.

Therefore, another object of the present invention is to provide a 3D image generation and display system for creating a 3D image that generalizes the left and right disparity signal format, wherein it is possible to create a signal that can assimilate various input images and these input images Format differences, as well as common platform differences for various display devices, and for displaying these 3D images on web browsers.

To achieve these objects, a 3D image generation and display system according to claim 1 constitutes a computer system for generating 3D objects for displaying three-dimensional (3D) images on a web browser, the 3D image generation and display system comprising: 3D Object generating means for creating 3D images from a plurality of different images and/or computer graphics models and for generating 3D objects from these images with texture and attribute data; 3D description file output means for converting generated 3D objects The format of the 3D object generated by the device, and the data output as a 3D description file for displaying 3D images according to the 3D graphics description language; the 3D object processing device is used to extract the 3D object from the 3D description file, set various attribute data, Editing and processing of 3D objects to bring in animations, etc., and output of the resulting data again as a 3D description file or as a temporary file for setting properties; texture processing means for extracting textures from 3D description files, editing and processing textures to reduce color number, etc., and output the resulting data again as a 3D description file or as a texture file; a 3D effect application device for extracting a 3D object from a 3D description file, processing the 3D object and assigning various effects such as lighting and material properties, and outputting The resulting data again as a 3D description file or as a temporary file for assigning effects; network 3D object generation means for extracting from 3D description files, texture files, temporary files for setting properties and temporary files for assigning effects Various elements required for rendering 3D images in a web browser, and for generating various web-based 3D objects with texture and attribute data compressed to be displayed on a web browser; behavioral data generating means, For generating behavior data by controlling properties and assigning effects of 3D objects to display 3D scenes on a web browser through animation; and executable file generating means for web 3D generated, edited and processed based on the behavior data and by the above means The object generates an executable file, wherein the executable file includes a web page and includes scripts, plug-ins, and applets for rendering and displaying a 3D scene on a web browser using a stereoscopic image generated from a plurality of composite images distributed by a specified disparity one or more programs.

In addition, the 3D object generating apparatus according to claim 2 includes: a turntable on which the object is mounted and rotates the object horizontally or vertically; a digital camera for capturing an image of the object mounted on the turntable and creating a digital image of the image document; a turntable control device for rotating a turntable to a specified position; a photographing device for photographing an object set in a specified position by the turntable control device using a digital camera; a continuous image creation device for continuously creating using the turntable control device and the photographing device a plurality of image files; and 3D object synthesizing means for generating a 3D image based on the plurality of image files created by the sequential image creating means, and generating a 3D object having texture and attribute data from the 3D image for displaying the image in 3D.

Furthermore, the 3D object generation apparatus according to claim 3 generates a 3D image according to a silhouette method in which, since the object is rotated on a turntable, multiple images from a single camera shot around the entire periphery of the object are used. The contour data of the image estimates the three-dimensional shape of the object.

Furthermore, the 3D object generating apparatus according to claim 4 generates a single 3D image as a synthetic scene obtained by synthesizing various image data including an image captured by a camera, an image produced by computer graphics modeling, an image produced by Images scanned by a scanner, hand-drawn images, image data stored in other storage media, etc.

Furthermore, the executable file generating means according to claim 5 includes: automatic left and right disparity data generating means for automatically generating images for drawing and displaying left and right disparity data of a stereoscopic image; disparity data compressing means for compressing each of the left and right disparity data generated by the automatic left and right disparity data generating means; disparity data synthesizing means for synthesizing the compressed left and right disparity data; disparity data expansion means for dividing the synthesized left and right disparity data into left and right parts, and expanding the data to be displayed on a stereoscopic image display device; and display data conversion means for displaying the stereoscopic image display device according to The viewing angle (aspect ratio) that transforms the data that will be displayed.

Further, the automatic left and right disparity generating means according to claim 6 automatically generates left and right disparity data corresponding to the 3D image generated by the 3D object generating means based on the virtual camera set by the rendering function.

Furthermore, the parallax data compressing apparatus according to claim 7 compresses pixel data of the left and right parallax data by skipping pixels.

Furthermore, the stereoscopic display device according to claim 8 uses at least one of a CRT screen, a liquid crystal panel, a plasma display, an EL display, and a projector.

Furthermore, the stereoscopic display device according to claim 9 displays a stereoscopic image that can be seen when the viewer wears stereoscopic glasses or displays a stereoscopic image that can be seen when the viewer does not wear the glasses.

The 3D image generation and display system of the present invention can configure a computer system that generates 3D objects to be displayed on a 3D display. The 3D image generation and display system has a simple structure in which an object placed on a scanning table is modeled using a scanning table system by collecting images around the entire periphery of the object with a single camera while the turntable rotates. Furthermore, the 3D image generation and display system facilitates the generation of 3D objects by utilizing commercially sold general software.

The 3D image generation and display system can also be displayed on a web browser by installing a special plug-in program for drawing and displaying a 3D scene on a web browser or by generating a small program for effectively displaying a 3D image on a web browser animation.

The 3D image generation and display system can also compose a display program capable of displaying stereoscopic images based on LR parallax image data, such 3D images that do not "jump out" at the viewer, and 2D images on the same display device.

Description of drawings

In the attached picture:

1 is a flow chart showing steps in processing performed by a 3D image generation and display system according to a first embodiment of the present invention;

FIG. 2 is a schematic diagram illustrating a 3D object generating device of the 3D image generating and display system described in FIG. 1;

3 is a flow chart showing the process from generation of a 3D object to rendering and display of a 3D scene in a web browser;

4 is a perspective view of a printer as an example of a 3D object;

5 is a schematic diagram illustrating a 3D object generation and display system according to a second embodiment of the present invention;

Fig. 6 is a schematic diagram showing the 3D image generator with 2 to n cameras of Fig. 5;

FIG. 7 is an exemplary diagram illustrating a method of setting a camera position in the renderer of FIG. 5;

FIG. 8 is an example diagram showing a process for creating a simple stereoscopic image;

Fig. 9 is the block diagram of the LR data processing circuit in the VGA display;

FIG. 10 is an exemplary diagram illustrating operations for zooming in and out, moving and rotating a 3D image;

11 is a block diagram showing an LR data processing circuit of a video signal type display;

12 is a schematic diagram showing a stereoscopic display system using a projector;

Fig. 13(a) is a schematic diagram of a traditional 3D modeling display device;

FIG. 13( b ) is an example diagram illustrating creation of a slit image;

FIG. 14 is a block diagram illustrating a conventional 3D modeling device illustrating multiple cameras;

Fig. 15 is a schematic diagram of a conventional 3D image signal generator;

FIG. 16 is an example diagram showing LR data of the signal generator of FIG. 15;

FIG. 17 is an exemplary diagram showing a process for compressing the LR data in FIG. 16;

FIG. 18 is an exemplary diagram illustrating a method of displaying LR data on the display device of FIG. 15;

19 is a schematic diagram of another conventional stereoscopic image display device; and

FIG. 20 is a diagram illustrating an example of LR data displayed on the display device of FIG. 19 .

specific embodiment

Next, preferred embodiments of the present invention will be described while referring to the accompanying drawings.

FIG. 1 is a flowchart showing steps in processing performed by the 3D image generation and display system according to the first embodiment of the present invention.

In the process of FIG. 1 described below, a 3D scanner described later is used to form a plurality of 3D images. A 3D object is generated from a 3D image and converted into a standard virtual reality modeling language (VRML: language for describing 3D graphics) format. The converted 3D object in the output VRML file is subjected to various processes for generating a web 3D object and a program file executable on a web browser.

First, for example, a 3D scanner of a 3D object generation device using a digital camera captures an image of a real object, obtaining 24 3D images taken at varying angles of 15 degrees (S101). The 3D object generation means generates a 3D object from these images, and the 3D description file output means temporarily converts the 3D object into a VRML format (S102). 3DSCanWare (product name) or similar programs can be used to create 3D images, generate 3D objects, and generate VRML files.

A 3D object generated with 3D design software (such as software mentioned below) is extracted from the VRML file and subjected to various editing and processing by the 3D object processing means (S103). Commercial product "3ds max" (product name) or other software is used to analyze necessary areas of 3D objects to extract texture images, set properties required for animation processing and generation of various 3D objects, and set various animation characteristics as necessary . After being edited and processed, the 3D object is saved again as a 3D description file in VRML format, or temporarily stored in a storage device or a memory area as a temporary file for setting properties. In the animation setting, the number of frames or time can be set in key frames for moving objects provided in the 3D scene at intervals of a certain number of frames. You can also use this technique as path animation and character studio for creating paths, such as Nurbs CV curves along which objects move. Using a texture processing device, the user extracts texture images applied to various objects in a VRML file, edits texture images for color, texture mapping, etc., reduces the number of colors, modifies the area and location/position where textures are applied, or performs other processing, and Save the resulting data as a texture file (S104). Texture editing and processing can be performed using commercial image editing software such as Photoshop (product name).

The 3D effect application device is used to extract various 3D objects from VRML files and use the extracted objects in combination with 3ds max or similar software and various plug-ins to process 3D objects and apply various effects such as lighting and material properties. The resulting data is re-stored as a 3D description file in VRML format or saved as a temporary file for applying effects (S105). In the above description, the 3D object has undergone processing for displaying as animation on a web page and processing for reducing file size as preprocessing in texture image processing and the like. The following steps cover the processing used to reduce and optimize the object size and file size to actually display the object on a web browser.

The network 3D object generation means extracts 3D objects, texture images, attributes, animation data, and other rendering elements from the VRML and temporary files created during editing and processing, and generates network 3D objects for displaying 3D images on the network (S106 ). Meanwhile, the behavior data generating means generates behavior data as a scenario for displaying the network 3D object as animation (S107). Finally, based on the above data for displaying 3D images, the executable file generation means generates plug-in software for web browsers or executable files in combination with Java applets, Jave scripts, etc. to draw and display 3D images on the web browser. An image is displayed (S108).

By using the VRML format supported by most 3D software programs, 3D images can be edited and manipulated using common commercial software programs. The system can also optimize the image for use on the network based on the transmission rate of the communication line, or when displaying the image on the web browser of the local computer, it can edit and process the image appropriately according to the display environment, thereby controlling image rendering in the display environment Effective and achieve optimal quality.

FIG. 2 is a schematic diagram illustrating a 3D object generating apparatus of the 3D image generating and display system described above with reference to FIG. 1 .

The network 3D object generating device in Fig. 2 comprises: a turntable 31, which supports an object 33 (corresponding to "object" in the claims section, and referred to as "object" or "real object" in this specification), and rotates 360 A background panel 32 of a single primary color, such as green or blue; a digital camera 34, such as a CCD; an illumination device 35; a table rotation controller 36, which rotates the turntable 31 through servo control; a photographing device 37 for controlling and calibrating digital cameras 34 and lighting equipment 35, performing gamma correction and other image processing of image data and capturing images of subject 33; and continuous image creation means 38 for controlling angles of rotation, and sampling and Collect images. These components make up a 3D modeling setup that employs a scanning stage and a single camera used to generate a series of images viewed from multiple angles. At this point, using commercial editing software such as AutoCAD and STL (product name), modify the image as necessary. The 3D object synthesizing means 39 extracts contours from a series of images, and estimates a 3D shape using a silhouette method or the like to create a 3D image, thereby generating 3D object data.

Next, the operation of the 3D image generation and display system will be described.

In the silhouette method, the camera is calibrated by computing, for example, the relationship between the world coordinate system, the camera coordinate system, and the image coordinate system. To process images in software, convert points in the image coordinate system to points in the world coordinate system.

After calibration is complete, the sequential image generation device 38 cooperates with the stage rotation controller 36 to control the turntable for a given number of scans (e.g., 36 scans of images every 10 degrees and 72 scans of images every 5 degrees) At the same time, the camera 37 captures an image of the object 33 . The contour data of the object 33 is obtained from the captured image by obtaining a background difference, where the background difference is the difference between a previously captured image of the background panel 32 and the current camera image. A silhouette image of the object is obtained from the background difference and the camera parameters obtained by calibration. Then, 3D modeling is performed on the contour image, for example, by placing a cube with a recursive octree structure in the three-dimensional space and determining intersections in the contours of the objects.

Fig. 3 is a flowchart giving a more explicit/concrete example according to the steps in the process shown in Fig. 1 for converting a 3D image, so that the steps shown in Fig. 1 can be better/further explained. The processing in Figure 3 is implemented by a Java applet that can display 3D images in a web browser without installing a plug-in program such as Live 3D for the viewer. In this example, all data required for displaying the interactive 3D scene is provided on a web server. When the server is accessed from a web browser running on a client computer, the 3D scene is displayed. Usually, after the 3D object is created, 3ds max etc. are used to modify motion, camera, lighting and material properties etc. in the generated 3D object. However, in this preferred embodiment, the 3D object or the entire scene is first converted into VRML format (S202).

Import the resulting VRML file into the 3DA system (S203: Here, 3DA describes a 3D image that is displayed as an animation on a web browser using a Java applet, and includes a whole system of design software for web-related editing and processing known as the 3DA system). Customize the 3D scene and provide data for rendering an image with a 3DA applet for drawing and displaying the 3D scene on the web browser (S205). All 3D scene data are simultaneously compressed and saved as a compressed 3DA file (S206). The 3DA system generates a toolbar file and an HTML file for interactive operation, wherein the HTML page reads the toolbar file into the web browser so that the toolbar file is executed and displays the 3D scene in the web browser (S207).

The new web page (HTML document) includes an applet tag for calling the 3DA applet. Java script code for accessing the 3DA applet may be added to the HTML document to improve operation and interactivity (S209). All files required for displaying the 3D scene created as described above are transferred to the server. These files include web pages (HTML documents) that control the applet markup for invoking the 3DA applet, toolbar files for interactive manipulation as options, texture image files, 3DA scene files, and 3DA applets for drawing and displaying 3D scenes. program (S210).

When the web browser then connects to the web server and requests the 3DA applet, the web browser downloads the 3DA applet from the web server and executes the applet (S211). Once the 3DA applet is executed, the applet displays a 3D scene in which the user can perform interactive operations, and the web browser can continuously display the 3D scene independent of the web server (S212).

In the process described up to this point, a 3DA Java applet file is generated after converting the 3D object into a web-based VRML file, and the web browser downloads the 3DA file and the 3DA applet. However, in addition to generating 3DA files, it is of course possible to install a plug-in program such as Live3D (product name) for the viewer and directly process the VRML3D description file. With the 3D image generation and display system of the preferred embodiment, a company can easily create a website using the display of products in three dimensions and mobile e-commerce and the like.

As an example of an e-commerce product, the following description covers a commercial website for launching printers, such as that shown in FIG. 4 .

First, the company's product (the printer 60 as the object 33) is placed on the turntable 31 shown in FIG. 2 and rotated while the photographing device 37 captures an image at a specified sampling angle. The continuous image creation means 38 sets the number of images sampled so that the photographing means 37 captures 36 images, where a sampling angle of 10 degrees is assumed (360 degrees/10 degrees = 36). The 3D object synthesizing means 39 calculates the background difference between the camera position and the previously photographed background panel 32, and converts each of the 36 images of the printer created by the continuous image creating means 38 through coordinate transformation in world coordinates, camera coordinates, and image positions. An image data converted into world coordinates. The silhouette method for extracting the outline of the object was used to model the shape of the printer and generate a 3D object of the printer. Temporarily output the object as a VRML file. At this time, all 3D images to be displayed on the network are created, including the operation screen at the rear, left and right side views, top and bottom views, operation screen at the front, and the like.

Next, as described in FIG. 1, the 3D object processing means, the texture processing means, and the 3D effect application means extract the generated 3D image data from the VRML file, analyze relevant parts of the data, generate 3D objects, apply various attributes, Perform animation, and apply various effects and other processing, such as lighting and surface formation, through color, material, and texture mapping properties. Save the resulting data as a texture file, a temporary file for attributes, and a temporary file for effects. Next, the behavior data generating means generates data necessary for motion in all 3D description files used on the printer's website. Specifically, the action data generation means generates a file for activating an actual operation screen in a setting wizard or the like.

By installing a plug-in program such as Live 3D for the viewer in the web browser, the 3D scene data created above can be displayed in the web browser. It is also possible to use a method for processing 3D scene data only in a web browser without using a viewer. In this case, as described above, the 3D file of the Java applet is downloaded to the web browser for drawing and displaying the 3D scene data extracted from the VRML file.

While viewing the above-created website displaying the 3D image of the printer, the user can operate the mouse to click an item displayed in the setting guide menu of the browser, thereby displaying a 3D animation sequence. The animation may show a sequence of operations of rotating the button 63 on the cover 62 of the printer 60 to detach the cover 62 and install the USB connector 66 .

When the user clicks on "Install Toner Cartridge" in the menu, a 3D animation sequence (not shown) will be played in which the entire printer is rotated to show its front surface. The top cover 61 of the printer 60 is opened, and one cartridge holder inside the printer 60 is moved to the middle position. The black and color toner cartridges are inserted into the cartridge holder, and then the top cover 61 is closed.

Also, if the user clicks on "Maintenance Screen", a 3D image (not shown) is displayed with all the plastic casings removed to reveal the inner mechanism of the printer. In this way, the user can clearly see the spatial relationship among the drive module, scanning mechanism, ink cartridge, etc. in three dimensions, which facilitates maintenance operations.

By displaying the operation window with 3D animation in this way, the user can check the product with the same realistic feeling as when actually operating the printer at a retail store.

The above description is an example for viewing the operation of a printer, and the 3D image generation and display system can also be used for other applications such as trying on clothes. For example, a 3D generation and display system may allow a user to try on an outfit from a women's clothing store or the like. Users can click on an outfit worn by a model; change the size and color of the garment; view the modeled garment from the front, back, and side; modify the button shape, size, and color; and even order the garment by email. It is also possible to display various commodities such as sculptures for auction or other fine arts and daily necessities in a three-dimensional image that is more realistic than a two-dimensional image.

Next, a second embodiment of the present invention will be described with reference to the drawings.

FIG. 5 is a schematic diagram showing a 3D image generation and display system according to a second embodiment of the present invention. The second embodiment further expands the 3D image generation and display system to allow the 3D image generated in the first embodiment and displayed on a web page to be displayed as a stereoscopic image using other 3D display devices.

The 3D image generation and display system in FIG. 5 includes a turntable type 3D object generator 71 which is the same as the 3D object generation device of the first embodiment shown in FIG. 2 . The 3D object generator 71 generates a 3D image by synthesizing images of the object captured by a single camera while the object is rotated on the turntable. The 3D image generation and display system of the second embodiment further includes a multi-camera 3D object generator 72 . Different from the turntable type 3D object generator 71, the 3D object generator 72 arranges from 2 stereo cameras corresponding to the positions of the left eye and the right eye to n cameras (not specifically limited to any number, used A large number of cameras enables more detailed images) Multiple cameras generate 3D objects. The 3D image generation and display system also includes a computer graphics modeling 3D object generator 73 for generating a 3D object while executing a computer graphics model through a graphical interface of a program (such as 3ds max). 3D Object Generator 73 is a computer graphics modeler that can combine scenes with computer graphics, photographs or other data.

After performing the processes S103 to S107 described in FIG. 1 of the first embodiment to temporarily save the 3D objects generated by the 3D object generators 71 to 73 as general-purpose VRML files, use a web design tool such as YAPPA 3D studio (product name)) to extract 3D scene data from a VRML file. Design software for editing and manipulating 3D objects and textures; adding animation; applying, setting, and manipulating other effects, such as camera and lighting effects; and generating web 3D objects and their behavioral data for drawing and display in web browsers Interactive 3D graphics. An example for creating a network 3D file is described in S202 to S210 of FIG. 3 .

Means 75 to 79 are executable file generating means used in the application in S108 of FIG. 1 for displaying left and right disparity data of a stereoscopic image. The renderer 75 applies a rendering function to generate left and right parallax images (LR data) necessary for displaying stereoscopic images. The LR data compression/synthesis means 76 compresses the LR data generated by the renderer 75, reconfigures the data in a composition process and stores the data in a display frame buffer. When displaying LR data, the LR data separating/expanding means 77 separates and expands left and right data. A data conversion device 78 composed of a down converter or the like adjusts the viewing angle (aspect ratio, etc.) for displaying a stereoscopic image so that the LR data is compatible with various 3D display devices. The stereoscopic display device 79 displays stereoscopic images based on the LR data and using various display devices such as a liquid crystal panel, a CRT screen, a plasma display, an EL (Electro Luminescence) display, or projector shutter type display glasses, and includes various display formats , for example, the general VGA format used in personal computer monitors and the like and the video format used for televisions.

Next, the operation of the 3D image generation and display system according to the second embodiment will be described.

First, 3D object generation processing performed by the 3D object generators 71 to 73 will be briefly described. The 3D object generator 71 is the same as the 3D object generating device described in FIG. 1 . A 3D object 33 forming a 3D image is placed on the turntable 31 . The stage rotation controller 36 adjusts the rotation of the turntable 31 while controlling the digital camera 34 and the lighting device 35 to take a sample photo by the photographing device 37 against a monochrome screen such as a blue screen as a background (background panel 32 ). Then, the sequential image creating means 38 performs processing of synthesizing the sampled images. Based on the resulting synthesized images, the 3D object synthesizing means 39 extracts the outline (shape) of the object, and estimates the three-dimensional shape of the object using a silhouette method or the like to generate a 3D object. For example, this method is performed using the following equation.

Equation 1

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="S"\; filenames="S2005800498393E00161.png"\; state="\{\{state\}\}">S</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& <span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&CenterDot;</span>& {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="S"\; filenames="S2005800498393E00161.png,S2005800498393E00171.png"\; state="\{\{state\}\}">S</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}\mathrm{<span\; class="notranslate">\; no</span>}}\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& <span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&Center\; Dot;</span>& {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="P"\; filenames="S2005800498393E00161.png"\; state="\{\{state\}\}">P</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; no</span>}}\\ <span\; class="notranslate">\; \&CenterDot;</span>& <span\; class="notranslate">\; \&CenterDot;</span>& & <span\; class="notranslate">\; \&CenterDot;</span>\\ <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span>& <span\; class="notranslate">\; \&CenterDot;</span>\\ <span\; class="notranslate">\; \&CenterDot;</span>& <span\; class="notranslate">\; \&CenterDot;</span>& & <span\; class="notranslate">\; \&Center\; Dot;</span>\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="P\; m"\; filenames="S2005800498393E00161.png"\; state="\{\{state\}\}">P</figure-callout></span><figure-callout\; id="2"\; label="P\; m"\; filenames="S2005800498393E00161.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; </span>}}& <span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&CenterDot;</span>& {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; mn</span>}}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Coordinate conversion (calibration) is performed using the camera coordinate Pfp and the world coordinate Sp of the point P to convert the three-dimensional coordinates at the vertices of the 3D image into the world coordinate system [x, y, z, r, g, b]. Various modeling programs are used to model the resulting coordinates. 3D data generated by this process is saved in an image database (not shown).

The 3D object generator 72 is a system that captures an image of an object by placing multiple cameras around the object. For example, as shown in FIG. 6, 6 cameras (first to sixth cameras) are set around the object. The control computer obtains photo data from the camera via the USB hub, and reproduces the 3D image of the object on the first and second projectors in real time. 3D object generator 72 is not limited to 6 cameras, but may capture images with any number of cameras. This system generates a 3D image in a world coordinate system from multiple overlapping photos obtained by these cameras, and falls under the category of image-based rendering (IBR). Therefore, the structure and processing of this system are quite complicated compared to the 3D object generator 71 . By the 3D object generator 71, the generated data is saved in the database.

The 3D object generator 73 mainly focuses on computer graphics modeling using modeling software such as 3ds max and YAPPA 3D studio, wherein YAPPA 3D studio assigns "upper" to each of the four views in the divided viewport. ”, “Left”, “Right”, “Front”, “Perspective”, and “Camera”, to establish a grid corresponding to the vertices of the graphics on the display screen and use various objects, shapes and Other data are modeled images. These modeling programs may combine computer graphics data with photographic or image data created with 3D object generators 71 and 72 . This combination is easily achieved by adjusting the viewing angle of the camera, the aspect ratio of the rendered image in the bitmap of the photo data and computer graphics data.

A camera (virtual camera) can be created at any position for setting or modifying the viewpoint of the composited scene. For example, to change the default camera position (the user's point of view) from the front to a position offset 30 degrees to the left or right, set the coordinates of the camera angle and position by using [X,Y,Z,W]. The composite image is displayed at a position offset 30 degrees from the scene. In addition, virtual cameras that can be created include a free camera that can be freely rotated and moved to an arbitrary position, and a target camera that can rotate around an object. When the user wants to change the viewpoint of the synthetic image scene or the like, the user can do so by setting a new property. With the lens function, etc., users can quickly change viewpoints by selecting from groups of about 10 virtual lenses ranging from WIDE to TELE or switching touch buttons. Lighting settings can also be changed in the same way through various functions that can be applied to rendered images. All data generated are saved in the database.

Next, processing for generating left and right parallax images by the rendering and LR data (parallax image) generating means 75 will be described. The LR data corresponding to the disparity signals of the left and right eyes can be easily acquired using the camera position setting function of the above-described modeling software program. Next, a specific example for calculating the camera positions of the left eye and the right eye in this case will be described with reference to FIG. 7 . As shown in Figure 7(a), the coordinates of each camera position are represented by a vector perpendicular to the modeled object (a phone in this instance). Here, the coordinates of the camera position are set to O; the focusing direction of the camera is set to a vector OT; and the vector OU is set to an upward direction from the camera and is orthogonal to the vector OT. To achieve stereoscopic display with the positions of the left and right eyes, the positions of the left and right eyes (L, R) are calculated according to the following equation (2), where θ is the tilt of the left and right eyes (L, R) angle, and d is the distance from the convergent point P of zero disparity between the left and right eyes.

Equation 2

$<span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; OR</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; OL</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; the\; tan</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}$

$\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; OR</span>}}<span\; class="notranslate">\; =</span>\frac{\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; ou</span>}}<span\; class="notranslate">\; \&times;</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; OT</span>}}}{<span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; ou</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; OT</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; the\; tan</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}$

$\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; OL</span>}}<span\; class="notranslate">\; =</span>\frac{\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; \&times;</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; ou</span>}}}{<span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; OT</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; |</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; ou</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; the\; tan</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Here, (0<d, 0≤θ<180)

The method for calculating the above-mentioned position is not limited to this method, but may be any calculation method that achieves the same effect. For example, since the default camera position is set to front, it is obvious that the coordinates [X, Y, Z, w] can be directly input using the above-mentioned method of studying the camera (virtual camera) position.

After setting the eye position (camera position) found in the above method in the camera function, the user selects "renderer" etc. in the window toolbar displaying the scene to convert and render the 3D scene into a two-dimensional image, thereby obtaining a stereoscopic display Left and right disparity images.

LR data is not limited to the use of synthetic image scenes, but can also be created for photo images taken by the 3D object generators 71 and 72 . By setting the coordinates [X, Y, Z, w] of the camera position (virtual camera) corresponding to the positions of the left and right eyes, a photo image can be rendered, saving the image data of the subject captured around the entire periphery to obtain the left and right LR data for disparity images. LR data can also be created from image data captured around the periphery of the object held in the same manner as a 3D object obtained from a computer image image modeled by the 3D object generator 73 or the like. LR data can be easily created by rendering various synthetic scenes.

In the actual rendering process, the coordinates of each vertex of the polygon in the world coordinate system are converted into a 2D screen coordinate system. Therefore, 3D/2D conversion is performed by inverse conversion of Equation 1 for converting camera coordinates into 3-dimensional coordinates. In addition to calculating the camera position, shadowing (brightness) due to virtual light from light sources needs to be calculated. For example, the light source data Cnr, Cng, and Cnb describing the material colors Mr, Mg, and Mb can be calculated using the following conversion matrix Equation 3.

Equation 3

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; Cnr</span>}\\ \mathrm{<span\; class="notranslate">\; Cng</span>}\\ \mathrm{<span\; class="notranslate">\; CNb</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="Pnr"\; filenames="S2005800498393E00151.png,S2005800498393E00161.png"\; state="\{\{state\}\}">Pnr</figure-callout></span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="Png"\; filenames="S2005800498393E00151.png,S2005800498393E00161.png"\; state="\{\{state\}\}">Png</figure-callout></span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; Pnb</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; Mr.</span>}\\ \mathrm{<span\; class="notranslate">\; Mg</span>}\\ \mathrm{<span\; class="notranslate">\; MB</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Here, Cnr, Cng, Cnb, Pnr, Png, and Pnb represent the n-th vertex.

The LR signals of the left and right parallax images obtained by this rendering process are automatically generated by calculating the coordinates of the camera position and shadows based on the light source data. Various filtering processes are also performed simultaneously, but are omitted from this description. In a display device, an up/down converter or the like converts image data into bit data and adjusts an aspect ratio before displaying an image.

Next, a method of automatically generating simple LR data will be described as another example of the present invention. FIG. 8 is an example diagram showing a method of generating simple left and right parallax images. As shown in the example of FIG. 8, the LR data of the character "A" has been created for the left eye. If the object is left-right symmetrical, the disparity image for the right eye is created as a mirror image of the LR data for the left eye by simply inverting the disparity data for the left eye. This inversion can be calculated using Equation 4 below.

Equation 4

$<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}{\mathrm{<span\; class="notranslate">\; Y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; X\; Y</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&times;</span>\left|\begin{array}{cc}\mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="Rx"\; filenames="S2005800498393E00151.png,S2005800498393E00161.png"\; state="\{\{state\}\}">Rx</figure-callout></span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; Ry</span>}\end{array}\right|<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Here, X represents the X coordinate, Y represents the Y coordinate, and X' and Y' represent new coordinates in the mirror image. Rx and Ry are equal to -1. This simple process works well when there are small changes in the image data and can greatly reduce storage consumption and processing time.

Next, an example of displaying actual 3D images on various display devices using the LR data found in the above processing will be described.

For simplicity, this description will cover the case where LR data is input into the conventional display device shown in FIG. 19 to display a 3D image. The display device shown in FIG. 19 is a liquid crystal panel (LCD) used in a personal computer or the like, and employs a VGA display system using a continuous display technique. FIG. 9 is a block diagram showing a parallax image signal processing circuit. When the LR data automatically generated according to the present invention is supplied to this type of display device, the LR data of the left and right parallax images shown in FIGS. 20(a) and 20(b) are input to the compressor/compositor 80. As shown in FIG. 20(c), the compressor/compositor 80 reconfigures the image data with alternating R and L data, and compresses the image by half by skipping pixels as shown in FIG. 20(d). The resulting LR composite signal is input to a splitter 81 . The separator 81 performs the same processing in reverse, and reconfigures the image data by separating the R and L lines as shown in FIG. 20(c). This data is decompressed and expanded by expanders 82 and 83, and supplied to the display driver to adjust the aspect ratio and the like. The driver displays the L signal that can only be seen by the left eye and the R signal that can only be seen by the right eye, realizing stereoscopic display. Since pixels skipped during compression are lost and cannot be reproduced, image data is adjusted using interpolation or the like. This data can be used on displays in notebook personal computers, liquid crystal panels, direct-view game consoles, and the like. There is no particular restriction on the signal format of the LR data in these cases.

A web 3D design tool such as YAPPA 3D Studio is configured to convert image data into LR data according to the Java applet process. Operation buttons such as those shown in FIG. 10 can be implemented by attaching toolbar files to a Java applet and by downloading data from a web server (3d scene data, Java applets, and HTML files) to a web browser via a network. displayed on the screen of the web browser. By selecting the buttons, the user can manipulate the stereoscopic image (car in this case) displayed on the web browser to zoom in and out, move or rotate the image, and the like. Processing details for operations such as enlargement and reduction, movement and rotation, etc. are represented in a transformation matrix. For example, the movement can be represented by Equation 5 below. Other operations may be represented similarly.

Equation 5

$<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}{\mathrm{<span\; class="notranslate">\; Y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="XY"\; filenames="S2005800498393E00161.png,S2005800498393E00171.png"\; state="\{\{state\}\}">X\; Y</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; =</span>\left|\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; Dx</span>}& \mathrm{<span\; class="notranslate">\; Dy</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right|<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

Here, X' and Y' are new coordinates, X and Y are original coordinates, and Dx and Dy are distances moved in the horizontal and vertical directions, respectively.

Next, an example of displaying images on an interlace-type display such as a television screen will be described. Various converters are marketed as display devices in personal computers and the like for converting image data into general TV and video images. This example uses this converter to display stereoscopic images on a web browser. The structure and operation of the converter itself will not be described again.

The following examples use a liquid crystal panel (or a CRT screen, etc.) shown in Fig. 19 for reproducing video signals. A parallax barrier, a lenticular sheet, and the like for displaying stereoscopic images are installed on the front surface of the display device. Display processing will be described using the block diagram in FIG. 11 showing a signal processing circuit of a parallax image. The LR data of the left and right disparity images (eg, those shown in FIGS. 20( a ) and 20 ( b ) generated according to the automatic generation method of the present invention) are input to the compressors 90 and 91 , respectively. Compressors 90 and 91 compress images by skipping every other pixel in the video signal. As shown in Figures 20(c) and 20(d), the combiner 92 combines and compresses the left and right LR data. A video signal composed of this synthesized LR data is transmitted to a receiver or recorded and reproduced on a recording medium (eg, DVD). As shown in Figures 20(c) and 20(d), splitter 93 performs the same operation in reverse, splitting the composite LR data into left and right signals. The expanders 94 and 95 expand the left and right image data into the original form shown in Figs. 20(a) and 20(b). A stereoscopic image can be displayed on a display or the like as shown in FIG. 19 because display data is arranged in the order of R, G, and B in alternating left and right video data on horizontal scanning lines. For example, configure the R (red) signal as "R0 (left) R0 (right), R2 (left) R2 (right), R4 (left) R4 (right)...". Configure G (green) as "G0 (left) G0 (right), G2 (left) G2 (right), G4 (left) G4 (right)...". Configure B (blue) as "B0 (left) B0 (right), B2 (left) B2 (right)...". Furthermore, by dividing LR data of a parallax image signal into odd and even segments and processing these two data segments simultaneously, it is possible to realize stereoscopic display in the same manner using shutter glasses having liquid crystal shutters or the like as a display device.

Next, a description will be given of displaying stereoscopic images on a projector used for performances or as a home theater or the like.

12 is a block diagram of a home theater, which includes: a projector screen 101, the surface of which is optically treated (for example, silver metal coating is applied); two projectors 106 and 107 are arranged in front of the projector screen 101; Light sheets 108 and 109 are disposed in front of each projector 106 and 107, respectively. Each component of the home theater is controlled by the controller 103 . If the projector 106 is provided for the right eye and the projector 107 is for the left eye, the filter 109 is of the type for vertically polarized light, and the filter 108 is of the type for horizontally polarized light. The type of projector may be an MLP (Meridian Lossless Packaging) liquid crystal projector using a DMD (Digital Micromirror Device). The home theater also includes a 3D image recorder 104 that supports DVD or other media (of course, the device can also generate images through modeling), and based on the 3D image data input from the 3D image recorder 104, the display driver of the present invention can automatically generate LR The left and right disparity image generator 105 of the signal. The aspect ratio of the LR data generated by the left and right parallax image generator 105 is adjusted by a down converter or the like and supplied to the corresponding left and right projectors 106 and 107 . Projectors 106 and 107 project images through polarizing filters 108 and 109 that polarize the image horizontally and vertically, respectively. The viewer wears polarizing glasses 102 having a vertical polarizing filter for the right eye and a horizontal polarizing filter for the left eye. Therefore, when viewing an image projected on the projector screen 101, the viewer can see a stereoscopic image because the image projected by the projector 106 can only be seen with the right eye, and the image projected by the projector 107 can only be seen by the right eye. Can see with left eye.

industrial application

By using a web browser for displaying 3D images in this way, only an electronic device having a browser is required instead of a dedicated 3D image display device, and 3D images can be supported on various electronic devices. The present invention is also more user-friendly because there is no need to provide different stereoscopic display software (e.g., stereo drivers, etc.) ).

Claims (9)

1. a 3D rendering generates and display system, and by being used for generating the computer system configurations that is used for showing the 3D object of three-dimensional (3D) image on web browser, described 3D rendering generates and display system comprises:

3D object generating apparatus is used for creating 3D rendering by a plurality of different images and/or computer graphical modeling, and is used for generating the 3D object by these images with texture and attribute data;

3D description document output unit is used to change the form of the described 3D object that is generated by described 3D object generating apparatus, and according to the data of 3D picture description language output as the 3D description document that is used to show 3D rendering;

3D object handles device, be used for extracting the 3D object from described 3D description document, various attribute datas are set, edit and handle described 3D object introducing animation etc., and the output result data are once more as the 3D description document or as the temporary file that is used to set a property;

The texture processing device is used for extracting texture from described 3D description document, editor and handle texture reducing number of colours etc., and the output result data is once more as the 3D description document or as texture file;

The 3D effect application apparatus, be used for extracting the 3D object from described 3D description document, handle the various effects of described 3D object and distribution such as illumination and material behavior, and the output result data is once more as the 3D description document or as the temporary file that is used for distribution effects;

Network 3D object generating apparatus, be used for extracting and be used for playing up the required various key elements of 3D rendering, and be used to generate have and be compressed to be presented at the texture on the web browser and the various based on network 3D object of attribute data at web browser from described 3D description document, texture file, the temporary file that is used for setting a property and the temporary file that is used for distribution effects;

The behavioral data generating apparatus is used for generating behavioral data to show the 3D scene by animation on web browser by attribute and the distribution effects of controlling described 3D object; And

The executable file generating apparatus, be used for based on described behavioral data and by said apparatus generate, editor and the network 3D object handled generate executable file, wherein, described executable file comprises that webpage is used for utilizing one or more programs of being drawn and shown script, plug-in card program and the small routine of 3D scene by the stereo-picture of a plurality of composographs generations that distribute by the appointment parallax with comprising on web browser.

2. 3D rendering according to claim 1 generates and display system, and wherein, described 3D object generating apparatus comprises:

Turntable is equipped with object on it, and level or vertically rotate described object;

Digital camera is used to catch the image that is installed in the described object on the described turntable, and creates the digital image file of described image;

The turntable control device is used to rotate described turntable to assigned address;

Filming apparatus uses digital camera to take by described turntable control device and is arranged on object in the assigned address;

The consecutive image creation apparatus is used to use described turntable control device and described filming apparatus to create a plurality of image files continuously; And

3D is used for generating 3D rendering based on described a plurality of image files of being created by described consecutive image creation apparatus, and generates the 3D object with texture and attribute data by the 3D rendering that is used for the 3D display image picture synthesizer.

3. 3D rendering according to claim 2 generates and display system, wherein, described 3D object generating apparatus generates 3D rendering according to silhouette method, wherein, because described object is rotated on described turntable, so the outline data of a plurality of images that the single camera around the whole periphery of the next free object of use is taken is estimated the 3D shape of described object.

4. 3D rendering according to claim 1 generates and display system, wherein, described 3D object generating apparatus generates single 3D rendering as the synthetic scene that obtains by synthetic various view data, wherein, various view data comprise the image taken by camera, the image, image, the hand-drawing image by scanner scanning that are produced by the computer graphical modeling, are stored in view data in other storage mediums etc.

5. 3D rendering according to claim 1 generates and display system, and wherein, described executable file generating apparatus comprises:

Automatically a left side and right parallax data generating apparatus are used for based on the left-eye image and the eye image of distributing parallax from the specified camera position, generate left parallax data and the right parallax data that is used to draw and show stereo-picture automatically according to playing up function;

The parallax data compression set is used to compress by a described automatic left side and the described left parallax data of right parallax data generating apparatus generation and each of described right parallax data;

The parallax data synthesizer is used to synthesize the left parallax data and the right parallax data that are compressed;

The parallax data expanding unit, the left parallax data and the right parallax data that are used for being synthesized are divided into left half and right half, and growth data is to be presented on the stereoscopic display device; And

The video data conversion equipment, change the data that are shown at the visual angle (aspect ratio) that is used for according to described stereoscopic display device.

6. 3D rendering according to claim 5 generates and display system, wherein, a described automatic left side and right parallax data generating apparatus generate and the corresponding left parallax data of 3D rendering and the right parallax data that are generated by described 3D object generating apparatus automatically based on by the virtual camera of playing up function setting.

7. 3D rendering according to claim 5 generates and display system, and wherein, described parallax data compression set compresses the pixel data of left parallax data and right parallax data by skipping pixel.

8. 3D rendering according to claim 5 generates and display system, and wherein, described 3 d display device uses at least one in CRT screen, liquid crystal panel, plasma display, EL display and the projector.

9. 3D rendering according to claim 5 generates and display system, and wherein, described 3 d display device shows that when the beholder wears anaglyph spectacles appreciable stereo-picture or demonstration are when beholder's appreciable stereo-picture during wearing spectacles not.

Images