JP6653940B2 - Content generation device and content generation method - Google Patents

Description

  The present invention relates to a content generation device and a content generation method for generating content including a plurality of still images to be displayed on an external information terminal.

  A part of an image photographed at a wide angle of view is displayed on a screen of an information terminal such as a personal computer (hereinafter, sometimes referred to as “PC”) or a smartphone, and the image displayed on the screen is displayed on a terminal. By moving the terminal up and down or left and right according to the user's operation direction and operation amount, even a remote terminal user can freely change the viewpoint direction as if they were at the place. (For example, see Patent Document 1 below).

JP 2015-216543 A

2. Description of the Related Art In recent years, a service has been provided for guiding indoors and outdoors while switching and displaying a plurality of images shot at a plurality of shooting locations on a screen of an information terminal. However, when creating a content (a program that outputs at least one of character, voice, and image information) that guides a wide range, it is necessary to acquire images at many shooting points. In such a case, it is necessary to repeat the work of installing a camera and acquiring an image at each shooting location, and the more shooting locations, the more the cost and the number of steps required for shooting increase.
In view of the above circumstances, the present invention provides a content generation device and a content generation method capable of generating content including a plurality of still images to be displayed on an external information terminal in a short time at low cost. It was made for the purpose.

The present invention is an apparatus for generating content including a plurality of display images displayed on an external information terminal,
Image conversion means for converting a captured moving image into a number of still images corresponding to each frame ,
Image extraction means for extracting a specific still image from the many still images based on a predetermined condition as the display image,
Image connection means for connecting and connecting images corresponding to a specific frame as the display image extracted without using the imaging position information about the display image,
Without passing through a three-dimensional model generated based on an image corresponding to the specific frame, an image corresponding to the specific frame obtained from the moving image is displayed on the screen of the information terminal as the display image. The content to be switched and displayed is generated.

According to the content generation device of the present invention, a plurality of still images can be obtained from a shot moving image. A moving image for obtaining a still image can be created by moving a camera in a shooting state (a camera for shooting a moving image) along a point where an image is desired to be obtained, and the shooting operation is completed in one basic operation.
For this reason, in order to obtain a still image, the repetition of the previously required tasks, ie, fixing the camera, photographing (acquiring the still image), and moving to the next photographing point, is unnecessary or greatly reduced. Therefore, the cost and the construction period required for the photographing operation can be reduced.

  The captured moving image is composed of a series of frames, and a still image can be extracted from the moving image for each frame. Therefore, many still images can be obtained from one captured moving image. Further, according to the present invention, there is provided an image extracting means for extracting a specific still image as a display image from a large number of still images based on predetermined conditions. The extraction conditions for extracting the display image can be determined as appropriate. For example, the number of display images to be extracted from a moving image is designated as five, and five still images can be extracted at regular intervals every predetermined elapsed time from the start of shooting. In addition, it is also possible to specify the elapsed time from the time of shooting a moving image, and to extract still images individually at the 3rd, 15th, 28th,...

  The present invention includes an image connection unit, and connects the extracted display images with each other. Thereby, a group of contents including a plurality of display images displayed on the external information terminal is generated.

  As described above, according to the content generation device of the present invention, it is possible to automatically generate content including a plurality of still images to be displayed on an external information terminal based on a captured moving image and extraction conditions. Including the above-described reduction in the shooting period, the period required for generating the content can be significantly reduced.

Here, in the present invention, in the image connection means, when arranged in the shooting order, the display images adjacent to each other are connected so as to be switchable and displayable on the screen of the information terminal, and connected .
When selecting a forward icon superimposed on the display image on the screen of the information terminal, the content for switching and displaying the display image in the shooting path of the moving image can be generated . By doing so, the images displayed on the external information terminal are switched and displayed in the shooting order. For this reason, the content creator sets the shooting route in the order in which the user wants to view the video when shooting the video, and if the video is shot along the route, the terminal user who browses the content can order the content in the order of the shooting route. That is, a plurality of images can be browsed in the order intended by the creator.

In the present invention, the moving image may be a panoramic moving image, and the display image obtained from the panoramic moving image may be a panoramic still image. Here, the panoramic moving image and the panoramic still image refer to images having an angle of view of at least 180 degrees in any one of the vertical direction and the horizontal direction, and desirably an angle of view of approximately 360 degrees in the vertical and horizontal directions.
On the other hand, the moving image is a general moving image that does not have a wide angle of view as a panoramic moving image, and the display image obtained from the moving image is a general moving image that does not have a wide angle of view as a panoramic still image. A still image.

  The content generation device of the present invention and the content generated by the content generation device include an AR (Augmented Reality), a VR (Virtual Reality), an MR (Mixed Reality), and an SR (Substitutional Reality). Can be suitably used in the field of

  The present invention can also be configured as another content generation method.

  According to the present invention as described above, it is possible to provide a content generation device and a content generation method capable of generating content including a plurality of still images to be displayed on an external information terminal at low cost in a short time. .

FIG. 1 is a block diagram illustrating a configuration of a server as a content generation device according to an embodiment of the present invention, together with an external terminal and the like. FIG. 3 is an explanatory diagram illustrating an example of a moving image creation method according to the embodiment. It is a flowchart which shows the procedure of a process in a client terminal and a server. FIG. 4 is a diagram schematically illustrating generated contents. FIG. 4 is a diagram showing a screen of a user terminal when browsing content. FIG. 6 is a diagram showing a screen of the user terminal following FIG. 5.

Next, an embodiment of the present invention will be described in detail with reference to the drawings.
In FIG. 1, reference numeral 10 denotes a server as a content generation device in the present embodiment, 12 denotes a camera for capturing a panoramic video, and 14, 16, 18, and 20 denote information terminals. The server 10 and the information terminals 14, 16, 18, and 20 are connected via the Internet 15. Note that the connection between the server 10 and each of the information terminals 14, 16, 18, and 20 can use a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), or the like in addition to the Internet 15.

  In this example, using the panoramic video created by the camera 12, the server 10 generates content for a panoramic tour. Here, the content for the panoramic tour includes a plurality of panoramic still images that can be displayed on a screen of the information terminal 16 or the like connected via the Internet 15.

  The camera 12 includes two fisheye lenses 13a and 13b having an angle of view of 180 degrees or more in the vertical direction and the horizontal direction and two image sensors (not shown) corresponding to the fisheye lenses 13a and 13b. By combining the hemispherical images, a panoramic video having an angle of view of approximately 360 degrees in the vertical and horizontal directions is obtained. The moving image created by the camera 12 is output in a predetermined file format such as MPEG (Moving Picture Expert Group) and stored in the storage unit 12a inside the camera 12.

  The information terminal 14 is a client terminal that accesses the server 10 and requests the server 10 to generate panorama tour contents. Information necessary for content generation, such as a moving image file captured by the camera 12, is transmitted from the information terminal 14 to the server 10. The information terminal 14 is a portable terminal such as a PC or a smart phone, but is not limited thereto as long as it can be connected to the Internet 15 and has a Web browser (hereinafter simply referred to as a browser).

  The information terminals 16, 18, and 20 are user terminals for browsing panorama tour contents. It is a mobile terminal such as a PC or a smart phone, but is not limited thereto as long as it can be connected to the Internet 15 and has a browser.

The server 10 includes a communication unit 22, a control unit 24, and a storage unit 26.
The communication unit 22 transmits and receives data to and from the information terminal 14 via the Internet 15, transmits instruction information to the information terminal 14, and acquires a panoramic video or the like from the information terminal 14. Further, data is transmitted / received to / from the information terminals 16, 18, and 20 via the Internet 15, and contents of a panorama tour content described later are provided to the information terminals 16, 18, and 20.

  The control unit 24 controls each component of the server 10 and may be, for example, a processor such as a CPU. In this case, each component is controlled by executing a control program stored in the storage unit 26 by a processor such as a CPU. The control unit 24 in the server 10 includes an image conversion unit 30, an image extraction unit 32, and an image connection unit 34 described below.

  The image conversion unit 30 decomposes the panoramic video acquired through the communication unit 22 into frames and outputs the result as a panoramic still image. At this time, the panoramic still image is output in a predetermined file format such as JPEG (Joint Photographic Experts Group) or PNG (Portable Network Graphics). In the present embodiment, a panoramic still image having an angle of view of about 360 degrees is output from a panoramic video having an angle of view of about 360 degrees shot by the camera 12.

  The moving image captured by the camera 12 is composed of a series of frames, and the number of frames per unit time is usually represented by a frame rate fps (frames per second). For example, when a moving image is captured at a frame rate of 30 fps, 1800 panoramic still images are output from the first frame to the 1800th frame from a moving image of one minute (60 seconds), and these are stored in the storage unit 26. It is memorized. When the captured moving image has a frame rate of 100 fps, 6000 panoramic still images from the first frame to the 6000th frame are output from a moving image of one minute (60 seconds).

  The image extracting means 32 extracts a specific panoramic still image as a display image from a large number of panoramic still images stored in the storage unit 26 using the extracted information received from the information terminal 14. Then, the extracted display images (panoramic still images) are sequentially given file names in the order in which the images were captured.

  The image connecting means 34 connects the extracted display images with each other. When the display images extracted by the image extracting means 32 are arranged in the order of photographing, a plurality of display images displayed on the external information terminal 16 or the like are connected by associating and connecting adjacent display images. A set of panoramic tour contents including a display image is generated and stored in the storage unit 26.

Next, a procedure for creating content for a panoramic tour will be described. Here, as an example, it is assumed that content for introducing the inside of the shopping center 40 is generated.
A client who wants to generate content first creates a panoramic video using the camera 12. Here for taking the inside of the shopping center 40, as shown in FIG. 2, the client starts shooting from the point P _S of the inlet 41 near, substantially the central passage 42 while continuing maintained photographing state rightward in the figure moving at a constant speed, completing the photographing at a point P _E immediately beyond the outlet 43. At this time, if the time required from the start of shooting to the completion is 3 minutes, a panoramic video file of which shooting time is 3 minutes is stored in the storage unit 12a in the camera 12. Note that the moving image frame rate is 30 fps.

  Next, the client temporarily stores the moving image file created by the camera 12 in a storage unit (not shown) of the client terminal 14 and accesses the server 10 at the terminal 14. Then, as shown in FIG. 3, the server 10 transmits the information on the input screen to the terminal 14 (Step S101). Then, the client terminal 14 that has received this information displays an input screen (step S102).

  The input screen displayed on the terminal 14 has an input section for moving image file information and an input section for extraction conditions. The client inputs the moving image file information and the extraction condition into the input section of the input screen. Here, as the extraction condition, the number of panoramic still images to be extracted is defined as “7”, and seven panoramic still images obtained when the shooting time from the start to the end of the shooting is equally divided are extracted. And Then, when the client selects the transmission button displayed on the input screen, the moving image file specified by the input unit and the extraction condition are transmitted to the server 10 (step S103).

  Upon receiving the moving image file and the extraction condition (step S104), the server 10 converts the next received moving image (panoramic moving image) into a panoramic still image (step S105). Here, based on a panoramic video with a frame rate of 30 fps and a shooting time of 3 minutes, a total of 5400 frames from the panoramic still image of the first frame immediately after the start of the shooting to the 5400th frame of the panoramic still image at the time of the completion of the shooting Panoramic still images are output and stored in the storage unit 26, respectively.

In the next step S106, based on the extraction conditions received in step S104, a panoramic still image that satisfies the extraction conditions is extracted from the 5,400 panoramic still images. In this example, the first frame, the 900th frame, the 1800th frame, the 2700th frame, the 3600th frame, the 4500th frame, and the 5400th frame obtained when the shooting time from the start to the completion of the shooting is equally divided are obtained. A total of seven panoramic still images are extracted as display images. Here, the extracted seven display images are assumed to be S-1, S-2,..., S-7 in the shooting order (frame order). The shooting locations of these seven panoramic still images correspond to P ₁ , P ₂ ,..., P ₇ in FIG.

  In the next step S107, as shown in FIG. 4, the extracted seven display images S-1 to S-7 are arranged in the order of photographing, adjacent display images are connected with each other, and connected together. Of the panorama tour is generated. Here, adjacent display images are connected to each other in an external information terminal connected via the Internet 15 so as to be switchable and displayable. Then, the generated content 45 is stored in a predetermined area of the storage unit 26 (step 108), and information of a unique URL (Uniform Resource Locator) corresponding to the content 45 is generated and transmitted to the client terminal 14. (Step S109).

  On the other hand, the client terminal 14 receives the URL information and displays the URL information on the screen of the client terminal 14 (step S110). Then, if the client publishes the URL information on its own homepage or the like, a third party (user) other than the client can browse the contents of the content 45 with reference to the published URL information.

For example, when the user browses the content 45 on the terminal 16 with reference to the URL information, first, the display image S-1 is displayed. The initial display image S-1 is a panoramic image taken at the inlet 41 photographing point P ₁ in the vicinity of the shopping center 40. FIG. 5 shows a state in which the display image S-1 is displayed on the screen 16a of the terminal 16. In the same figure, a forward-looking forward-facing icon 50 superimposed on the display image S-1 is displayed on the screen 16a. The forward icon 50 is an icon for switching to the next display image. The cursor linked to the pointing device is moved over the forward icon 50, and a button provided on the pointing device is pressed (hereinafter, this series of icons). Is simply "clicked"), the next display image S-2 connected in association with the display image S-1 in FIG. 4 is displayed.

FIG. 6 shows a state in which the display image S-2 is displayed on the screen 16a of the terminal 16. This image is immediately entered from the inlet 41 to the interior of the building, a panoramic still images captured by the imaging point P ₂ in FIG. As shown in the figure, in this display image S-2 showing the interior of the building, a central passage 42 extends forward (upward on the screen) at the center of the screen, and stores are located on both sides thereof. It is projected.

As shown in the figure, on the screen 16a, in addition to the forward-looking forward-facing icon 50, a downward-facing retreat icon 51 is displayed below the forward-facing icon 50 on the display image S-2. Is superimposed and displayed. The forward icon 50 is an icon for switching the display to the next display image S-3 connected to the display image S-2, as in the case of FIG. , The display image S-3 is switched and displayed. In other words, by clicking the forward icon 50, the point of view of the terminal user will be sequentially moved along the path of the moving image shooting, the inside of the terminal user is shopping center 40, photographing point P _1, P ₂ ,..., it is possible to experience a panoramic tour of the while moving in the order of P _7.

  On the other hand, when the reverse icon 51 is clicked, the display image S-1 that has been displayed so far is switched and displayed, contrary to the forward icon 50.

  As shown in FIGS. 5 and 6, near the lower end of the screen 16a of the terminal 16, there are displayed arrow-shaped icons 52, 53, 54, and 55 indicating left, right, up, and down directions, respectively. These are icons for appropriately changing the direction of the viewpoint of the terminal user. For example, when the terminal user selects and presses the icon 52 indicating the left direction in FIG. 6, the image displayed on the screen moves rightward, and the store L1 is displayed at the center of the screen 16a. Is displayed, and the terminal user can hear and know the inside of the store L1. When the icon 53 indicating the right direction is selected and pressed, the image displayed on the screen moves to the left, and the state of the store R1 is displayed in the center of the screen 16a, and the terminal user Can inquire about the inside of the store R1.

  Further, when the icon 54 indicating the upward direction is selected and pressed, the image displayed on the screen moves downward, and the ceiling of the building is projected in the center of the screen 16a, indicating the downward direction. When the icon 55 is selected and pressed, the image displayed on the screen moves upward, and the floor of the building is displayed in the center of the screen 16a. When the plus icon 56 displayed alongside these arrow icons is selected and pressed, the image is enlarged and displayed. When the minus icon 57 is selected and pressed, the image is displayed. Is displayed in a reduced size.

As described above, the server 10 of the present embodiment includes the image conversion unit 30 and can obtain a plurality of panoramic still images from a panoramic video. A panoramic video for obtaining a panoramic still image can be created by moving the camera 12 maintaining the shooting state along the point where the image is desired to be obtained as shown in FIG. Complete.
For this reason, in order to obtain a panoramic still image at a plurality of points, it is not necessary to repeat the photographing work, such as installing and fixing a camera, photographing (acquiring a still image), and moving to the next photographing point, which was required conventionally. The cost and work period required for the work can be reduced.

  The server 10 further includes an image extracting unit 32 that extracts a specific panoramic still image as a display image from a large number of panoramic still images extracted from a moving image, and an image connecting unit 34 that associates and connects the display images. ing. In this server 10, since the conversion, extraction and connection of the panoramic still image are automatically performed and the content for the panoramic tour is generated, the time required for generating the content including the reduction in the number of shooting steps described above is greatly reduced. Can be shortened.

  Further, in the present embodiment, the display image (panoramic still image) is connected to the external information terminal in the order in which the images were taken because the image connection means 34 connects adjacent display images when they are arranged in the order of shooting. 16 and so on. For this reason, the creator of the content sets the shooting route in the order in which the user wants to view the video when shooting the moving image, and shoots the moving image along the route. That is, a plurality of display images can be browsed in the order intended by the creator.

  The embodiment of the present invention has been described in detail above, but this is merely an example. In the above-described embodiment, a panoramic still image having an angle of view of about 360 degrees is converted and extracted from a panoramic video having an angle of view of about 360 degrees. Therefore, it is also possible to convert and extract a general still image that does not have a wide angle of view as a panoramic still image. Further, the number of still images to be extracted from a moving image, the order in which display images (extracted still images) are displayed on an information terminal, and the like can be appropriately changed. In the above embodiment, the required still image is obtained from one moving image. However, in some cases, the required still image can be obtained from two or more moving images. The present invention can be implemented with various modifications without departing from the spirit of the invention.

10 server (content generation device)
14 Client terminals (information terminals)
16, 18, 20 User terminals (information terminals)
30 image conversion means 32 image extraction means 34 image connection means

Claims (4)

An apparatus for generating content including a plurality of display images displayed on an external information terminal,
Image conversion means for converting a captured moving image into a number of still images corresponding to each frame ,
Image extraction means for extracting a specific still image from the many still images based on a predetermined condition as the display image,
Image connection means for connecting and connecting images corresponding to a specific frame as the display image extracted without using the imaging position information about the display image,
Without passing through a three-dimensional model generated based on an image corresponding to the specific frame, an image corresponding to the specific frame obtained from the moving image is displayed on the screen of the information terminal as the display image. A content generation apparatus for generating the content to be switched and displayed. In the image connection unit, when arranged in the shooting order, the display images adjacent to each other are connected to be switchable and displayable on the screen of the information terminal and connected,
When selecting a forward icon superimposed and displayed on the display image on the screen of the information terminal, generating the content for switching and displaying the display image in the shooting path of the moving image. The content generation device according to claim 1.   The content generation device according to claim 1, wherein the moving image is a panoramic moving image, and the still image obtained as the display image from the panoramic moving image is a panoramic still image. . A method for generating content including a plurality of display images displayed on an external information terminal,
The captured moving image is converted into a number of still images corresponding to each frame, and a specific still image is extracted from the number of still images based on a predetermined condition as the display image. Connect the images corresponding to the specific frame as the display image extracted without using the imaging position information and connect them,
Without passing through a three-dimensional model generated based on an image corresponding to the specific frame, an image corresponding to the specific frame obtained from the moving image is displayed on the screen of the information terminal as the display image. A content generation method, wherein the content to be switched and displayed is generated.

Images