JP2007221498A - Camera, music image device, and music image reproducing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera, music image device, music image reproducing method or the like for simultaneously reproducing a music image composed in appropriate combination when a photographed image is viewed. <P>SOLUTION: Music 7b and a sample image 7a suitable for the music are recorded in a first memory 7. A similarity detecting part 18a of a similar image selecting part 18 refers to first auxiliary data of the sample image 7a or second auxiliary data and detects a photographed image that resembles the sample image among photographed images 9b recorded in a second memory 9. When a proper photographed image can not be detected, the sample image is selected. When reproducing a music image, a similar image selected in place of the sample image is reproduced in accordance with the progress of the music. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a camera that performs music image reproduction so that music and images can be enjoyed simultaneously, such a music image reproduction method, a music image device, and the like.

  Many cameras are equipped with so-called slide show playback, in which captured images are switched and played back over a certain period of time so that the playback function of the digital camera is further utilized. In this slide show function, since it is tedious to simply play back the slide show, a camera that can add a title or layout an image has been proposed (Patent Document 1).

Further, since the effect is limited only by image reproduction, a method of reproducing the music and the image at the same time by switching the image according to the music has been proposed. As one of the methods, a file processing method that manages still image file and music file and reproduces an image at a predetermined timing of music has been proposed (Patent Document 2).
JP 2005-176216 A JP 2002-300520 A

  As described above, not only images but also music can be reproduced at the same time, and the images can be switched at the effective timing according to the music, so that the images can be enjoyed more enjoyably. However, in order for the effect to be fully demonstrated in the reproduction of music images, it is necessary to match the music and images well. If the switching between music and an image or an image does not match the music, the feelings are inconsistent, which is counterproductive. For beginners, it is not always easy to select music according to the image or to switch the image according to the progress of the music. Compared to this, an image switching timing and image selection created by an expert have a marked difference compared to amateur ones. On the other hand, it is not easy for a beginner to imitate an expert, and considerable experience is required to become an expert. There is a need for a camera that enables even beginners to perform combined playback of music images that are at the level of an expert.

  In view of the above problems, an object of the present invention is to provide a camera, a music image device, a music image playback method, and the like that play back music images having an appropriate combination when viewing captured images.

In order to achieve the above object, a camera according to the present invention includes an imaging unit that captures a subject,
A control unit for controlling music image playback for displaying images according to music;
A similar image selection unit that selects an image similar to the sample image recommended for combination with music from among the captured images, and the control unit selects the image selected by the similar image selection unit instead of the sample image An image is displayed in accordance with music.

  In addition, the music image device according to the present invention selects from the photographed images an image similar to a sample image for which a combination of music is recommended and a control unit that controls music image playback for displaying an image in accordance with music. A similar image selection unit, wherein the control unit displays a photographed image selected by the similar image selection unit instead of the sample image in accordance with music.

  The music image reproduction method according to the present invention is a sample image that is recommended to be combined with music in a music image reproduction method for switching and displaying an image in accordance with music, and is similar to a sample image previously recorded on a recording medium. An image is selected from captured images, and the selected captured image is displayed instead of a sample image in accordance with music.

A music distribution system according to the present invention is a music distribution system for distributing music from a provider's computer to a user's computer connected via a network.
The computer of the user transmits an image used for music image playback that performs image playback in accordance with music and information on the mood of the music image playback to the provider computer.
The provider computer selects music according to the transmitted image and mood and transmits the selected music to the user computer.

  According to the present invention, it is possible to provide a camera, a music image device, a music image reproduction method, and the like that reproduce music images made up of appropriate combinations when viewing captured images.

The best mode for carrying out the present invention will be described below with reference to the drawings.
(First embodiment)
The first embodiment will be described with reference to FIGS. When viewing images while switching images, for example, a screen such as a TV is often used. If no sound is produced at this time, the atmosphere becomes unsatisfactory. Compared to this, not only images but also music can be played back at the same time, and images can be switched sequentially at an effective timing according to the music, so that the images can be enjoyed more enjoyably. However, when the music does not match the image or the switching of the image, or the music does not match, the emotional disagreement is caused. The switching timing and image selection for this hit are far more impressive when coordinated by an expert.

Therefore, in the present embodiment, a picture similar to the sample image is searched from images taken by the user on the basis of “music and sample images and switching timings” created in advance by experts, and this image is sampled. Play instead of images. The sample image is an image taken by an expert as an image combined with music. As a result, using the images taken by the user, the expert's knowledge is utilized to make it possible to play the album effectively by valuing the general flow such as the timing and change of switching. In the following description, the combined reproduction of music and images is also called album reproduction. Details will be described below. FIG. 1 is an overall block diagram of a digital camera to which the present invention is applied.

  The camera 21 includes an imaging unit 19, a signal processing unit 5, a RAM 6, and a second memory 9. The imaging unit 19 includes a photographic lens 2, an imaging element 3, and an analog front end 4. The taking lens 2 forms an image of the incident subject 20 on the image sensor 3. The image sensor 3 is composed of a CCD or a CMOS, and converts the formed subject image into an electrical signal. The analog front end 4 AD-converts the electrical signal from the image sensor 3 and outputs it as digital image data.

  The signal processing unit 5 includes a compression unit, an expansion unit, and the like, and performs processing such as color correction and signal compression on the image data. The compression unit performs, for example, JPEG compression or MPEG compression for recording the image data in the second memory 9, and has a sound compression function. The decompressing unit performs processing for restoring the compressed image data. The RAM 6 is used as a work area for temporary storage of various calculation data and image data during compression / decompression.

  The second memory 9 is a memory for storing image data, and is composed of, for example, a flash memory or a hard disk. The second memory unit 9 is also provided with an area for a transmission memory 9a. The image data processed by the signal processing unit 5 is recorded in the second memory 9 as photographed image data 9b.

  Further, the camera 21 is provided with a reproduction control unit 17, a display unit 8 for displaying images, a microphone 12, and a speaker 13. The reproduction control unit 17 controls display of an image on the display unit 8 and audio output to the speaker 13. The reproduction control unit 17 sequentially reproduces and displays the image output from the image sensor 3 on the display unit 8 for image confirmation at the time of shooting. At the time of reproduction, the reproduction control unit 17 causes the display unit 8 to display an image read from the first memory 7 or the second memory 9 and expanded by the expansion unit.

  Further, the camera 21 is provided with a print signal output unit 10 and a wireless transmission unit 11. The print signal output unit 10 is an interface unit that outputs captured image data to a connected printer. The wireless transmission unit 11 is an interface unit that wirelessly transmits image data to a printer or a PC (Personal Computer).

  The camera 21 includes a first memory 7, a similar image selection unit 18, MPU 1, an operation unit 16 connected to the MPU 1, a clock unit 14, and a position detection unit 15. The first memory 7 is a non-volatile and recordable memory, for example, a flash ROM. The first memory 7 stores a control program for performing camera processing and records sample image data and music data in advance. Details of the sample image data and music data will be described later. The similar image selection unit 18 will also be described later.

  The MPU 1 is a control unit that controls the entire camera, such as shooting and playback, according to a program stored in the first memory 7. Further, the operation unit 16 notifies the MPU 1 of the photographer's instruction. As the operation unit 16, switches 16a, 16b, and 16c are, for example, a release, a mode switching instruction, and a power ON / OFF switch. The clock unit 14 is used for recording the shooting date and time, recording the exposure time and the shooting time of a moving image, and measuring a predetermined timing interval. The position detection unit 15 includes a GPS sensor, detects a captured position, and notifies the MPU 1 of position data. The detected shooting position is recorded in the second memory 9 along with the image data.

  FIG. 2 is a diagram showing a relationship between music, a sample image incorporated in accordance with the music, and a captured image that is actually reproduced. FIG. 2A is a diagram showing a part of a musical score of reproduced music. FIG. 5B shows the switching of sample images set in accordance with the progress of this music. Sample images B1 (landscape), B2 (landscape and person), B3 (person), and B4 (flower) are set to be switched in the order of the number in accordance with the progress of music. The sample image is an image taken by a professional photographer and is prepared in advance as an image that matches the progress of music.

  FIG. 6C is an image taken by the user and is actually reproduced on the display unit 8 as the music progresses. The images C1, C2,... Are images that are searched automatically or manually from the captured images as images similar to B1, B2,. The selection criterion here is a scene (for example, landscape) of each image, and an image of the same scene as the sample image is selected. That is, C1 that is a landscape image is selected instead of B1 that is a landscape image. If there is no suitable image similar to the sample image in the captured image, the sample image is displayed as it is. For example, C4 remains the sample image of B4. This makes it possible to reproduce professional music images.

  FIG. 3 is a functional block diagram of a portion related to the music reproduction. FIG. 3 is a functional unit that is a feature of the present invention extracted from the overall block diagram of FIG. The same components as those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted. At the time of shooting, the image data output from the image pickup unit 19 and compressed and processed by the signal processing unit 5 is recorded in the second memory 9 as the shot image data 9b. When the audio signal is recorded, the audio data is also recorded linked to the photographed image data 9b. In addition to the program, sample image data 7a and music data 7b are recorded in the first memory 7 in advance. Music is recorded multiple songs. In the sample image data 7a, the corresponding music title and display timing according to the progress of music are also recorded. A combination of this music and sample images is registered in the first memory 7 as an album. The reproduction control unit includes a display control unit 17a and a speaker control unit 17b. The display control unit 17a controls image display on the display unit 8 during reproduction. The speaker control unit 17b reproduces music from the speaker 13 during reproduction.

  The similar image selection unit 18 includes a similarity detection unit 18a, an image selection unit 18b, and a characteristic extraction unit 18c. The similar image detection unit 18 a is a unit that detects an image similar to the sample image from the captured images recorded in the second memory 9. The image selection unit 18b is a unit that selects an image detected by the similarity detection unit 18a. The characteristic extraction unit 18c extracts scene conditions attached to the sample image, and further analyzes image data to extract image characteristics. This data is used to determine a “blue image”, a “geometric image”, etc. from the color and contour information (step S53 described later).

  The process by the above block is demonstrated easily. An album playback mode is designated by the user, and a specific album is designated by name and number. The MPU 1 reads the music corresponding to the album from the first memory 7 and the sample image or the selected captured image as the display image from the second memory 9. Then, the reproduction control unit 17 displays the image on the display unit 8. The MPU 1 controls the reproduction control unit 17 in accordance with the music timing based on the display timing data.

  Here, two modes of “automatic” and “manual” are prepared as album creation modes, which are selected by the user. “Auto” is a mode in which the similarity detection unit 18a automatically selects an image similar to the sample image set for the selected music from the photographed image data 9a to create a new album. . “Manual” is a mode in which the user selects an image while creating a new album while comparing the sample image and the captured image. Details will be described later.

  In addition, although the said structure demonstrated in the example applied to the camera, it can apply also to apparatuses other than a camera except an imaging part. Therefore, the present invention is also applicable to a music image device 30 that has a memory for storing music and images, as shown by the broken line range, and reproduces music and images.

4 and 5 are flowcharts showing the control processing relating to such music image reproduction. This control process is mainly executed by the MPU 1.
First, it is determined whether or not there is an instruction to play an album by a user operation (step S1). When there is no album playback instruction, a shooting operation is accepted as a shooting mode (step S1N). Then, it is determined whether or not the user has operated the release button (step S2). When there is no release button operation (step S2N), it is determined whether or not there is an operation for controlling a zoom lens for photographing or a mode selection operation (step S3). When a zoom lens control operation or mode selection operation instruction is issued (step S3Y), control according to the instruction is performed (step S4). If there is no such operation instruction, the process jumps to step S4 and proceeds to step S5.

  Next, audio information input from the microphone is detected prior to shooting (step S5). This is to determine whether it is characteristic such as laughter or cry. Subsequently, it is determined whether or not the voice information has a feature (step S6). When there is a feature (step S6Y), this audio information is recorded in the first memory 7 (step S7). After completing the above, the process returns to step 2.

  On the other hand, when the release button is operated in step S2 (step S2Y), shooting is performed. First, focusing by autofocus (step S11) and exposure control are performed (step S12). The captured image data is recorded in the second memory 9 as captured image data 9b (step S13). Further, information based on the focus position information, mode setting information, zoom information, exposure control, and audio information at this time is recorded as “auxiliary information” in the second memory 9 in association with each captured image (step S14). . These pieces of information are used as auxiliary data at the time of similar image search described later. This completes the shooting.

Next, the album playback operation will be described. When album playback is selected by the user in step S1 (step S1Y), the process proceeds to step S21 in FIG.
An album instructed by the user is selected from a plurality of albums registered in the first memory 7 (step S21). Next, it is determined whether or not an instruction to create a new album has been issued from the album selected by the user (step 22). When there is no instruction (step 22N), the process proceeds to the album playback process from step 23 onward. First, an image corresponding to the selected album is read out. When the image to be used is a sample image, the image data is read out from the first memory 7 and when it is a photographed image from the second memory 9 and decompressed and stored in the RAM (step S23). This is so that the image can be reproduced immediately.

  Then, the reproduction of the selected music and image is started (step S24). The image switching timing is determined from the switching timing data (step S25), and when that timing is reached (step S25Y), the images are sequentially switched (step S26). Then, the user's end instruction or the end of the song is determined (step S27). When the end instruction or the song ends (step S26Y), the reproduction ends.

  Next, returning to step S22, the album creation process will be described. This is a process of creating a new album by registering an image taken by itself instead of the sample image in the selected album. When the user instructs creation of a new album in step S22 (step S22Y), the process proceeds to step 31 and subsequent steps. As described above, “manual” and “automatic” are prepared as image selection conditions, and one of them is selected by the user (step S31).

  The explanation starts with “Manual”. When “manual” is selected (step 31Y), an image list as shown in FIG. 7 is displayed on the display unit 8 (step S32). FIG. 7 is an image selection screen for “manual”. A sample image m1, a captured image m2, a modified image m3, and an adopted image m4 are displayed on the screen. For example, three sample images m1 are displayed vertically, and image characteristics are displayed as feature comments (for example, portraits ...) beside each image. This is for selection assistance.

  The captured image m2 is an image based on the captured image data 9b, and here six images are displayed. The changed image m3 is an image currently registered in this album, and is an image to be changed on this screen. The changed image m3 is a sample image in the default state. The adopted image m4 is an image that is adopted (registered) in the album instead of the changed image m3. Any of the displayed sample image m1 and photographed image m2 may be selected. This is because there is not always an appropriate image in the captured image. The changed image m3 and the adopted image m4 are displayed in a large size with respect to the sample image m1. Switching between the sample image m1 and the photographed image m2 and determination of the adopted image are performed by the operation of the operation unit 16 by the user.

  Returning to FIG. The changed image m3 is selected by the user who has viewed the displayed image list (step S33). Then, the adopted image m4, which is an image selected by the user, is registered in place of the changed image m3 (step S34). As a result, a new album is created and registered (step S48). Data relating to the album in the first memory 7 is updated.

  On the other hand, if automatic is selected (step 31N), the process proceeds to step S41. The characteristic extraction unit 18c extracts image characteristics from the sample image (step S41). The image characteristics are characteristics (portrait, landscape, etc.) as described next to the sample image in FIG. 5 and are used as necessary.

  Next, the total number of images constituting the one album is set to n0 (for example, 10), and n0 is set to the counter n (step S42). Thereafter, images similar to the n0 images are sequentially searched. An image similar to the nth image in the album is searched (step S43). First, an image similar to the n0th image (for example, n = 10) is searched. By default, all the images constituting the album are sample images. Details of the similar image search procedure will be described with reference to the subroutine of FIG.

  FIG. 6 is a subroutine showing the details of the similar image search. Two types of auxiliary data are attached to the sample image. Of the auxiliary data, the first auxiliary data is data indicating an approximate type of image. For example, there are the types indicated by “portrait”, “bright landscape”, and “macro” beside the sample image in FIG. The second auxiliary data is data indicating an image that gives an equivalent atmosphere instead of the sample image even if they are not similar. That is, there is an image that can produce the same effect even if it is not similar to the original image, and the second auxiliary data indicates the type of the image.

  FIG. 8 shows a table of specific examples of the first auxiliary data and the second auxiliary data. This table shows the contents of the first auxiliary data and the second auxiliary data corresponding to the sample image for each sample image (No. 1, 2, 3, 4...) Belonging to each album.

  First, the first auxiliary data of the sample image is referred to (step S51). Then, a photographed image photographed in a scene similar to the sample image is searched from the first auxiliary data of the sample image (step S52). The scene of the captured image is determined from the information recorded in step S14 at the time of shooting. This is because a landscape or a macro image can be determined from the distance or the like from the focus information attached to the photographed image. A highly accurate similar image can be selected.

  If a similar captured image is detected (step S52Y), the process proceeds to step S53. In step S53, the image data of the sample image is further analyzed to extract image characteristics (step S53). Specifically, a “blue image”, a “geometric image”, or the like is determined from the color and outline information. As a result, even in the same landscape, it is possible to determine in detail whether it is a blue sky or a building. Correspondingly, the photographed image data is analyzed, and the photographed image having the corresponding characteristics is detected (step S54). If a plurality of captured images are detected in comparison with the first auxiliary data in step S52, it is possible to narrow down from a plurality of candidates here. When a similar captured image is detected (step S54Y), the captured image is adopted (step S55), and the process proceeds to step S44 in FIG.

  On the other hand, when a similar captured image based on the analysis data is not detected (step S54N), the process proceeds to a step of using the second auxiliary data. The second auxiliary data is data indicating an image that can be substituted for the sample image as described above. First, the second auxiliary data of the sample image is referred to (step S56). An alternative image indicated by the referenced auxiliary data is further detected from the images selected in step S52 (step S57). When a substitutable image is detected in the captured image (step S57Y), this is adopted (step S55), and the process returns to step 44. If no substitutable image is detected (step S57N), the process proceeds directly to step S44. Also, when a similar image based on the first auxiliary data is not detected in the captured image (step S52N), the process proceeds to step S44.

  Returning to FIG. The presence / absence of an adopted image (photographed image) as a result of the similar image selection is determined (step S44). When there is an image that can be adopted (step S44Y), the adopted image is replaced with a sample image (step S45). Conversely, when there is no adopted image, the process jumps to S45. Then, the counter n is decremented by 1, and the target sample image is set to the next numbered sample image (step 46). This is performed for all sample images (step S47). In the subroutine of FIG. 6, the photographed images to be compared and selected are compared with priority from the newest photographed date. This is to give priority to new images with a strong impression. When the comparison with all sample images is completed (step S47Y), a new album is created and registered based on the result (step S48).

  Since a similar image is selected in this way, a certain viewing effect can be obtained even if there is no image that exactly matches the sample image. In addition, since an image that cannot give a predetermined effect is excluded in step 52, a certain level can be guaranteed. Further, since the auxiliary data for detecting the similar image is attached to the sample image for detecting the similar image, the accuracy of the detection is improved. Further, the alternative display by the second auxiliary data can cope with the case where there is no direct similar image in the captured image, and the range of image selection is widened. Further, when no image corresponding to the sample image is found in the photographed image, the sample image can be displayed as it is, so that the display image can be prevented from being lost. In addition, you can select not only the camera, but also manual selection, so you can make a discerning album and fine-tune it to your liking. Since the selection screen is displayed at this time, comparison and selection can be easily performed.

By playing the album created in this way, the pleasure of playing music images is doubled. This is because the image reproduced together with the music is not an image of another person but an image taken by himself, and the selection and switching of the image conforms to professional expressions. It is very easy and efficient to play because it is a method that fits and reconstructs similar images taken by ordinary users on the example of combination playback of music and images made by experts. .
(Second Embodiment)
A second embodiment will be described with reference to FIG. FIG. 9 is a diagram showing an overall system and data flow of the second embodiment. The second embodiment relates to a sample image acquisition route. In the first embodiment, it has been described that music data, sample image data, and the like are recorded in the camera in advance. However, since such music data has a large amount of data, it is an example in which it is obtained from the outside as needed.

  This system includes a camera 21, a user PC (Personal Computer) 22 connected to the camera and performing data communication between the camera and the outside, and a network 23 to which the user PC 22 is connected. Furthermore, a base station 26 that connects the network 23 and the mobile phone 27 wirelessly and a provider PC 24 that provides data to the user are connected to the network 23. The provider PC 24 is further provided with a database 25. The database 25 stores music data used in the current album reproduction, sample image data with auxiliary data, and data of switching timing of sample image data.

  An arrow R1 is a flow of data transmission from the provider PC 24 to the camera 21. A data request is transmitted from the user PC 22 to the provider PC 24. Upon receiving this request, the provider PC 24 searches the database 25 for the requested music data, image data, and switching data. Then, each data searched by the provider PC 24 is transmitted to the user PC 22. The user PC 22 receives and stores the received music data and the like. Then, data is transferred from the user PC 22 to the camera 11.

An arrow R2 indicates the flow of data transmitted to the friend's mobile phone 27 via the user PC 22. The album data created by the camera 21 described above can also be transmitted to a friend's mobile phone 27 via the user PC 22.
(Third embodiment)
A third embodiment will be described with reference to FIGS. In the first embodiment, it has been described that a similar image is selected in accordance with music. In the present embodiment, on the contrary, the user selects an image that the user wants to view as an album and then selects music. The configuration of the system of this embodiment is the same as that shown in FIG. FIG. 10 is a flowchart 1 illustrating the flow of a system composed of users and providers. Similarly, FIG. 11 is a flowchart 2 illustrating the flow. This system is executed in cooperation with the user PC 22 and the provider PC 24. Therefore, as a flowchart, the processing executed by the user PC 22 and the processing executed by the provider PC 24 will be described in parallel. Processes executed by the user PC 22 are indicated by steps T1 to T23, and processes executed by the provider PC 24 are indicated by steps U1 to U13.

  First, the user performs an album creation operation with the camera 10 or the user PC 22. The user enters the “mood” (eg fun, sad, sad) of the album he wants to create. One “mood” is selected according to this input (step T1). Further, the user instructs an image to be made into an album from the photographed image of his camera 21. The instructed image is selected (step T2). Then, the user PC 22 transmits data regarding the selected “mood” and the selected image to the provider (step T3). “Mood” and data relating to the image are transmitted (arrow F1). After the transmission, the user PC 22 enters a standby state (step T4) and waits for reception from the provider PC 24 (step T5).

  The provider PC 24 is in a reception waiting state (step U1). When the provider PC 24 receives the data related to the “mood” and the image (step U1Y), the provider PC 24 narrows down the music that matches the transmitted “mood” from the database 25 (step U2). Data on each music in the database 25 is attached with data related to “mood”, and the provider PC 24 narrows down music that matches “mood” based on the attached data.

  Next, the music narrowed down by “mood” is further narrowed down based on the image transmitted from the user. A process of further narrowing down music as candidates from the above-mentioned auxiliary data (first auxiliary data and second auxiliary data) attached to the image is set to n0 images transmitted from the user PC 22, and the image 1 Do it one by one. First, a counter n for counting the number of images is initialized to 1 (step U3). Then, the aforementioned auxiliary data attached to the first image (n = 1) is referred to (step U4). Then, music candidates are narrowed down based on the auxiliary data (step U5). The counter n is incremented by 1 (step U6), and the same narrowing is performed using the next numbered image. It is determined whether all the images have been completed (step U7). When not completed (step U7N), the process returns to step U5, and after completion (step U7Y), the process proceeds to step U8.

  And it is judged whether the music which satisfy | fills a predetermined condition was detected (step U8). When music is detected (step U8Y), a list of song names of the detected music is transmitted to the user's PC (F2, step U10). On the other hand, when there is no corresponding music (step U8N), the auxiliary data attached to the music is changed (step U9). Then, the process returns to step U4 so that the candidate music search is again performed.

  When the waiting user PC 22 receives the music candidate data (step T5Y), the process proceeds to step T11 in FIG. Note that the user PC 22 checks the elapse of a predetermined time in a reception waiting state (step T6). If reception is not possible even if a certain period of time is exceeded due to the communication line or the like (step T6Y), the user PC 22 displays a warning (step T7) and finishes the process in consideration of energy saving and connection charges.

  The user PC 22 that has received the music title list displays a list of the sent song titles (step T11). The user selects music while watching the screen. When music is selected (step T12Y), the selected song name is transmitted to the provider (steps T13 and F3). And it will be in a standby state (step T14). In this selection state (step T12), when a predetermined time or more has elapsed (step T15Y), after the warning (step T16), the end processing is performed.

  The provider PC 24 is in a reception waiting state (step U11). When the provider PC 24 receives the song title transmitted from the user PC 22 (step U11Y), the corresponding music data (actual data) is read from the database 25 and transmitted to the user PC 22 (step U12). The music data to be transmitted is accompanied by image switching data.

  When the user PC receives the music data (step T17), a corresponding album is created (step T18). The user checks the effect (step T19). When OK is input by the confirmed user, the user PC 22 transmits an end signal to the provider PC 24 (steps T23 and F5). After the transmission, the provider PC 24 enters an end signal waiting state (step U13) and a song title reception waiting state (step U11). When receiving the end signal, the provider PC 24 ends this process. On the other hand, if the music is not liked as a result of the check, a retry is made (step T20Y), and the process returns to step T11. Further, after a predetermined time has elapsed in the reception waiting state (step T17) (step T21N), a warning is issued (step T22) and the process is terminated.

As described above, the third embodiment can provide a service for creating an album by attaching appropriate music to an image selected by the user. It is possible to provide a system that enables even beginners to view images with music effectively. Since common auxiliary data (as shown in FIG. 8) is attached to the image data and the music data, it is possible to simply perform a music refinement search at a high speed. Further, following the list display in step T11, the sample image may be viewed with BGM. Furthermore, the completeness of the album can be improved surely. In this case, the image is replaced by a method for discriminating the feature matching between the features of each sample image and the features of the image selected by the user. If you check the effect and then download all the data, you can save camera memory.
In the above description, the user PC 22 performs the process on the user side. However, if the camera 21 has a communication function, the MPU 1 of the camera 21 may perform the process independently without using the user PC 22. Naturally good.

The following invention is extracted by each embodiment described above.
1) an imaging unit 19 for photographing a subject;
A control unit (MPU1) for controlling music image playback for displaying images in accordance with music;
A similar image selection unit 18 for selecting an image similar to the sample image 7a recommended for combination with music from the photographed image 9b, and the control unit uses the similar image selection unit 18 instead of the sample image 7a. A camera 21 that displays a selected photographed image 9b according to music.

  2) The camera 21 according to 1), wherein the similar image selection unit 18 includes a similarity detection unit 18a that detects a captured image 9b similar to the sample image 7a.

3) a control unit (MPU1) for controlling music image reproduction for displaying an image in accordance with music;
A similar image selection unit 18 that selects an image similar to the sample image 7a recommended for combination with music from the photographed image 9b, and the control unit uses the similar image selection unit 18 instead of the sample image 7a. A music image apparatus 30 that displays a selected photographed image 9b in accordance with music.

  Note that part or all of the processing of the MPU 1 described in the first embodiment may be configured by hardware. Conversely, the similar image selection unit may naturally be processed by software, and the specific configuration is a design matter. Further, each control process by the MPU 1 is realized by a software program stored in the first memory 7 being supplied to the MPU 1 (recording control unit 1a) and operating according to the supplied program. Therefore, the software program itself realizes the functions of the MPU 1, and the program itself constitutes the present invention. A recording medium for storing the program also constitutes the present invention. As a recording medium, besides a flash memory, an optical recording medium such as a CD-ROM or DVD (Digital Versatile Disk), a magnetic recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used.

  Furthermore, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

1 is an overall block diagram of a digital camera to which the present invention is applied in a first embodiment. The figure which shows the relationship between the sample image incorporated according to the music and the music in 1st Embodiment, and the picked-up image actually reproduced | regenerated. The functional block diagram of the part which concerns on music reproduction in 1st Embodiment. 1 is a flowchart 1 showing a control process related to music image reproduction in the first embodiment. 2 is a flowchart 2 illustrating a control process related to music image reproduction in the first embodiment. The subroutine which shows the detail of a similar image search in 1st Embodiment. The figure which shows the screen for image selection at the time of "manual" in 1st Embodiment. The figure which shows the table | surface of the specific example of 1st auxiliary data and 2nd auxiliary data in 1st Embodiment. The figure which shows the whole system and the flow of data in 2nd Embodiment. Flowchart 1 illustrating the flow of a system composed of a user and a provider in the third embodiment. Flowchart 2 for explaining the flow of a system composed of users and providers in the third embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... MPU, 2 ... Shooting lens, 3 ... Imaging element, 4 ... Analog front end, 5 ... Signal processing part,
6 ... RAM, 7 ... first memory, 8 ... display unit, 9 ... second memory, 9a ... memory for transmission,
9b ... photographic image data, 10 ... print signal output unit, 11 ... wireless transmission unit, 12 ... microphone,
DESCRIPTION OF SYMBOLS 13 ... Speaker, 14 ... Clock part, 15 ... Position detection part, 16 ... Operation part, 17 ... Playback control part,
17a ... display control unit, 17b ... speaker control unit, 18 ... similar image selection unit, 18a ... similarity detection unit,
18b ... Image selection unit, 18c ... Characteristic extraction unit, 19 ... Imaging unit, 20 ... Subject image,
DESCRIPTION OF SYMBOLS 21 ... Camera, 22 ... User PC, 23 ... Network, 24 ... Provider PC, 25 ... Database, 26 ... Base station, 27 ... Mobile phone, 30 ... Music image apparatus

Claims (17)

An imaging unit for photographing a subject;
A control unit for controlling music image playback for displaying images according to music;
A similar image selection unit that selects an image similar to the sample image recommended for combination with music from among the captured images, and the control unit selects a captured image selected by the similar image selection unit instead of the sample image A camera that displays music in time with music. The camera according to claim 1, wherein the similar image selection unit includes a similarity detection unit that detects a captured image similar to the sample image. The camera according to claim 2, wherein the similarity detection unit performs detection in consideration of incidental information attached to the sample image. In the sample image, information on the shooting scene is attached as the accompanying information,
The similarity detection unit detects a captured image of the same shooting scene as the sample image,
The camera according to claim 2. The camera according to claim 4, wherein the similarity detection unit detects a shooting scene of the shot image based on shooting information attached to the shot image. The sample image is accompanied by information indicating an image that can be substituted as the accompanying information.
The similarity detection unit detects a captured image similar to the sample image based on the information indicating the replaceable image;
The camera according to claim 2. 3. The camera according to claim 2, wherein when the captured image similar to the sample image is not detected by the similarity detection unit, the control unit displays the sample image as it is. The camera according to claim 1, wherein the control unit displays a screen on which the sample image and the captured image are displayed simultaneously as a screen for allowing the operator to select a captured image to replace the sample image. 3. The camera according to claim 2, further comprising a memory in which the music data, sample image data, and data relating to image switching timing according to music are recorded. A control unit for controlling music image playback for displaying images according to music;
A similar image selection unit that selects an image similar to the sample image recommended for combination with music from among the captured images, and the control unit selects a captured image selected by the similar image selection unit instead of the sample image A music image device that displays music according to music. The similar image selection unit includes a similarity detection unit that detects a captured image similar to the sample image,
The music image apparatus according to claim 10, wherein the similarity detection unit detects in consideration of incidental information attached to the sample image. 11. The music image apparatus according to claim 10, wherein the control unit creates a new album in which the sample image is replaced with the selected similar image from the album related to the combination of music and the sample image. In a music image playback method of switching and displaying images according to music,
A sample image that is recommended to be combined with music, and select a similar image to the sample image recorded in advance on the recording medium, and replace the selected captured image with the sample image. A music image reproduction method characterized by displaying the image in accordance with the above. In a program for causing a computer to execute a music image reproduction method for switching and displaying images according to music,
The music image playback method is as follows:
A sample image that is recommended to be combined with music, and select a similar image to the sample image recorded in advance on the recording medium, and replace the selected captured image with the sample image. A program characterized by being displayed to match. In a computer-readable recording medium on which a program for causing a computer to execute a music image reproduction method for switching and displaying images according to music is recorded,
The music image playback method is as follows:
A sample image that is recommended to be combined with music, and select a similar image to the sample image recorded in advance on the recording medium, and replace the selected captured image with the sample image. A computer-readable recording medium characterized by being displayed according to the above. In a music distribution system that distributes music from a provider's computer to a user's computer connected via a network,
The computer of the user transmits an image used for music image reproduction that performs image reproduction according to music and information on the mood of the music image reproduction to the provider computer,
The computer of the provider selects music according to the transmitted image and mood and transmits the selected music to the user's computer. In a music receiving method in a user's computer connected via a network to a provider's computer that provides a music distribution service,
The computer of the user transmits an image used for music image playback that performs image playback in accordance with music and information on the mood of the music image playback to the provider computer.
A music receiving method in a user's computer, wherein the music selected and transmitted from the computer of the provider according to the transmitted image and mood is received.

Images