JP2018125591A - Image reproducer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image reproducer which allows for efficient browsing of images, related to a target image with multiple different directionality, when browsing an image in a limited screen area.SOLUTION: An image reproducer has attribute information analysis means for analyzing the attribute information of a main image, relevant information selection means for selecting multiple pieces of relevant information according to individual attribute information, relevant information selection means for grouping the images of management object conforming to the individual relevant information, and selecting which relevant information is employed according to the number of groups and the number of images of a group including the main image, image retrieval means for retrieving an image conforming to the relevant information selected by the relevant information selection means, auxiliary image selection means for selecting the image retrieved by the image retrieval means as an auxiliary image, and auxiliary image display means for displaying the image selected by the auxiliary image selection means.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an image reproduction apparatus that reproduces a plurality of images.

  As the capacity of recording media increases, the opportunity to view a large number of images with a device having a small screen area such as a mobile phone or a digital camera is increasing. 2. Description of the Related Art Conventionally, there is an image display device that displays an image related to a main image. (See Patent Document 1.)

International Publication No. 2007/126097

  However, in Patent Document 1, when sequentially browsing a large number of images within a limited screen area, it has not been possible to efficiently browse images related to a main image of interest in a plurality of different directions. .

  In order to solve the above-described problem, the image reproduction device of the present application aggregates the images to be managed based on the related information, selects the related information according to the result, and displays the matched image.

  It becomes possible to efficiently browse images related to a main image of interest in a plurality of different directions.

It is a block diagram which shows the structure of the image reproduction apparatus which concerns on one Embodiment of this invention. It is a figure which shows the reproduction | regeneration screen which the image reproduction apparatus which concerns on one Embodiment of this invention displays on a display apparatus. It is an example of the data which shows a response | compatibility with the attribute information and related information in the image reproduction apparatus which concerns on one Embodiment of this invention. It is a conceptual diagram of the database used in the image reproduction apparatus which concerns on one Embodiment of this invention. It is a conceptual diagram of the database used in the image reproduction apparatus which concerns on one Embodiment of this invention. It is a flowchart figure which shows the process of the related information processing of the image reproduction apparatus which concerns on one Embodiment of this invention. It is an example of the list | wrist of the relevant information memorize | stored in the image reproduction apparatus which concerns on one Embodiment of this invention. It is a flowchart figure which shows the process of the sub image selection process of the image reproduction apparatus which concerns on one Embodiment of this invention. It is a flowchart figure which shows the process of the image reproduction process of the image reproduction apparatus which concerns on one Embodiment of this invention. It is an example of the list | wrist of the relevant information memorize | stored in the image reproduction apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the image reproduction apparatus which concerns on one Embodiment of this invention. It is a figure which shows the reproduction | regeneration screen which the image reproduction apparatus which concerns on one Embodiment of this invention displays on a display apparatus. It is a flowchart figure which shows the process of the image reproduction process of the image reproduction apparatus which concerns on one Embodiment of this invention. It is a flowchart figure which shows the process of the sub image selection process of the image reproduction apparatus which concerns on one Embodiment of this invention.

[Example 1]
The image reproduction apparatus according to the present embodiment uses still image data obtained by an electronic camera such as a digital still camera or a digital video camera that performs imaging by converting an optical image into electrical image data using a solid-state image sensor. Display video data. In this embodiment, the image reproducing apparatus of this embodiment that operates on a smartphone that is a multi-function mobile phone made based on the functions of a personal computer will be described.

  The image reproduction apparatus according to the present embodiment displays an associated image together with the reference image with an arbitrary image as a reference. Hereinafter, the reference image is referred to as a main image. A related image is referred to as a sub-image. Still image data and moving image data obtained by an electronic camera hold attribute information in a file header or an associated file. The attribute information is information necessary for reading and reproducing information at the time of photographing and images. The attribute information in the still image data includes shooting date / time, shooting location, scene information, shooting mode, subject information, event information, and the like. In this embodiment, the shooting location is information on latitude and longitude acquired by the electronic camera using GPS.

  In this embodiment, it is assumed that the shooting mode and event information are set by the user at the time of shooting. The attribute information in the moving image data includes the recording information, the recording start time, the recording end time, the number of frames recorded per second, the number of frames played back per second, and information in addition to the attribute information of the still image data. , Etc.

  In the image reproducing apparatus of this embodiment, information obtained by converting attribute information into information that can be easily interpreted by a person is referred to as related information. When the attribute information is the shooting date and time, the corresponding related information is a season, a month, a day of the week, and a time zone constituting one day. When the attribute information is a shooting location, the location information, landmark information, and information indicating the characteristics of the region are used. When the attribute information is a subject, the information includes information indicating the characteristics of the subject, information indicating the relationship with the owner, and the name of an individual or an individual. When the attribute information is a scene, it is information on the shooting mode, event information, and scene analysis result specified by the user at the time of shooting. Details of the attribute information and the related information will be described later with reference to FIG.

Hereinafter, still image data and moving image data are collectively referred to as image data. A smartphone is described as a smartphone.
[Smartphone operating as an image playback device]
First, a smartphone that operates as the image playback device of the present embodiment will be described. Further, in order to clarify the point of the invention, the description will be focused on the image reproduction function for reproducing the image data in the storage medium, not the photographing function and communication function of the smartphone.

  FIG. 1 is a block diagram showing a smartphone that operates as an image playback device according to an embodiment of the present invention. By executing a predetermined control program in the smartphone, an image reproduction process described below is realized and functions as an image reproduction apparatus.

  In FIG. 1, reference numeral 101 denotes a CPU that controls the entire apparatus. Reference numeral 102 denotes a ROM that stores operation processing procedures of the CPU 101 (for example, programs such as computer startup processing and basic input / output processing). Reference numeral 103 denotes a RAM that functions as a main memory of the CPU 101. Various programs including a control program for realizing processing described below are loaded into the RAM 103 from the hard disk drive 105 or the like and executed by the CPU 101. The RAM 103 provides a work area when the CPU 101 executes various processes.

  A hard disk drive (hereinafter referred to as HDD) 105 is used for saving and reading application programs, data, libraries, and the like. Reference numeral 106 denotes an input device such as a touch panel and buttons. A display 104 performs various displays under the control of the CPU 101.

  Reference numeral 107 denotes a storage medium mounting unit (media drive) which mounts a detachable storage medium and can read out data stored and captured by a still digital camera. A network interface 108 is connected to the computer network 111 via a communication line 110. Data is transmitted and received from a communicable device via the network interface 108. Reference numeral 109 denotes a system bus (consisting of an address bus, a data bus, and a control bus) that connects the units described above.

  In the image reproducing apparatus of the present embodiment, it is assumed that image data and moving image data to be processed are stored in the HDD 105.

[User interface of image playback device]
FIG. 2 is a front view of a smartphone that operates as an image playback device according to an embodiment of the present invention. As in the description of FIG. 1, in order to clarify the point of the invention, the description will be focused on the image reproduction function. Note that the screen shown in FIG. 2 is the screen after the application software for the image playback function is started from the screen immediately after the user starts the smartphone (hereinafter referred to as the home screen). Details of the user interface of the image playback apparatus of the present embodiment will be described below with reference to FIG.

  201 is a display 104 that performs various displays. The 201 has a touch panel function as well as a display function, and fulfills the function of the input device 106. When the user presses any point on the screen with the finger, the CPU 101 determines that there is an input instruction from the user, determines the operation content from the pressed position, and performs various processes such as display update .

Reference numeral 202 denotes a home button. When the user presses the home button 202 after starting the application software for the image playback function, the CPU 101 determines that there is an instruction to return to the home screen from the user, and updates the display in the display 201 to the home screen.
Reference numeral 203 denotes a main image. The CPU 101 identifies image data from the main image information stored in the RAM 103, reads the corresponding image data from the HDD 105, and displays it. The main image information will be described later. 204, 205, 206, and 207 are sub-images. After the CPU 101 performs an image search process to be described later, the determined image data in the HDD 105 is read and displayed.

[Image playback processing]
FIG. 9 is a flowchart showing image reproduction processing steps in the image reproduction apparatus according to the embodiment of the present invention. This process is executed when the CPU 101 executes a control program stored in the RAM 103.

  In step S900, the CPU 101 identifies image data from the main image information stored in the RAM 103, reads the corresponding image data from the HDD 105, and displays it on the display 104 as the main image. A state in which the main image is displayed is indicated by 203 in FIG. If the main image information does not exist in the RAM 103 immediately after the image playback device is activated, the CPU 101 searches the image data of the latest shooting date and time using the database described later, and the image data is used as the main image. And

  In step S901, the CPU 101 stores the main image displayed in S900 as image data that has been displayed in the RAM 103 as the main image. The data displayed as the main image is held in the RAM 103 as a main image displayed list while the image reproducing apparatus is activated. In step S902, the CPU 101 performs related information selection processing which will be described later with reference to FIG. In step S903, the CPU 101 performs a sub-image selection process which will be described later with reference to FIG. In step S904, the CPU 101 displays the sub-image selected in S903 on the display 104. A state in which the sub-image is displayed is shown in 204, 205, 206, and 207 in FIG.

  In step S905, the CPU 101 determines whether there is a user operation. If the CPU 101 determines in step S905 that there is a user operation, the process proceeds to step S906. If the CPU 101 determines in step S905 that there is no user operation, the process returns to step S905. In step S906, the CPU 101 determines whether the user operation is an end instruction. The termination instruction is pressing of the home button 202. If the CPU 101 determines in step S906 that the instruction is to end, the information stored in the RAM 103 is discarded, and the image playback device is ended.

  If the CPU 101 determines in step S906 that the instruction is not an end instruction, the process advances to step S907. In step S907, the CPU 101 determines whether or not the user operation is a sub-image selection instruction. The sub image selection instruction is an operation in which the user presses any position of the sub images 204, 205, 206, and 207 with a finger. If the CPU 101 determines in step S907 that the instruction is a sub-image selection instruction, the process advances to step S908. If the CPU 101 determines in step S907 that the instruction is not a sub-image selection instruction, the operation returns to S905 as an invalid operation.

  In step S908, the CPU 101 determines which sub-image is selected by the user from the coordinates pressed by the user with the finger, and stores the selected sub-image in the RAM 103 as the next main image.

[List of attribute information and related information]
FIG. 3 is an example of data indicating the correspondence between attribute information and related information in the image reproduction device according to the embodiment of the present invention. It is assumed that it is stored in HDD105. When the CPU 101 executes DB registration processing and related information selection processing described later, it is read from the HDD 105 and used. 301 is attribute information No. for sequentially reading attribute information and executing processing to be described later. 302 is attribute information. Reference numeral 303 denotes related information No. for sequentially reading related information and executing processing to be described later. 304 is related information.

  The attribute information 305 is the shooting date and time, and the corresponding related information is 309, 310, 311 and 312. The attribute information 306 is a shooting location, and the corresponding related information is 313, 314, 315, 316, 317, 318, 319. The attribute information 307 is a subject, and the corresponding related information is 320, 321, and 322. The attribute information 308 is a scene, and the corresponding related information is 323, 324, and 325. The related information 309 is a season. One of spring, summer, autumn, and winter. The related information 310 is the month. One of the values from January to December.

  The related information 311 is a day of the week. One of the values from Sunday to Saturday. The related information 312 is a time zone constituting one day. When classified, the value is one of morning, noon, evening, night, or midnight. The related information 313 is a region. One of North America, South America, Asia, Oceania, Europe, and Africa. The related information 314 is a country. It becomes the value of the country name such as United States, Japan, China. The related information 315 is a region. If the country is Japan, the value will be one of Hokkaido, Tohoku, Kanto, Chubu, Kinki, China, Shikoku and Kyushu.

  The related information 316 is a prefecture. If the country is Japan, the values of prefecture names such as Hokkaido, Aomori, Iwate, and Akita. The related information 317 is a place name. If the country is Japan, the place names are Sapporo, Tokyo, Osaka, Kyoto, Fukuoka. The related information 318 is a landmark. This is the value of the name of the landmark building or symbolic monument near the shooting location. Statue of Liberty, Eiffel Tower, Sky Tree, Arc de Triomphe, Leaning Tower of Pisa.

  Related information 319 is regional characteristics. The value indicates the characteristics of the region such as sea, river, lake, pond, mountain, valley, wetland, town, and countryside. The related information 320 is a subject type. The value indicates the type of subject such as a person, an animal, a plant, an artifact, or a natural object. The related information 321 is information indicating a relationship with the owner. It shows the relationship between family, friends, pets, and crops. The related information 322 is the name of an individual or an individual. It is the value of the name of an individual or individual, such as the name of a person, the name of a pet, or the name of a work of artificial products. The related information 323 is a shooting mode. The values specified by the user at the time of shooting are landscape, portrait, night view, underwater, fireworks, sports, children, auto, and so on.

  The related information 324 is event information. This value indicates events such as travel, walks, weddings, sports day, entrance ceremony, graduation ceremony. The related information 325 is scene analysis information. This is the result of analyzing scenes such as the sea, night view, autumn leaves, sunset, and portrait.

[Database]
4 and 5 are conceptual diagrams of a database used in the image reproduction apparatus according to the embodiment of the present invention. FIG. 4 is a table for storing one piece of information in one piece of image data, and FIG. 5 is a table for storing information on a plurality of subjects from 0 for one piece of image data. In the image playback device of the present embodiment, the CPU 101 reads all the image data stored in the HDD 105 at the first start-up, associates the image data with attribute information and related information, and generates the database shown in FIG. 4 and FIG. To do. Then, the CPU 101 stores the database in the HDD 105. The search processing and grouping processing described later are realized by the CPU 101 using the databases shown in FIGS. Hereinafter, the database is referred to as DB.

  401 is a column that stores FileNo for uniquely identifying image data in the DB. When the CPU 101 registers data in the DB shown in FIG. 4, it stores a value incremented from 1 for each image data. Reference numeral 402 denotes a column for storing file names. The CPU 101 reads and stores the file name of each image data. Reference numeral 403 denotes a shooting date and time, and 404 denotes a column for storing shooting location information. In any case, the CPU 101 reads and stores the attribute information of each image data. Reference numeral 405 denotes a season, 406 denotes a month, 407 denotes a day of the week, and 408 denotes a column for storing time zone information. In any case, the CPU 101 reads the shooting date and time of each image data, converts it into each information, and stores it. In the conversion, it may be acquired by communicating with the server computer via the network interface 108.

  409 is a region, 410 is a country, 411 is a region, 412 is a prefecture, 413 is a place name, 414 is a landmark, and 415 is a column that stores information on region characteristics. In any case, the CPU 101 reads the shooting location of each image data, converts it into each information, and stores it. In the conversion, it may be acquired by communicating with the server computer via the network interface 108.

  Reference numeral 416 denotes a shooting mode, and reference numeral 417 denotes a column for storing event information. In any case, the CPU 101 reads and stores the attribute information of each image data. Reference numeral 418 denotes a column for storing scene analysis information. The CPU 101 analyzes each image data and stores the result of estimating the scene from the contents of the image. If the result of analysis shows that it is a sea scene, the sea. If it is determined that the scene is autumn leaves, it is autumn leaves. If it is determined that the scene has a person as the main subject, the portrait is stored. Reference numerals 419, 420, 421, and 422 are examples of rows that store information associated with one piece of image data.

  Reference numeral 501 denotes a column for storing a subject number for uniquely identifying subject data in the DB. When the CPU 101 registers data in the DB shown in FIG. 5, the value incremented from 1 is stored for each subject. Reference numeral 502 denotes a column for storing FileNo for specifying a file including a subject. This corresponds to 401 in FIG. The CPU 101 analyzes the subject for each image data, and when the subject is detected as a result of the analysis, information on the subject is stored in the table shown in FIG. At that time, the FileNo of the image data to be analyzed is stored in 502.

  Reference numeral 503 denotes a subject type, reference numeral 504 denotes a relationship with the owner, and reference numeral 505 denotes a column storing names of individuals or individuals. The CPU 101 analyzes each image data and stores the result of estimating the subject from the contents of the image. When a plurality of subjects are included in the image data, information on the subjects for the number of detected subjects is stored. For estimation of the subject, general person authentication, animal authentication, and individual identification processes are used. It is assumed that information necessary for subject detection and authentication is stored in advance in the HDD 105, and the CPU 101 reads the information to execute subject detection processing and subject estimation processing.

  Reference numerals 506, 507, and 508 are columns for storing information related to the position of the detected subject. Reference numeral 506 denotes a center coordinate of the subject, 507 denotes a subject width, and 508 denotes a column for storing the subject height. The subject detection process is as described above. Reference numerals 509, 510, 511, 512, and 513 are examples of rows that store information associated with one subject.

  For the image data shown in 419, when the CPU 101 cannot detect the subject, the subject corresponding to File No. 1 is not stored in the table of FIG. For the image data shown in 420, when the CPU 101 detects three subjects, three rows of subjects corresponding to File No. 2 are stored in the table of FIG. 509, 510, and 511 are applicable. When the CPU 101 detects two subjects for the image data shown in 420, two rows of subjects corresponding to File No. 3 are stored in the table of FIG. 512 and 513 are applicable.

[Criteria for selecting related information and reasons]
The reason why the user can view the image efficiently and happily when the sub-image that is an image related to the main image is displayed will be described. When grouping with related information, the number of groups including the main image and the sub-image is desirably 30% or less. This is because if it exceeds 30%, an image with low relevance will be displayed. It is desirable that the number of groups including the main image and the sub image is 60 or more. The reason is that if you browse images sequentially with the same relevance, if you have fewer than 60 images, you will end up seeing them immediately. If there are more than 60, you can browse for 5 to 10 seconds and enjoy around 5 minutes.

  It is desirable that the dates and times of the group images including the main image and the sub-image are distributed at different dates and times. This is because when browsing images sequentially with the same relevance, if all the images are concentrated on the same date and time, there is no difference from browsing on the date and time.

  A relevance satisfying the above conditions and a large number of groups when related information is grouped is desirable. This is because the larger the number of groups, the higher the relevance of images within the group.

  It is desirable that the relationship satisfies the above conditions and the attribute information is different. Rather than displaying multiple related sub-images with similar directions, it is possible to select related image data with the desired direction for the viewer when individual related sub-images are displayed with different directions. It is.

[Select related information]
FIG. 6 is a flowchart showing related information selection processing steps of the image reproduction device according to the embodiment of the present invention. This process is executed when the CPU 101 executes a control program stored in the RAM 103. FIG. 7 is an example of a list of related information stored in the image reproduction device according to the embodiment of the present invention.

  The process in which the CPU 101 selects related information will be described on the premise of the above-described related information selection criteria, with reference to FIGS. In step S600, the CPU 101 reads the list of attribute information and related information shown in FIG. 3 from the HDD 105 in order to execute a series of processes described later. In step S601, CPU 101 assigns 0 to counter k. The counter k is a counter for the CPU 101 to execute a series of processing from step S602 to step S612 to be described later for each piece of target attribute information. The value of the counter k corresponds to the attribute information No in FIG.

  In step S602, the CPU 101 determines whether the counter k is smaller than the number of target attribute information. The number of attribute information is the number of attribute information shown in FIG. If the CPU 101 determines in step S602 that the counter k is less than the number of target attribute information, the process proceeds to step S603. In step S602, if the CPU 101 determines that the counter k is equal to or greater than the number of target attribute information, the CPU 101 determines that a series of processing for the target attribute information has been completed, and proceeds to step S613.

  In step S603, the CPU 101 refers to the list of attribute information and related information in FIG. 3, and reads related information from the attribute information whose attribute information No is k. In step S604, CPU 101 assigns 0 to counter i. The counter i is a counter for the CPU 101 to execute a series of processes from step S605 to step S611, which will be described later, for each piece of related information to be processed. The value of the counter i corresponds to the related information No in FIG.

  In step S605, the CPU 101 determines whether or not the counter i is less than the number of pieces of related information targeted. The number of related information is the number of related information shown in FIG. 3 in the attribute information with attribute information No. k. If the CPU 101 determines in step S605 that the counter i is less than the number of relevant information, the process proceeds to step S606. In step S605, when the CPU 101 determines that the counter i is equal to or greater than the number of related information to be processed, it is determined that a series of processes for the related information to be processed has been completed, and the process proceeds to step S612.

  In step S606, the CPU 101 refers to the list of attribute information and related information in FIG. 3 and performs grouping on the target image data in the HDD 105 with the related information No. related information No. i. For grouping, the databases shown in FIGS. 4 and 5 are used. In step S607, the CPU 101 determines whether the ratio of the number of groups including the main image is 30% or less of the total number of image data.

  If the CPU 101 determines in step S607 that the total number of image data is 30% or less, the process proceeds to step S608. If it is determined in step S607 that the CPU 101 has more than 30% of the total number of image data, the process proceeds to step S611. In step S608, the CPU 101 determines whether the number of groups including the main image is 60 or more. If the CPU 101 determines in step S608 that the number is 60 or more, the process proceeds to step S609. If the CPU 101 determines in step S608 that the number is less than 60, the process proceeds to step S611.

  In step S609, the CPU 101 determines whether the images in the group including the main image are distributed at different dates and times. If it is determined in step S609 that the CPU 101 is distributed at different dates and times, the process proceeds to step S610. If it is determined in step S609 that the CPU 101 is concentrated on consecutive dates and times, the process proceeds to step S611. In step S610, CPU 101 stores related information and group information in RAM 103 for related information whose related information No is i. The information stored at this time is shown in FIG. The description of FIG. 7 will be described later. In step S611, CPU 101 increments counter i and returns to step S605.

  In step S612, CPU 101 increments counter k and returns to step S602. In step S613, the CPU 101 refers to the list of related information information in the RAM 103, and selects related information having a large number of groups for the number of sub-images. The number of sub-images is the number of sub-images displayed on the user interface shown in FIG. If there are multiple items with the same number of groups and the number of sub-images has been selected as a result of selection, the related information is selected from the selected related information so that the attribute information is as different as possible. . If it still exceeds the number of sub-images, related information with a large related information No is selected.

  In step S614, CPU 101 stores which related information is selected in the list of related information in RAM 103. At that time, the number is incremented from 0 in the selected order and the value is stored. Then, the processing is terminated assuming that a series of processing has been completed.

  A column 701 stores related information. In S610, the CPU 101 stores related information whose related information No is i. A column 702 stores attribute information. In S610, the CPU 101 stores attribute information whose attribute information No is k. Reference numeral 703 denotes a column for storing values of related information in the main image. When the main image is 421 in FIG. A column 704 stores the number of groups. In S610, the CPU 101 stores the number of groups as a result of grouping with the related information whose related information No is i.

  Reference numeral 705 denotes a column for storing the number of image data belonging to the group including the main image. When the related information is the month and the value of the main image is July, the CPU 101 stores the number of image data whose shooting date is July. A column 706 stores a ratio of the number of image data of 705 to the entire image data. When the number of image data 705 is 1000 and the total number of image data is 6000, the CPU 101 rounds off the decimal part and stores 17%. Reference numeral 707 denotes a column for storing selection numbers. In S613, the CPU 101 increments the number from 0 in the order in which the related information is selected, and stores the value.

[Sub image selection]
FIG. 8 is a flowchart showing the steps of the sub image selection process of the image reproduction device according to the embodiment of the present invention. This process is executed when the CPU 101 executes a control program stored in the RAM 103. With reference to FIG. 8, a process of lowering the priority order and selecting a sub-image for image data that has been displayed as a main image by the CPU 101 will be described.

  In step S801, CPU 101 substitutes 0 for counter n. The counter n is a counter for the CPU 101 to execute a series of processes from step S802 to step S812, which will be described later, for each sub-image to be displayed. In step S802, the CPU 101 determines whether the counter n is less than the number of sub-images scheduled to be displayed. In step S802, if the CPU 101 determines that the counter n is less than the number of sub-images to be displayed, the process proceeds to step S803.

  In step S802, if the CPU 101 determines that the counter n is equal to or greater than the number of sub-images to be displayed, it determines that a series of processing has been completed and ends the processing. In step S803, the CPU 101 reads the related information 701 having the selection number n from the list of related information in the RAM 103. In step S804, the CPU 101 reads the value 703 of the main image of the related information with the selection number n from the list of related information in the RAM 103. In step S805, the CPU 101 searches for image data whose related information in S803 has a value of S804 using the DB shown in FIGS.

  In step S806, the CPU 101 refers to the main image displayed list in the RAM 103. Then, the image data obtained by excluding the image data that has become the main image from the image data of the search result in S805 is counted. In step S807, the CPU 101 determines whether or not the count result in S806 is zero. If the CPU 101 determines that the value is 0 in step S807, the process proceeds to step S808. If it is determined in step S807 that the CPU 101 is not 0, the process proceeds to step S809. In step S808, the CPU 101 randomly selects one image data as the sub image from the image data excluding the current main image from the image data searched in S805, and stores it in the RAM 103.

  In step S809, the CPU 101 determines whether or not the count result in S806 is 1. If the CPU 101 determines 1 in step S809, the process proceeds to step S810. If it is determined in step S809 that the CPU 101 is not 1, the process proceeds to step S811. In step S810, the CPU 101 selects, as the sub image, the only image data that has never become the main image from the image data searched in S805, and stores it in the RAM 103. In step S811, the CPU 101 randomly selects one image data as the sub image from the image data that has not become the main image from the image data searched in S805, and stores it in the RAM 103. In step S812, the CPU 101 increments the counter n and returns to S802.

[Example of sub-image according to main image]
In the case where the main image is image data during a walk taken with Taro and Hanako in Yokohama on July 10, 2013, what sub-images will be displayed will be described. When the image data is the image data 421 in FIG. 4 and the list of related information shown in FIG. 7 is created as a result of the related information selection processing in FIG. There are four images: the image that appears in the picture, the image of Mr. Taro as the subject, the images taken in July of various years, and the image of the walk.

  A description will be given of what sub-image is displayed when the main image is image data during a trip taken on April 7, 2013 in Katsura Beach, Kochi. If the image data is the image data 419 in FIG. 4 and the list of related information shown in FIG. 10 is created as a result of the related information selection processing in FIG. There are four images: an image taken in the moon, an image taken in the sea, an image taken in the landscape, and an image taken in Shikoku.

  Although the case where the country is mainly Japan has been described as the related information corresponding to the shooting location, it may be changed depending on the country. Although the related information 316 is expressed as a prefecture, it may be a section of a region that is finer than the regional division. If the country is the United States, state.

  If the relevant information is local, and the country is the United States, New England, Mid Atlantic, East North Central, West North Central, South Atlantic, East South Central, West South Central, Mountain, Pacific become.

  In the present embodiment, the case where there are four sub-images has been described as an example. However, the sub-image may be variable or set by the user according to the screen size of the output device.

  As described above, according to the image reproducing device of the present embodiment, images to be managed that match individual related information are grouped, and the optimum relationship is determined according to the number of groups and the number of images in the group including the main image. By selecting information, it is possible to efficiently browse images related to a main image of interest with different directions.

[Example 2]
In the first embodiment, every time the main image changes, the number of sub-images is given as an example of reselecting related information, but in this embodiment, the related information of the vertical sub-image is not changed, A case where only the related information of the sub-image in the horizontal direction is selected again will be described. In the first embodiment, the image reproducing apparatus that operates on a smartphone has been described. In the present embodiment, an image reproducing apparatus that operates on a digital still camera will be described.

[Digital still camera operating as an image playback device]
A digital still camera that operates as the image control apparatus of the present embodiment will be described. In order to clarify the point of the invention, the description will focus on the image reproduction function for reproducing the image data in the storage medium, not the photographing function of the digital still camera.

  FIG. 11 is a block diagram showing a digital still camera that operates as an image playback device according to an embodiment of the present invention. By executing a predetermined control program in the digital still camera, an image playback process described below is realized and functions as an image playback device. FIG. 12 is a diagram showing a digital still camera operating as an image playback device according to an embodiment of the present invention as seen from the back side. The function of the digital still camera as an image playback device will be described with reference to FIGS.

  In FIG. 11, reference numeral 1100 denotes an imaging apparatus, which converts an optical image into electrical image data using a solid-state imaging device, and generates image data. A CPU 1101 controls the entire digital still camera. A flash ROM 1102 stores an operation processing procedure of the CPU 1101 (for example, a program such as processing when the digital still camera is turned on or basic input / output processing). The flash ROM stores the same data as the DB shown in FIGS. 4 and 5 of the first embodiment. When image data is generated by the imaging apparatus 1100, the CPU 1101 generates information from the attribute information of the image data and the image analysis result, and stores it in the DB.

  Reference numeral 1103 denotes a RAM that functions as a main memory of the CPU 1101. Various programs including a control program for realizing processing to be described later are loaded into the RAM 1103 from the flash ROM 1102 or the like and executed by the CPU 1101. The RAM 1103 provides a work area when the CPU 1101 executes various processes. Reference numeral 1104 denotes a display 1201 shown in FIG. 12, which performs various displays under the control of the CPU 1101. For example, the data stored in the storage medium is displayed.

  Reference numeral 1105 denotes an input device such as a touch panel and buttons for performing various operations shown in FIG. Detailed description of the buttons will be described later. Reference numeral 1106 denotes a storage medium mounting unit (media drive) which mounts a detachable storage medium and stores data or reads stored data.

A network interface 1107 is connected to the computer network 1110 via a communication line 1109. Data is transmitted and received from the server and personal computer through this network interface. Also, when acquiring the location, the CPU 1101 is connected to the computer network 1110 via the communication line 1109 and communicates with a server that calculates the location using a satellite positioning system (hereinafter referred to as GPS). get.
Reference numeral 1108 denotes a system bus (consisting of an address bus, a data bus, and a control bus) for connecting the units described above.

[Digital still camera user interface]
Details of various buttons and the user interface of the digital still camera operating as the image reproducing apparatus according to the present embodiment will be described with reference to FIG. The digital still camera described in this embodiment has three types of modes, and operates according to each mode. The first is a “shooting mode” in which shooting is performed, the second is a “playback mode” in which image data in the storage medium is displayed, and the third is a “setting mode” in which various settings are made. The mode mainly described in the present embodiment is “reproduction mode”. Regarding all the buttons other than the power button 1209, the operation when the digital still camera is turned on will be described.

  1200 is a release button. When the user presses down, the CPU 1101 determines that there is a shooting instruction from the user, switches to the shooting mode, and performs shooting processing. Reference numeral 1201 denotes a display, and the display function is as described in 1104. 1201 has a touch panel function as well as a display function, and fulfills the function of the input device 1105. When the user presses any point on the screen with a finger, the CPU 1101 determines that there is an input instruction from the user, determines the operation content from the pressed position, and performs various processes such as display update .

  A power button 1202 switches the power ON / OFF. When the user presses the digital still camera in a state where the power is not turned on, the CPU 1101 determines that there is an instruction for power input from the user, and turns on the power. When the user presses the button while the power is on, the CPU 1101 determines that the user has instructed to turn off the power and turns off the power. Reference numeral 1203 denotes a shooting mode button. When the user presses the shooting mode button 1203, the CPU 1101 determines that there is an instruction to switch to the shooting mode from the user, switches to the shooting mode, prepares for shooting by the imaging device 1100, and displays the formed image on the display 1201. To display.

  1204 is a playback button. When the user presses the play button 1204, the CPU 1101 determines that there is an instruction to switch to the play mode from the user, reads image data from the storage medium, and displays an image on the display 1201. The display 1201 in FIG. 12 shows a state where a playback screen is displayed. 1205 is a setting button. When the user presses a setting button 1205, the CPU 1101 determines that the user has issued an instruction to switch to the setting mode, and displays a setting screen on the display 1201.

  Reference numeral 1206 denotes a main image. The CPU 1101 identifies image data from the main image information stored in the RAM 1103, reads the corresponding image data from the storage medium, and displays it. Reference numerals 1207, 1208, 1209, and 1210 denote sub-images. After the CPU 1101 performs a sub-image selection process to be described later, the image data in the determined storage medium is read and displayed. In the present embodiment, image data for the sub-images 1207 and 1208 is determined by sub-image selection processing that will be described later with reference to FIG. On the other hand, in 1209 and 1210, image data is determined by the same sub-image selection processing as in the first embodiment.

[Image playback processing]
FIG. 13 is a flowchart showing image reproduction processing steps of the image reproduction apparatus according to the embodiment of the present invention. This process is executed by the CPU 1101 executing a control program stored in the RAM 1103. With reference to FIG. 13, a case will be described in which the CPU 1101 does not change the related information of the vertical sub-image and reselects only the related information of the horizontal sub-image. For the same processing as that described in FIG. 9, the corresponding steps in FIG. 9 are described, and description thereof is omitted.

  In step S1301, the CPU 1101 performs a sub-image selection process in the vertical direction. The selection process in S1301 will be described later with reference to FIG. In step S1302, CPU 1101 performs a sub-image selection process in the left-right direction. The selection process in S1302 is almost the same as in S903. The difference is that if the number of sub-images selected in the sub-image selection process in S1301 is less than 2, the number of sub-images is increased in S1302, and four sub-images are selected in the processes of S1301 and S1302. It is a point to adjust to. In step S1303, CPU 1101 stores related information corresponding to the sub-image stored in S908 in RAM 1103 as related information designated by the user.

[Sub image selection]
FIG. 14 is a flowchart showing the steps of the sub image selection process of the image reproduction device according to the embodiment of the present invention. In this embodiment, this corresponds to the vertical sub-image selection process. This process is executed by the CPU 1101 executing a control program stored in the RAM 1103. A process in which the CPU 1101 selects a sub-image in the vertical direction based on the related information designated by the user will be described with reference to FIG.

  In step S1401, CPU 1101 determines whether or not related information designated by the user is stored in RAM 1103. The related information designated by the user is related information stored in S1303. If the CPU 1101 determines in step S1401 that there is related information specified by the user, the process proceeds to step S1402. If the CPU 1101 determines in step S1401 that there is no related information designated by the user, the process proceeds to step S1409. In step S1402, CPU 1101 reads the value of the main image relating to related information designated by the user.

  In step S1403, the CPU 1101 uses the DB in the flash ROM 1102 to search for image data whose related information specified by the user is the value of S1402. In step S1404, CPU 1101 refers to the main image displayed list in RAM 1103. Then, the image data obtained by excluding the image data that has become the main image from the image data of the search result of S1403 is counted. In step S1405, CPU 1101 determines whether the count result in S1404 is 2 or more. If the CPU 1101 determines that the number is 2 or more in step S1405, the process proceeds to step S1406. If it is determined in step S1405 that the CPU 1101 is less than 2, the process proceeds to step S1407.

  In step S1406, CPU 1101 randomly selects two image data as sub-images from the image data excluding the current main image from the image data searched in S1403, and stores them in RAM 1103. In step S1407, CPU 1101 determines whether or not the count result in S1404 is 1. If the CPU 1101 determines 1 in step S1407, the process proceeds to step S1408. If it is determined in step S1408 that the CPU 1101 is not 1, the process proceeds to step S1409. In step S1408, CPU 1101 selects, as the sub image, the only image data that has never become the main image from the image data searched in S1403, and stores it in RAM 1103.

  In step S1409, CPU 1101 sets the number of sub-images to be zero, assuming that there are no sub-images that can be selected. In step S1410, CPU 1101 stores in RAM 1103 the number of sub-images selected in a series of sub-image selection processes from S1401 to S1409.

  In this embodiment, if the number of sub-image selections is less than 2 in S1301, the number of sub-image selections is increased in S1302. Image data searched by value may be selected as a sub-image. If all April images for various years are displayed, the May image is displayed. When all images taken in Shikoku are displayed, Kyushu images are displayed. When Hanako displays all the images of the subject, Taro displays the images of the subject. If the next value is not uniquely determined, the selected number of sub-images may be increased in S1302 as described above.

  In the present embodiment, the relevance of the image in the vertical direction is not changed, but only the relevance of the image in the left and right direction is changed as an example. In addition, it is possible not to change the relevance in only one of the four directions.

  As described above, according to the image reproduction device of the present embodiment, the relevance of the images in the vertical direction does not change, and only the relevance of the images in the left and right directions changes. Can be viewed, and can be switched when you want to see related images in different directions.

  As described above, according to the image playback device of this embodiment, not only smartphones but also digital still cameras can efficiently browse images related to a main image of interest in a plurality of different directions. Will be able to.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

101 CPU
102 ROM
103 RAM

Claims (9)

Attribute information analysis means for analyzing attribute information of the main image, related information selection means for selecting a plurality of related information according to each attribute information, and images to be managed that match each related information are grouped, and the number of groups Related information selecting means for selecting which related information is adopted according to the number of images in the group including the main image,
Image search means for searching for an image that matches the related information selected in the related information selection means;
A sub-image selection unit that uses the image searched by the image search unit as a sub-image;
An image reproduction apparatus comprising: a sub-image display unit that displays an image selected by the sub-image selection unit. 2. The image reproducing device according to claim 1, wherein when the user selects the sub-image, the sub-image becomes a next main image and the series of sub-image display processes are performed. 3. The image reproduction according to claim 2, further comprising: main image storage means for storing an image displayed as a main image; and display priority order determination means for lowering the priority order for displaying the sub image as an image stored in the main image storage means. apparatus. 2. The image reproduction apparatus according to claim 1, wherein the attribute information is information relating to any of shooting date / time, shooting location, subject, and scene information. When the attribute information is a shooting date and time, the related information is any one of a season, a month, a day of the week, a time zone constituting a day such as morning / noon / evening / night or am / pm. The image reproducing device according to claim 1. 2. The image according to claim 1, wherein when the attribute information is a shooting location, the related information is any one of hierarchical region information, landmark information, and information indicating a region characteristic. Playback device. 2. The attribute information according to claim 1, wherein when the attribute information is a subject, the related information is any one of information indicating a type of the subject, information indicating a relationship with a user, and an individual or individual name. Image playback device. 2. The image reproduction according to claim 1, wherein when the attribute information is scene information, the related information is any one of information specified by a user at the time of shooting, event information, and information of a scene analysis result. apparatus. When a user selects one sub-image, the sub-image becomes the next main image, and related information holding means for holding first related information corresponding to the sub-image,
First sub-image display means for displaying a sub-image that matches the first related information on the top and bottom or the left and right of the main image;
A second sub-image that matches the second related information different from the first related information selected by the series of related information selecting means is displayed at a position different from the sub-image that matches the first related information. 3. The image reproduction apparatus according to claim 2, further comprising: a sub-image display unit.