JP6089892B2 - Content acquisition apparatus, information processing apparatus, content management method, and content management program - Google Patents

Description

  The present invention relates to a content acquisition device, an information processing device, a content management method, and a content management program.

  Devices with a camera function that can take still images and moving images are widespread. The user can easily take an image using these devices. As a result, the number of images stored in the device becomes large, and it becomes difficult to extract images that the user likes. Therefore, a method for automatically arranging images has been proposed (see, for example, Patent Documents 1 to 3).

JP-A-2005-39486 JP 2007-226555 A JP 2010-187320 A

  However, even if a method for automatically organizing content such as images and sounds is provided, when the user sets content classification information, it is troublesome for the user.

  Therefore, an object of one aspect is to facilitate the management of content.

In one proposal, the acquisition unit that acquires content, the type of touch operation on the touch screen of the displayed content among the acquired content , at least one of the number of touch operations, the scaling rate, the angle, and the priority, A recording unit that records additional information including the content in correspondence with the content, and by referring to the recording unit, the type of touch operation included in the additional information, the number of touch operations, the scaling rate, the angle, and the priority There is provided a content acquisition apparatus comprising: an extraction unit that classifies the content according to at least one of them and extracts content of a specific touch operation classification.

  According to one aspect, content management can be facilitated.

It is the figure which showed the example of whole structure of the content acquisition apparatus concerning one Embodiment. It is the flowchart which showed the additional information provision process concerning one Embodiment. It is an example of a recording of additional information for content operation according to an embodiment. It is the figure which showed the example of recording of the additional information to the recording part concerning one Embodiment. It is the flowchart which showed the content extraction process concerning one Embodiment. It is an example of a recording of additional information for content operation according to an embodiment. It is the figure which showed the example of whole structure of the information processing apparatus concerning one Embodiment. It is an example of a recording of additional information for content operation according to an embodiment. It is a hardware block diagram of the content acquisition apparatus and information processing apparatus concerning one Embodiment. It is the flowchart which showed the classification information provision process example. It is the flowchart which showed the classification information provision process example.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, the duplicate description is abbreviate | omitted by attaching | subjecting the same code | symbol.

<Introduction>
Users often carry devices with camera functions and easily take images. As a result, the number of images stored in the device becomes large, and it becomes difficult to extract images that the user likes. For example, when photographing a child race of a child with a camera at an athletic meet, the user may shoot dozens of images from the start to the goal in the continuous shooting mode because the moving speed of the child as a subject is fast. The user wants to select and save and print only his / her favorite photos. On the other hand, since the number of captured images is large, the user may end up as it is, assuming that it takes time to check and sort the images one by one.

(Classification by face recognition)
Recently, the face recognition function of still images and moving images has been developed, and if a person's face is recognized and registered, software that realizes a function that can sort images showing that person is increasing. . However, with this sorting method, it is difficult to sort the best shots in the case of the above-mentioned child photographs because they are all the same person.

(Classification by keyword)
There is also a method of classifying an image by a keyword by adding a keyword or the like to the image. A method for classifying images by keywords will be briefly described with reference to FIG. In this case, the control unit of the device first displays an image thumbnail on the device (step S10). Next, the control unit determines whether there is a sorting target in the displayed image (step S11). When it is determined that there is a sorting target, the control unit displays an image of a target clicked by the user (step S12). Next, the control unit determines whether to sort by keyword (step S13). If it is determined to sort by keyword, the control unit displays a keyword input image (step S14) and performs keyword input according to the user operation (step S15). Next, the control unit assigns a keyword to the displayed content (here, an image) (step S16), returns to step S11, and determines whether there is a next sorting target.

  If it is determined in step S11 that there is no sorting target, or if it is determined in step S13 that no sorting by keyword is performed, the control unit ends this process.

In the case of the classification based on the keywords described above, the user has to input the keyword from the keyword input screen, which is troublesome for the user.
(Classification based on conversation information and gaze time)
There is also a method of classifying images using conversation information recorded while browsing images and gaze time. A method for classifying images according to conversation information and gaze time will be briefly described with reference to FIG. In this case, the control unit of the device first displays an image thumbnail on the device (step S20). Next, the control unit determines whether there is a sorting target in the displayed image (step S21). When it is determined that there is a sorting target, the control unit displays an image of a target clicked by the user (step S22).

  Next, the control unit acquires conversation information while displaying an image, and measures the gaze time (step S23). Next, the control unit adds conversation information and a gaze time to the displayed content (here, an image) (step S24).

  In addition, when it determines with there being no classification object in step S21, a control part complete | finishes this process.

  In the case of the classification based on the conversation information and the gaze time described above, since the conversation is not performed when one person is browsing, the conversation information cannot be obtained and the classification based on the conversation information cannot be performed. In addition, when an image is viewed in a small space such as a small store where an unspecified number of people are crowded, conversation information of surrounding people who are not viewers may be captured. In this case, the degree of attention may be erroneously determined, and the content may be sorted into a classification different from the original degree of interest, resulting in a lack of classification accuracy.

  Therefore, in the embodiment described below, a data classification method using the operation information of the viewer is proposed. In other words, in the following embodiment, additional information indicating an operation on a viewer's image data when browsing captured image data is associated with the corresponding image data (for example, Exif) or the corresponding image data. Record operation information (tagging). For example, the operation of the viewer on the image data includes a touch operation on the image data. Thus, it is possible to automatically extract image data of a specific classification automatically classified according to the type of operation of the viewer from a large amount of recorded data.

  In the following embodiment, automatic classification of an image and extraction of an image of a specific classification will be described. In a modification of the embodiment, automatic classification of audio and extraction of a voice of a specific classification will be described. . However, the content according to the present embodiment is not limited to this, and can be applied to, for example, still image data, moving image data, audio data, and data obtained by combining these.

[Overall configuration of the device]
First, a content acquisition apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram illustrating an overall configuration of a content acquisition apparatus according to an embodiment. The content acquisition device 10 according to the embodiment includes, for example, a digital camera, a recording device, a mobile phone with a camera function or a recording function, a tablet terminal with a camera function or a recording function, a smartphone with a camera function or a recording function, a recording A music player with a function may be used. The content acquisition apparatus 10 has at least one of a function of capturing content (image) and a function of recording content (sound) as means for acquiring content.

  The content acquisition apparatus 10 includes a lens 11, an image processing unit 12, a recording unit 13, a microphone 14, an audio processing unit 15, a display unit 16, a control unit 17, an operation unit 18, and an extraction unit 19.

  For example, the lens 11 is configured as an optical system together with a shutter (not shown), and condenses light. The light condensed through the lens 11 is photoelectrically converted by a CMOS image sensor (not shown). The photoelectrically converted analog electrical signal is converted into a digital signal, and the digital signal is converted into a color difference signal (image signal) and output to the image processing unit 12.

  The image processing unit 12 performs JPEG encoding processing or the like on the input image signal, and outputs the encoded image data to the recording unit 13.

  The microphone 14 collects sound and outputs the obtained audio signal to the audio processing unit 15. The lens 11 and the microphone 14 are an example of an acquisition unit that acquires content.

  The audio processing unit 15 outputs audio data obtained by performing predetermined processing such as amplification processing on the audio signal to the recording unit 13.

  The display unit 16 displays an image or the like indicating an image captured by the camera function of the content acquisition device 10 or a sound recorded by the recording function. A touch panel may be stacked on the display unit 16. In that case, the display unit 16 and the operation unit 18 are integrated, and the user's contact operation on the screen of the display unit 16 becomes an input operation using the operation unit 18. However, the display unit 16 and the operation unit 18 may be separate, and the operation unit 18 may be a keyboard or a mouse.

  The control unit 17 acquires additional information indicating that an operation has been performed on an image displayed on the display unit 16 among images captured using the lens 11. The image is recorded in the recording unit 13 in association with the displayed image. The recording unit 13 records additional information in association with image or audio data.

  The extraction unit 19 classifies the images according to the additional information recorded in association with the images, and extracts an image with a specific classification.

  For example, when an image captured by the camera function of the content acquisition apparatus 10 is immediately displayed on the display unit 16 and viewed by the user, the image 1 is captured → the image 1 is displayed → the image 2 is captured → the image 2 is displayed → Display processing is performed as follows. The display unit 16 may display thumbnails of a plurality of images recorded in the recording unit 13.

  The user may perform a touch operation such as a touch operation or an enlargement operation on the displayed image. The control unit 17 recognizes the contact operation on the image as a predetermined input operation, and executes necessary processing (for example, enlarged display) for the input operation. In addition, the control unit 17 acquires this contact operation as additional information, and the recording unit 13 records the additional information in association with the image subjected to the contact operation. That is, in this embodiment, when the user performs a contact operation on the displayed image while browsing the image, additional information indicating that the operation has been performed is recorded in a linked manner with the image. The added information is used for image classification (sorting). According to this, user operation at the time of image browsing is used for sorting images. Specifically, when the image is viewed, how much the contact operation is performed by the user who is the viewer correlates with the degree of attention indicating how much the user is interested in the image. The degree of attention to is estimated. As a result, it is possible to extract content that matches the degree of attention of the user. Therefore, manual data sorting by the user is unnecessary, and a large amount of images can be easily managed while saving the user from having to set classification conditions such as keywords in advance.

Here, as an operation performed on the displayed image, for example, the following operation can be given as an example.
1. 1. Touch operation to the displayed image and the number of touch operations 2. An operation for enlarging the displayed image. 3. Operation for rotating the displayed image 4. An operation for performing some processing on the displayed image, for example, processing such as pressing a stamp or adding a balloon to the image using software (application program). Specific touch operations customized by the user such as tracing with a finger, etc. However, operations recorded as additional information among operations performed on the displayed image are not limited to the above-described operation examples. For a plurality of operations on the same image, each of the plurality of operations is recorded as additional information in association with the image.

[Additional information assignment processing]
Next, the additional information addition process executed in the content acquisition apparatus according to the present embodiment will be described with reference to FIG. FIG. 2 is a flowchart showing additional information provision processing according to an embodiment.

  For example, when a user browses a picture of a child's athletic meet alone, the control unit 17 causes the display unit 16 to display an image or thumbnail of the child's athletic meet in accordance with the user's browsing operation (step S30). Next, the control unit 17 determines whether an operation to be classified (classified), that is, an operation to add additional information has been performed on any of the displayed contents (here, images). (Step S31). As a specific example, the control unit 17 obtains information indicating that a desired position on the touch screen has been operated based on an electrical signal from a touch panel stacked on the display unit 16. Based on this information, the control unit 17 determines whether an operation on the image to be sorted has been performed. The operation of the image is not limited to the contact operation on the screen on which the image is displayed. If the proximity operation on the screen on which the image is displayed can be detected, the detected proximity operation is also performed on the image. Included when is manipulated.

  When it is determined that an operation to be classified is performed, the recording unit 13 acquires an operation type (step S32), and records additional information including the operation type in association with the image (step S33). Here, there are two possible ways of recording additional information. One is a method in which the recording unit 13 records additional information in the recording area of the content acquisition apparatus 10 in association with image data. The other is a method in which the recording unit 13 records additional information in a recording area called Exif included in the image data itself. Exif is metadata embedded in image data or the like, in which position information (latitude and longitude) at the time of shooting and shooting conditions (such as exposure and shutter speed) are recorded. A special tool is required to view and rewrite information recorded in the Exif area.

  Whether the additional information is recorded in association with the image in the Exif area of each image data or the additional information is recorded in association with each image in the recording area of the content acquisition apparatus 10, the recording unit 13 displays the displayed content. Is recorded in association with the content. The image data itself and additional information may be recorded in association with each other, or an image ID for identifying an image and additional information may be recorded in association with each other.

  If it is determined in step S31 that the operation to be classified is not performed, the process proceeds to step S34 without adding additional information. In step S34, the control unit 17 determines whether to continue displaying images. If it is determined that the image display is to be continued, the processes of steps S31 to S34 are repeated. If it is determined in step S34 that the image display is not to be continued, this process ends.

  For example, FIG. 3A shows a result of an additional information adding process executed when an arbitrary image is touched by the user from a plurality of thumbnails displayed on the display unit 13. In the right frame of FIGS. 3A to 3E, the contents recorded in the Exif area provided in each image are shown in a simplified manner. Here, the content ID of the touched image is “0001”. In this case, the recording unit 14 uses the Exif of the image having the content ID “0001” as additional information of <Touch> and <cout> 2 indicating that the touch operation has been performed and the number of touches is two. Record in the area.

  The additional information may be recorded in association with the content ID “0001” in a recording area (database) of the content acquisition apparatus 10 other than the Exif area. In this case, as shown in FIG. 4, the content ID 401 is associated with “0001”, the operation type 402 is “touch”, and the number of times 403 is “ 2 "is recorded.

  Other examples of the additional information include an enlargement / reduction ratio 404, an angle 405, and a priority 406. However, the additional information is not limited to this. Further, additional information such as the enlargement / reduction ratio 404, the angle 405, and the priority 406 may not be recorded. The priority 406 may be automatically assigned based on the additional information, or may be given in response to a user request. For example, the user can increase the favorite degree (priority) of the touched image by increasing the number of touches.

  For example, FIG. 3B shows the result of the additional information adding process executed when the image displayed on the display unit 13 is zoomed. Here, the content ID of the zoomed image is “0002”. In this case, the recording unit 14 is provided with an additional information of <Touch> and <zoom> 5 indicating that the screen is touched and an operation of zooming 5 times is performed on an image having a content ID of “0002”. Recorded in the Exif area. Or, for example, as shown in FIG. 4, it is recorded in a desired recording area that the content ID 401 is associated with “0002”, the operation type 402 is “zoom”, and the enlargement / reduction ratio 404 is “250%”. May be.

  For example, FIG. 3C shows the result of the additional information adding process executed when the image displayed on the display unit 13 is rotated. Here, the content ID of the rotated image is “0003”. In this case, the recording unit 14 is provided with the additional information of <Touch> and <rotation> 90 indicating that the screen is touched and an operation of rotating 90 degrees is performed on the image having the content ID “0003”. Record in the Exif area. Or, for example, as shown in FIG. 4, it is recorded in the desired recording area that the content ID 401 is associated with “0003”, the operation type 402 is “rotation”, and the angle 405 is “90 degrees”. Also good.

  For example, FIG. 3D shows additional information executed when an image displayed on the display unit 13 is subjected to processing such as pressing a stamp or adding a balloon using software (application). The result of the grant process is shown. In this case, the recording unit 14 includes the Exif area provided in the image having the content ID of the image, such as <Touch> and <edit> 25 indicating that the screen is touched and 25% of the image has been processed, or Record in the desired recording area.

  For example, FIG. 3 (e) shows the result of the additional information adding process executed when a specific touch operation such as tracing with a finger on the screen on which the image is displayed is performed. In this case, for example, if the operation on the screen is defined in advance, such as making an image displayed when the screen is traced round, the sorting can be customized for each user. In this case, since the recording unit 14 is touched so that the screen is rounded, the additional information of <Touch> and <custom1> delete indicating that the displayed image is a deletion candidate has the image content ID. Recording is performed in an Exif area provided in the image or a desired recording area. For example, even if the image is touched, an image that is clearly dark can be given additional information as a deletion candidate by performing an operation of tracing the screen round.

  As described above, in the content acquisition device 10 according to the present embodiment, the degree of attention of the user to the image is estimated based on the operation of the image by the user who is browsing the image. Then, the associated images are classified based on the additional information indicating the operation. As described above, the content acquisition apparatus 10 according to the present embodiment collects additional information using a user operation on an image at the time of browsing, and performs image classification described below based on the collected additional information. . Thereby, a large amount of images can be easily managed.

[Content extraction process]
Next, content extraction processing executed in the content acquisition apparatus according to the present embodiment will be described with reference to FIG. FIG. 5 is a flowchart illustrating content extraction processing according to an embodiment.

  The content extraction process shown in FIG. 5 is a process performed at the stage of outputting an image (for example, printing, redisplaying, etc.) based on additional information given at the time of browsing the image. First, the user selects what kind of photograph is to be printed. Here, the user designates printing of a photo from a group of images taken on the day of the athletic meet by selecting the day of the athletic meet.

  The extraction unit 19 determines whether there is content to be sorted (step S40). For example, as shown in FIG. 4, images with content IDs 401 of “0001”, “0002”, “0003”, “0004”.

  When it is determined that there is no sorting target, the display unit 16 displays, by default, thumbnails of the images taken on the day of the athletic meet (step S41), and the process ends. An example of the default display of thumbnails is to display an image according to the shooting time.

  On the other hand, when it is determined that there is a sorting target, the extraction unit 19 determines whether to display an image of a specific classification (step S42). When it is determined not to display an image of a specific category, the display unit 16 displays by default the thumbnails of the image group taken on the day of the athletic meet (step S41), and ends this process.

  On the other hand, when it is determined that an image of a specific classification is to be displayed, the extraction unit 19 determines whether additional information is given (step S43). When it is determined that the additional information is not given, the display unit 16 displays the thumbnails of the image group taken on the day of the athletic meet by default (step S41), and ends this process.

  On the other hand, when it is determined that additional information is given, the extraction unit 19 performs a classification process (step S44). Examples of the classification process include a case where the extraction unit 19 classifies images for each operation type 402 from the additional information recorded by the recording unit 13 and extracts content of a specific classification based on the classification result. As an example of extracting content of a specific classification, the extraction unit 19 extracts an image group in which the operation type 402 is classified as “touch” based on the classification result, or image data whose operation type 402 is a deletion candidate. After deletion, an image group in which the operation type 402 is classified as “touch” may be extracted. As another example of extracting content with a specific classification, the extraction unit 19 rearranges images in which the operation type 402 is classified as “touch” in the descending order of the number of touches 403, for example, five times or more. Images of the number of touches may be extracted.

  As another example of extracting content of a specific classification, the extraction unit 19 may extract an image classified into the operation type 402 having a higher priority 406 based on the priority 406 included in the additional information. For example, in the example of FIG. 4, the extraction unit 19 includes an image classified as “touch” with a priority “406”, an image classified as “zoom” with a priority “406”, and a priority 406. The images may be extracted in the order of the images classified as “rotation” of “3”. Further, the classification method is not limited to this, and various methods can be taken in consideration of the user's attention level. For example, the extraction unit 19 is one of a predetermined specific classification, a specific classification used for the last extraction, or a specific classification changed according to a user operation from the predetermined specific classification Content may be extracted according to one.

  Next, the display unit 16 displays the extracted thumbnail (step S45). Next, the extraction unit 19 determines whether to redisplay thumbnails (step S46). When it is determined to redisplay, the user changes the classification condition from the operation screen. The change of the classification condition includes, for example, changing the priority 406.

  Next, the extraction unit 19 performs a process of reclassifying the image based on the specific classification changed according to the user operation from the specific classification set in advance (step S47). While the user requests redisplay, the processing of step S45 to step S47 is repeated, and when the user does not request redisplay, this processing ends. Thereby, the user can print the image extracted by classification or reclassification.

  As described above, according to the content acquisition apparatus 10 according to the present embodiment, an image that has been operated during browsing is preferentially extracted. This is based on the viewpoint that how much the touch operation is performed on the image when browsing the image is correlated with the degree of attention indicating how much the user is interested in the image. Then, the content acquisition apparatus 10 according to the present embodiment records the above degree of attention of the viewer as additional information of the image, and classifies the image based on the additional information. In the present embodiment, an image of a specific classification is extracted from the classified image and displayed. As a result, content management can be easily performed. For example, it is possible to extract and print out only the image that the user touched during browsing. It is also possible to print out only the remaining image after deleting the image judged unnecessary from the touched image.

  Furthermore, according to the content acquisition apparatus 10 according to the present embodiment, the number of extracted images can be further narrowed down by repeating the processes in steps S45 to S47. For example, it is assumed that the extraction unit 19 extracts a touched image in a process of extracting an image of a specific classification. When the recording unit has recorded the content of the comment inserted at the time of processing the image as additional information, the extraction unit 19 further extracts an image with a favorite comment from the touched image that has been extracted. By doing so, the image can be further narrowed down. As described above, in the content acquisition apparatus 10 according to the present embodiment, it is possible to change the conditions for specific classification of images to be extracted based on the combination of additional information and the priority of the additional information. According to this, compared with the case where the classification conditions are fixed, there is a degree of freedom in classification, and the user's preference and interest tendency are easily reflected in the classification conditions. This makes it possible to extract content that matches the degree of user attention.

  For example, in FIG. 6, three operations of touch, zoom, and rotation are performed on the image whose content ID 401 is “0001”. Therefore, when the operation type 402 for the content is touch and an image is extracted using the number of touches 403 as a specific classification condition, an image with a content ID 401 of “0001” and an image of “0002” are extracted. . However, when the operation type 402 extracts an image with touch, zoom, and rotation as specific classification conditions, the image with the content ID 401 “0001” is extracted before the image with the “0002”, and the content ID 401 There may be a case where an image of “0002” is not extracted. Thus, by giving a degree of freedom to the setting of a specific classification and making it changeable by a user operation, it is possible to extract content that matches the degree of attention of the user.

[Example of overall configuration of information processing apparatus]
In the above, an image is captured by the content acquisition device 10 illustrated in FIG. 1, and the captured image is classified and managed using each function in the content acquisition device 10. However, the additional information adding process and the content extracting process according to the present embodiment can be executed not in the content acquisition apparatus 10 but in an information processing apparatus such as a PC. Hereinafter, an information processing apparatus according to an embodiment will be described with reference to FIG. FIG. 7 is a diagram illustrating an overall configuration of an information processing apparatus according to an embodiment.

  The information processing apparatus 20 according to an embodiment may be, for example, a PC, a tablet terminal, a smartphone, a mobile phone, an information terminal PDA, or the like. The information processing apparatus 20 includes a content input unit 21 as means for inputting content. The other functions (recording unit 26, display unit 27, control unit 28, operation unit 29, and extraction unit 30) of the information processing apparatus 20 are the same as those of the content acquisition apparatus 10 (recording unit 13, display unit 16, Since it is the same as the control part 17, the operation part 18, and the extraction part 19), description is abbreviate | omitted here.

  The content input unit 21 includes a USB (Universal Serial Bus) 22, a network I / F (interface) 23, a camera 24, and a microphone 25. However, the content input unit 21 may not have the camera 24 and the microphone 25 as long as the content input unit 21 has the USB 22 or the network I / F 23.

  The content input unit 21 may input image data from a medium connected to the USB 22, or may input image data from the content acquisition device 10 or other device connected via the network I / F 23. . Of course, an image captured by the camera 24 or a sound acquired by the microphone 25 may be input. The input content is recorded by the recording unit 13. The data is recorded in a recording area such as an HDD or a RAM provided in the information processing apparatus 20.

  Also in the information processing apparatus 20 according to the present embodiment, the control unit 28 acquires additional information indicating that an operation has been performed on the content displayed on the display unit 27 among the input content. Then, the recording unit 26 records the additional information in association with the content. Further, the extraction unit 30 classifies the content according to the additional information recorded in association with the content, and extracts the content with a specific classification. Also with the information processing apparatus 20 according to the present embodiment, the management of content can be facilitated as in the case of the content acquisition apparatus 10.

[Audio example]
In the above embodiment, the additional information adding process and the content extracting process are described by taking an image as the content as an example, and the image data is classified. However, in the content acquisition device 10 according to the present embodiment and the information processing device 20 according to the present embodiment, the audio data can be similarly classified by executing the additional information addition process and the content extraction process for the sound. .

  The additional information adding process and the content extracting process for the audio data will be described with reference to FIG. The audio data can be acquired from the microphone 14 or the microphone 25, or can be input from a medium connected to the USB 22 or a device connected via the network I / F 23. In the following, the content acquisition device 10 such as a music player will be described as an example, but the same can be executed in the information processing device 20 as well.

  In FIG. 8, the audio data is displayed as a thumbnail on the display unit 16. If the audio data is a song stored in a CD jacket, the display unit 16 displays an image of the jacket and an icon M1 indicating a content ID “0001”. If the audio data is recorded information, icons M2 and M3 indicating a recording mark, recording date and time, and content IDs “0002” and “0003” are displayed.

  When any of the displayed contents is operated, the control unit 17 acquires additional information indicating that the operation has been performed on the operated content. The recording unit 13 records the additional information in association with the content. For example, the user selects a content to be reproduced by touching or clicking the content. As a result, as shown on the right side of FIG. 8, the recording unit 13 records “selection operation” of the operation type 402 and “1” of the number of times 403 as additional information of the content whose content ID 401 is “0001”. . Further, the volume set by the user when the selected content is reproduced may be recorded as additional information. In FIG. 8, information with the volume 407 being “medium” is recorded as additional information of the content with the content ID 401 being “0001”.

  The extraction unit 19 classifies the content according to the additional information recorded in association with the content, and extracts the content with a specific classification. For example, the extraction unit 19 may display thumbnails by rearranging the display order of the contents in descending order of the number of reproductions. When the number of times of reproduction is the same, it may be estimated that the degree of attention is high when the volume during reproduction is high, and thumbnails may be displayed by rearranging the display order of the contents in descending order of volume. In the lower left of FIG. 8, the contents are displayed in the order of icons M2, M3, and M1 as a result of extracting and rearranging the contents with reference to the additional information in the order of number of times 403 → volume 407. At this time, the extraction unit 19 may not extract content that has not been reproduced at all. In this case, an icon with the number of reproduction times “0” is not displayed. In addition, the extraction unit 19 can employ various classification methods and extraction methods.

[Example of moving image]
In the content acquisition apparatus 10 and the information processing apparatus 20 according to the present embodiment, the additional information addition process and the content extraction process can be executed similarly for the moving image. In this case, for example, the recording units 13 and 26 may record not only the number of times that the moving image is operated (the number of selections) but also the degree of processing such as editing (trimming) of the moving image as additional information. The extraction units 19 and 30 may increase the priority if the number of operations of the moving image or the degree of processing is high, and may be a target to be extracted preferentially when extracting a moving image of a specific classification.

  As described above, in the content acquisition device 10 and the information processing device 20 according to the present embodiment, it is estimated that the user viewing the image has operated the image, and that the user is paying attention to the image. To do. Therefore, additional information indicating that the user viewing the image has operated the image is recorded. The additional information is used for automatic classification of images, and images of a specific classification are extracted. As a result, it is possible to automatically classify a rough image from a large number of images as intended by the user (viewer or operator). In addition, with respect to contents such as audio other than images, it is possible to automatically extract contents presumed to be noticed by the user by a similar method.

  According to this, since an image can be displayed at the time of shooting or after shooting, and the image can be sorted based on the degree of attention to the image at the time of browsing, sorting according to the user's subjectivity and situation becomes possible. Therefore, it is possible to reduce the possibility of performing sorting according to the erroneous attention level. Further, image classification is performed based on additional information, and the user does not need to set classification conditions. For this reason, the user is not burdened with troublesome time for content classification. Even in a situation where conversation information cannot be obtained, it is possible to classify and extract content with high accuracy.

(Hardware configuration example)
Finally, a hardware configuration example of the content acquisition device 10 and the information processing device 20 will be briefly described. FIG. 9 is a diagram illustrating a hardware configuration example of the content acquisition apparatus 10 and the information processing apparatus 20 according to the present embodiment.

  As shown in FIG. 9, the content acquisition device 10 and the information processing device 20 include an input device 101, a display device 102, an external I / F 103, a RAM (Random Access Memory) 104, a ROM (Read Only Memory) 105, a CPU (Central Processing Unit) 106, communication I / F 107, and HDD (Hard Disk Drive) 108 are connected to each other via bus B.

  The input device 101 is used when inputting each operation to the content acquisition device 10 and the information processing device 20. The input device 101 may be a keyboard or a mouse. The display device 102 includes a display and displays images and thumbnails taken by the user.

  The communication I / F 107 is an interface that connects the content acquisition apparatus 10 and the information processing apparatus 20 to a network. Thereby, the content acquisition apparatus 10 and the information processing apparatus 20 can transmit and receive image data and audio data via the network.

  The HDD 108 is a non-volatile recording device that stores programs and data. The stored programs and data include basic software that controls the entire apparatus and application software that provides various functions. The HDD 108 stores a program executed by the CPU 106 for performing the additional information adding process and the content extracting process of the above embodiment. The HDD 108 may store image data and audio data.

  The external I / F 103 is an interface with an external device. The external device includes a recording medium 103a. The information processing apparatus 20 can read and / or write to the recording medium 103 a via the external I / F 103. The recording medium 103a includes a USB (Universal Serial Bus memory), a CD (Compact Disk), a DVD (Digital Versatile Disk), and an SD memory card.

  The ROM 105 is a nonvolatile semiconductor memory (recording device), and stores programs and data such as BIOS (Basic Input / Output System), OS settings, and network settings that are executed at startup. The RAM 104 is a volatile semiconductor memory (recording device) that temporarily stores programs and data. The CPU 106 reads out a program or data from the recording device (for example, “HDD”, “ROM”, “RAM”, etc.) onto the RAM and executes processing, thereby executing control for implementing the entire device and mounting functions. Device.

  Each of the control units 17 and 28 and the extraction units 19 and 30 is realized by a process executed by the CPU 106 by a program installed in the HDD 108 or the like. For the recording area of content such as still image data, moving image data, and audio data, for example, a RAM 104, HDD 108, or a recording device (not shown) connected to the content acquisition device 10 and the information processing device 20 via a network is used. Can do.

  As described above, the content acquisition device, the information processing device, the content management method, and the content management program have been described with the embodiments. However, the present invention is not limited to the above embodiments, and various modifications and improvements can be made within the scope of the present invention. Is possible.

  For example, the content according to the present invention includes still image electronic data, moving image electronic data, audio electronic data, and the like. Further, the operated content according to the present invention may include an operation in a state where the content is not reproduced (for example, a folder move, an operation on a file, etc.). Further, the operated content according to the present invention may include an operation in a state of being reproduced (for example, an operation on a displayed image, an operation on a reproduced sound, etc.).

  In addition, when there are a plurality of the above-described embodiments and modifications, they can be combined within a consistent range. For example, the present invention can be applied to any operation of content according to the present invention, which is an operation of an image, sound, or a combination of image and sound.

Regarding the above description, the following items are further disclosed.
(Appendix 1)
An acquisition unit for acquiring content;
Among the acquired contents, a recording unit that records additional information indicating the operation on the operated content in association with the content;
An extraction unit that classifies the content according to the additional information recorded in association with the content, and extracts content of a specific classification;
A content acquisition apparatus comprising:
(Appendix 2)
The recording unit records the type of operation included in the additional information;
The content acquisition device according to supplementary note 1, wherein:
(Appendix 3)
The extraction unit is any one of a predetermined specific classification, a specific classification used for the last extraction, or a specific classification changed according to a user operation from the predetermined specific classification Extract content according to
The content acquisition device according to appendix 1 or 2, characterized in that:
(Appendix 4)
A display unit for displaying the extracted content;
The extraction unit changes the specific classification according to an operation performed by the user based on the display result of the content, and re-extracts the content of the changed specific classification.
The content acquisition device according to any one of appendices 1 to 3, characterized in that:
(Appendix 5)
A recording unit that records content, and records additional information indicating that the operation has been performed on the operated content among the recorded content in association with the content;
An extraction unit that classifies the content according to the additional information recorded in association with the content, and extracts content of a specific classification;
An information processing apparatus comprising:
(Appendix 6)
The recording unit records the type of operation included in the additional information;
The content acquisition apparatus according to supplementary note 5, wherein
(Appendix 7)
The extraction unit is any one of a predetermined specific classification, a specific classification used for the last extraction, or a specific classification changed according to a user operation from the predetermined specific classification Extract content according to
The content acquisition device according to appendix 5 or 6, characterized by the above.
(Appendix 8)
A display unit for displaying the extracted content;
The extraction unit changes the specific classification according to an operation performed by the user based on the display result of the content, and re-extracts the content of the changed specific classification.
The content acquisition device according to any one of appendices 5 to 7, characterized in that:
(Appendix 9)
Get content,
Of the acquired content, the additional information indicating that the operation has been performed on the operated content is recorded in association with the content,
Classifying the content according to the additional information recorded in association with the content, and extracting content of a specific classification;
A content management method in which processing is executed by a computer.
(Appendix 10)
In the recording process, the operation type is included in the additional information and recorded.
The content management method according to appendix 9, wherein the processing is executed by a computer.
(Appendix 11)
The extraction process is any one of a predetermined specific classification, a specific classification used for the last extraction, or a specific classification changed according to a user operation from the predetermined specific classification. Extract content according to
The content management method according to appendix 9 or 10, wherein the computer executes the process.
(Appendix 12)
Displaying the extracted content;
The extracting process changes the specific classification in accordance with an operation performed by a user based on the display result of the content, and re-extracts the content of the changed specific classification.
The content management method according to any one of appendices 9 to 11, wherein the process is executed by a computer.
(Appendix 13)
Get content,
Of the acquired content, the additional information indicating that the operation has been performed on the operated content is recorded in association with the content,
Classifying the content according to the additional information recorded in association with the content, and extracting content of a specific classification;
A content management program that causes a computer to execute processing.
(Appendix 14)
In the recording process, the operation type is included in the additional information and recorded.
Item 14. The content management program according to appendix 13, which causes a computer to execute processing.
(Appendix 15)
The extraction process is any one of a predetermined specific classification, a specific classification used for the last extraction, or a specific classification changed according to a user operation from the predetermined specific classification. Extract content according to
15. The content management program according to appendix 13 or 14, which causes a computer to execute processing.
(Appendix 16)
Displaying the extracted content;
The extracting process changes the specific classification in accordance with an operation performed by a user based on the display result of the content, and re-extracts the content of the changed specific classification.
The content management program according to any one of appendices 13 to 15, which causes a computer to execute processing.

  10: Content acquisition device, 13, 26: Recording unit, 16, 27: Display unit, 17, 28: Control unit, 18, 29: Operation unit, 19, 30: Extraction unit, 20: Information processing device, 21: Content Input section

Claims (6)

An acquisition unit for acquiring content;
Of the acquired content, additional information including the type of touch operation on the touch screen of the displayed content , and the number of touch operations, scaling ratio, angle, and priority is recorded in association with the content. A recording section;
Referring to the recording unit, the type of touch operation included in the additional information, the number of touch operations, scaling factor, and the angle and at least one of priority, according to classifying the content, the classification of particular touch operation An extractor for extracting the content of
A content acquisition apparatus comprising: The extraction unit is any one of a predetermined specific classification, a specific classification used for the last extraction, or a specific classification changed according to a user operation from the predetermined specific classification Extract content according to
The content acquisition apparatus according to claim 1 . A display unit for displaying the extracted content;
The extraction unit changes the specific classification according to an operation performed by the user based on the display result of the content, and re-extracts the content of the changed specific classification.
Content acquisition apparatus according to claim 1 or 2, characterized in that. The content is recorded , and the additional information including the type of touch operation on the touch screen of the displayed content and at least one of the number of touch operations, the enlargement / reduction ratio, the angle, and the priority is recorded among the recorded content. A recording unit for recording in correspondence with
Referring to the recording unit, the type of touch operation included in the additional information, the number of touch operations, scaling factor, and the angle and at least one of priority, according to classifying the content, the classification of particular touch operation An extractor for extracting the content of
An information processing apparatus comprising: The acquisition unit acquires the content,
Recording section, among the acquired content, corresponding with the type of the touch operation on the touch screen of the display content, the number of touch operations, scaling factor, and at least one of angle and priorities, the additional information including the said content Let me record,
The extraction unit refers to the recording unit , classifies the content according to the type of touch operation included in the additional information, and at least one of the number of touch operations, the enlargement / reduction ratio, the angle, and the priority . Extract the content of the classification,
A content management method in which processing is executed by a computer. Get content,
Of the acquired content, a recording unit that associates the additional information including the type of touch operation with respect to the touch screen of the displayed content and at least one of the number of touch operations, the scaling ratio, the angle, and the priority with the content recorded in,
Referring to the recording unit, the content is classified according to the type of touch operation included in the additional information and at least one of the number of touch operations, the scaling rate, the angle, and the priority. Extract,
A content management program that causes a computer to execute processing.

Images