KR101934108B1 - Method for clustering and sharing images, and system and application implementing the same method - Google Patents

Abstract

A method for image clustering of a server operated by at least one processor, the method comprising: extracting at least one image included in each web document and contents describing the image in a plurality of web documents; Determining one representative image representative of the redundant images and connecting image description contents corresponding to each of the redundant images to the representative image if there are redundant images in the plurality of images, Determining image information including an image description tag and a time stamp of the representative image from image description contents, mapping the image information and the representative image and storing the image information in an image repository, and requesting an image clustering request according to the clustering keyword If so, Extracting images associated with the clustering keyword, and clustering at least some of the extracted images based on the similarity of the extracted images to generate at least one cluster event.

Description

METHOD FOR CLUSTERING AND SHARING IMAGES, AND SYSTEM AND APPLICATION IMPLEMENTING THE SAME METHOD,

The present invention relates to image clustering.

Various images are posted on web pages such as news and blogs, and photographs taken at user terminals are quickly uploaded on various sharing platforms such as social media services. The web with such various images becomes a huge image storage, and the user can search the desired image.

However, the same images exist in duplicate in different web pages, or there are a large number of similar images captured at the same event. Therefore, in order to obtain a desired result among a large number of images that are not properly classified, the user must refine the search word and search for it. For this, the user must have prior knowledge of the person or event. Also, since it is difficult to reflect the latestness of the image, the search result may not change greatly depending on the search time. In recent years, there has been an attempt to classify and provide images by event, but the images are still listed and exposed in the event, so that there is no difference from the prior art in which images are exposed in chronological order. Therefore, there is a need for a method of classifying and sharing images in a meaningful manner to the user.

An object of the present invention is to classify various images of various sources according to attributes such as time, event, and character, to cluster images through tags for describing images, and to provide a system and an application for implementing the methods. In addition, a problem to be solved by the present invention is to provide a sharing method of generating content that can be shared based on clustering images, and a system and an application that implement the sharing method.

There is provided a method of image clustering of a server operated by at least one processor according to an embodiment, comprising the steps of: extracting at least one image included in each web document and contents describing the image in a plurality of web documents; Determining one representative image representative of the overlapping images if there are duplicated images among the images extracted from the web document and connecting image description contents corresponding to each of the overlapping images to the representative image; Determining image information including an image description tag and a time stamp of the representative image from a plurality of image description contents linked to the image, mapping the image information and the representative image and storing the image information in an image repository, If there is an image clustering request, In the image store it comprises a step of extracting an image associated with the keyword grouping and generates at least one event cluster by cluster at least a portion of the extracted image on the basis of the similarity of the extracted image.

Wherein the step of storing in the image repository determines the image description tag based on keywords appearing in the plurality of image description contents, and determines a time stamp of the representative image based on the creation date of the web documents including the redundant images .

The generating the cluster event may extract images associated with the clustering keyword in the image repository based on the image description tags of the images stored in the image repository.

The generating the cluster event may extract the images created in the search period on the basis of the time stamps of the images stored in the image storage, as the search images.

The generating of the cluster event may extract the name of the cluster event and the tags associated with the cluster event from the image description tags or the image description contents of the images grouped by the cluster event.

Wherein the image clustering method comprises the steps of: extracting at least one candidate from the specific image description content when the specific image description content extracted from an arbitrary web document includes person information of the same name or a subordinate expression; Evaluating whether each candidate is a person in a specific image extracted from the arbitrary web document based on the content and the person information of each candidate person, and determining the person in the specific image based on the evaluation score of each candidate person , And tagging the determined person identification information to the specific image.

The image clustering method may further include transmitting images clustered into the at least one cluster event to a user terminal, wherein the cluster images are displayed on the interface screen of the user terminal according to cluster events, If a plurality of characters are present in the images belonging to the user, the user can be classified and displayed by the characters in the interface screen.

The step of transmitting the clustered images may include generating the content displayed on the interface screen by randomly moving the clustered images or sequentially moving in accordance with the display order and transmitting the content to the user terminal have.

A method of sharing an image of a server operated by at least one processor in accordance with another embodiment, the method comprising: receiving a share request of a plurality of images clustered into a single event; identifying a character of each of the plurality of images; Estimating a person's movement within the images classified into the images, sorting the images of the characters, and transmitting the sorted plurality of images to the user terminal.

The aligning step may calculate the optical flow of the coordinates for all image pairs in the images classified to the same character and arrange the images in order to make the person movements consecutive based on the calculation result have.

The sorting step may classify the plurality of images according to the characters when the plurality of characters appear in the plurality of images, and sort the images classified by the characters.

The transmitting of the plurality of images may include generating the content to be displayed on the interface screen of the user terminal while randomly moving the plurality of images or sequentially moving in accordance with the display order and transmitting the content to the user terminal Step < / RTI >

The transmitting of the plurality of images may further transmit to the user terminal at least one tag associated with the single event or the plurality of images.

A method of image sharing of a server operated by at least one processor in accordance with yet another embodiment, the method comprising: retrieving, in an image repository, at least one stored image that is the same as or similar to the input image; Generating a tag list associated with the input image, and transmitting the shared information of the input image and the tag list to a user terminal, wherein the image repository comprises: The image can be managed by mapping the image description information describing the image.

The input image may be a scraped image from link information requested to be shared from the user terminal.

The information of the input image and the tag list may be displayed on an interface screen of the user terminal, and the sharing information of the input image may be a content of a thumbnail format.

A method of sharing an image of a user terminal operated by at least one processor in accordance with yet another embodiment, the method comprising: transmitting a search keyword requested to be searched to a server; receiving at least one community event associated with the search keyword from the server, Receiving at least one tag associated with the event, and displaying on the first interface screen the images clustered in each cluster event and at least one tag by cluster event. Clusters of images and at least one tag in each cluster event are information obtained from images extracted from a plurality of web documents and image description information describing the images.

Wherein the image sharing method comprises: transmitting a sharing request for an arbitrary cluster event received from the first interface screen to the server; receiving a plurality of images clustered in the random cluster event from the server; And displaying the received plurality of images on the display unit.

The plurality of images are arranged in order to continuously make the movements of the characters, and when a plurality of characters appear in the plurality of images, the plurality of images can be sorted by characters.

The step of displaying the plurality of images may display the generated content on the second interface screen in such a manner that the plurality of images are randomly moved or sequentially displayed according to the display order.

According to the embodiment of the present invention, various images of various sources can be clustered according to attributes such as time, event, person, and the like. According to the embodiment of the present invention, it is possible to generate a content that can be processed and shared by a large amount of images pouring into the web. According to the embodiment of the present invention, the user can obtain meaningful content from the keyword of interest, and visually and quickly confirm the image search result through the sorted images.

1 is a schematic configuration diagram of a system according to an embodiment of the present invention.
2 is a flow diagram illustrating a method for storing images in an image repository in accordance with an embodiment of the present invention.
3 is an exemplary diagram illustrating an image storing method according to an embodiment of the present invention.
4 is a flowchart of an image clustering method according to an embodiment of the present invention.
5 is an exemplary diagram for explaining an image clustering method according to an embodiment of the present invention.
6 is an exemplary diagram of a grouped image provided to a user terminal in accordance with an embodiment of the present invention.
FIG. 7 is a flowchart of a method of identifying an image according to an exemplary embodiment of the present invention.
FIG. 8 illustrates an example of a service using a method of identifying an image according to an exemplary embodiment of the present invention.
9 and 10 are views for explaining a cluster image sharing method according to an embodiment of the present invention.
11 is a diagram illustrating a system for sharing web documents in a messenger service according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as " comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise. Also, the terms " part, " " module, " and " module ", etc. in the specification mean a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software have.

1 is a schematic configuration diagram of a system according to an embodiment of the present invention.

Referring to FIG. 1, a system 10 includes a server 100 in which the functions described in the present invention are implemented, and a user terminal 200 in communication with the server 100. The system 10 includes an image storage 300 in which images processed by the server 100 are stored. The user terminal 200 is provided with an application programmed to provide functions described in the present invention, displays information provided by the server 100 through the application, transmits the information input by the user to the server 100 do.

The server 100 executes the operations described in the present invention by at least one processor. The server 100 classifies various images of various sources according to attributes such as time, event, and personality, and clusters the images through tags describing image attributes. The server 100 generates content that can be shared based on the clustered images and provides an image search result for the search request of the user terminal 200. [ The operation method of the server 100 will be described in detail below.

User terminal 200 includes applications stored on a computer-readable storage medium, and hardware such as a processor, memory, display, communication module, and the like. The application includes instructions for performing the operations described, for example, in the present invention. The processor runs the application and executes the programmed operation. The display may display a user interface screen provided by the application, receive the user input, and receive the touch input, for example. The communication module communicates with the server 100 through a communication network.

The user terminal 200 may be implemented in various forms, for example, in the form of a mobile terminal such as a smart phone, a pad type terminal such as a smart pad, various types of computers such as a laptop computer, a wearable device, Can be implemented.

FIG. 2 is a flowchart illustrating a method of storing images in an image storage according to an exemplary embodiment of the present invention, and FIG. 3 is an exemplary view illustrating an image storing method according to an embodiment of the present invention.

Referring to FIG. 2, the server 100 extracts at least one image included in each web document from a plurality of web documents and content describing the corresponding image (S110). A web document (or simply, a document) is a digital document published on the Web such as a news, a blog, or a social media service, and includes all the documents that the server 100 can access via the network. A web document is composed of at least one image and text, and the same / similar image can be extracted from a plurality of web documents. Since the image and the text included in the web document are correlated with each other, the server 100 can extract contents related to the image or contents describing the image in the text. Contents describing the image can be extracted from the title of the document from which the image is extracted, the words included in the document, the word phrase, and the like from the metadata of the document.

The server 100 tags content describing the image for each image (S120).

The server 100 determines whether the extracted images are duplicated and extracts redundant images (S130). The redundant image extraction method can be selected from various image processing techniques. For example, the server 100 can compare duplicate images by comparing feature values (e.g., 128-bit hash values) representing each image.

The server 100 determines a representative image among the redundant images, and connects the tagged image description contents to each of the redundant images (S140). That is, image information in which a plurality of pieces of web document information are linked to a single image is generated. The server 100 can determine an image having the highest creation date or an image having the highest quality among the redundant images as the representative image, but the representative image can also be selected in various ways.

The server 100 determines a time stamp of an image description tag and a representative image of the representative image based on a plurality of image description contents, maps an image description tag and a time stamp to the representative image, and stores the image description tag and the time stamp in the image storage 300 S150). An image description tag may be obtained from frequently appearing words in a plurality of documents describing the same image. The image description tag, which frequently appears in multiple documents, is the most representative description of the image. The server 100 can determine the earliest creation date among the web documents including the redundant images as the time stamp of the representative image. Alternatively, the server 100 may determine the timestamp of the representative image based on the document contents when a specific event time (for example, a music release date, a performance date, an event schedule, etc.) is described in the web document. News about the event that occurred on that day is written most of the day, but since the web document can include contents about already occurred event and contents about future events, It can also be used as a timestamp.

Referring to FIG. 3, when N documents include image # 1 and each document includes contents describing image # 1, server 100 generates a plurality of web documents around the same image # 1 Connect. Then, the server 100 extracts frequently appearing words (for example, red velvet, album) among the image description contents included in the web documents with the image description tag, It can be extracted with the time stamp of # 1. When document # 1 further includes image # 2, server 100 links a plurality of web documents around image # 2.

The representative image # 1, the image description tag, and the time stamp, which are stored in the image storage 300, can be stored as shown in Table 1. The representative image may be stored as a feature value and / or an image file.

representation
image image
Description Tag /
Timestamp Image descriptions included in web documents Image # 1 Red velvet
album
2016.09.21 Document 1: Red Velvet is about to release new album (2016.09.22)
Document 2: Transforming the Red Velvet Style (2016.09.21)
Document 3: Red Velvet is back with new album (2016.09.21)
Document 4: Red Velvet Style New Album (2016.09.25)
...
Document N: A fresh red velvet new song (2016.09.21) Image # 2 - -

In the following, a method of grouping the same / similar images according to a clustering criterion such as time / person / event kind will be described.

FIG. 4 is a flowchart illustrating an image clustering method according to an exemplary embodiment of the present invention, FIG. 5 is an exemplary view illustrating an image clustering method according to an embodiment of the present invention, FIG. 6 is a flowchart illustrating an image clustering method according to an embodiment of the present invention Fig. 8 is an exemplary diagram of a clustered image provided to a user terminal.

Referring to FIG. 4, the server 100 receives an image clustering request including a clustering keyword (S210). When the server 100 receives an image search request for a search keyword from the user terminal 200, the server 100 may apply the search keyword as a clustering keyword to start image clustering. Alternatively, the server 100 may periodically perform an image clustering operation for a specific keyword, or may perform an image clustering operation for the event whenever a specific event occurs.

In operation S220, the server 100 extracts images related to the clustering keyword from the image storage 300 in which the image description tag and the time stamp are mapped to the image extracted from the web document. At this time, the server 100 displays the image created in the specific search period (for example, three days before today, three days before today, one week before today, one month before today, one year before today, . The search period can be set by the user or the server. Meanwhile, the server 100 maps an image description tag and a time stamp to an image extracted from a large amount of web documents before image clustering, and stores the image description tags and the time stamp in the image storage 300. In the image clustering, It is assumed that the image associated with the keyword is extracted. However, the server 100 may map the image description tag and the time stamp to the image extracted from the web document, and store it in the image repository 300, while image clustering is performed.

The server 100 arranges the images associated with the clustering keywords in the time stamp order (S230). 5A, the images associated with the clustering keyword are extracted from the image storage 300 in which the duplicate images existing in the web document are arranged into a single image, so that the images extracted from the image storage 300 It is composed of unequal images. For example, when the retrieval period for the clustering keyword is November 1 to November 3, the server 100 displays M images (1101_image_1 to 1101_image_m) whose timestamp is November 1, the time stamp is November 2 N images (1102_image_1 to 1102_image_n), and K images (1103_image_1 to 1103_image_k) whose time stamp is November 3 are arranged in chronological order. Each image is an image extracted from web documents such as news, blog, and cafe, and the image description tag is tagged with metadata. In addition, each image may be additionally tagged with an image title or an image source.

The server 100 generates at least one cluster event by grouping the images based on the similarity of images arranged in the order of time stamp (S240). The similarity of the images may include similarity between images or similarity of the image description tag tagged in the image. Referring to FIG. 5B, the server 100 generates a cluster event # 1 and a cluster event # 2 by grouping images having a similarity value equal to or higher than the reference value out of M images of November 1st with a timestamp of 1 November, 3 is generated by grouping images having the similarity value equal to or higher than the reference value among the images of the Nth chapter on Nov. 2, and a cluster event # 4 is generated by grouping images having similarity of the reference value or higher among the K images having a time stamp of November 3 . At this time, the server 100 may exclude images that do not belong to any cluster event among the sorted images, or may expose a single image as a search result image as it is.

On the other hand, in step S230 and step S240, it is described that the server 100 arranges images in time stamp order to bundle images in units of time (for example, one day unit) that are separated. However, the server 100 can extract images created in a specific search period (for example, three days before today) without having to sort the images in a timestamp order (for example, today, yesterday, And all the extracted images can be grouped based on similarity.

The server 100 maps an event name and a cluster tag to each cluster event (S250). The server 100 analyzes the image description tags of the images belonging to the community to create an event name representing the community (for example, a city ooo, a rehearsal for ooo, a competition for ooo, a movie ooo preview, etc.) We analyze the image description tags of the images belonging to the cluster and construct a cluster tag that represents the cluster. The cluster tag may include all the image description tags of the images belonging to the cluster or may include the upper cluster tag among the image description tags of the images belonging to the cluster. In addition to the image description tag, the server 100 may determine the keywords associated with the image description tag as cluster tags, or may determine the keywords extracted from the information associated with the search keywords as cluster tags. For example, when the search keyword is a character, the server 100 extracts a feature film / feature broadcast of the character and a synonym / similar word therefrom, and extracts a keyword, which is determined to be highly related to the search keyword or the cluster event, It can be decided by a cluster tag. At least some of the cluster tags may be displayed on the user terminal 200 together with the clustered images. The clustered images, event names, and cluster tags for each cluster event are stored in the image repository 300.

The server 100 transmits the clustered images to the user terminal 200 using a specific cluster event or a cluster tag (S260). The clustered images are displayed on the user interface screen of the user terminal 200.

Referring to FIG. 6A, the user terminal 200 displays images on the user interface screen 400 for each cluster event. When there are a plurality of cluster events, the user terminal 200 can display the images in chronological order. For example, the user interface screen 400a displays an event name 410, a clustered image 420, and at least one tag 430 for each cluster event. The tag may be a tag associated with the clustering event or a tag associated with the clustering event. The user interface screen 400 may further display a share button 440, a date 450, and the like for each cluster event. The user interface screen 400 may further display a search keyword name 460 used in cluster event search. When the user selects the tag 430, the user terminal 200 can display the search result related to the tag 430 on the shared screen (for example, the chat screen of the instant messenger application).

6B, when the user selects the sharing button 440, the user terminal 200 transmits a community event corresponding to the sharing button 440 to a shared screen (for example, a chat screen of the instant messenger application 440) ) 400b. At this time, the shared clustering event may display the entire clustering image on the shared screen. However, the shared clustering event may include a thumbnail form or a content (clustering) that is displayed (played) randomly or randomly (Hereinafter, referred to as 'moving image content' or 'summary content') 470.

The sharing screen 400b may display the tags associated with the shared content together with the shared content.

FIG. 7 is a flowchart illustrating a method of identifying an image according to an exemplary embodiment of the present invention. FIG. 8 is an illustration of a service using a method of identifying an image according to an exemplary embodiment of the present invention.

Referring to FIG. 7, there are a number of web documents such as news, blogs, and social media services. When the images extracted from the web document are overlapped, the server 100 determines a representative image among the redundant images, Connect the tagged image description to each of the images. However, since the content of the image description is text, the image search result may be inaccurate if the same name or ambiguous expression exists. Although the image classifier can classify a person, the learning-based image classifier is not scalable because the number of people is constantly increasing. Accordingly, the server 100 confirms the person in the image as follows based on the image description content extracted together with the image.

The server 100 manages person information for each person (S310). The person information includes a feature value of the person image, unique person identification information (ID) assigned to the person, and additional information (e.g., job, etc.) that can identify the person. On the other hand, the server 100 can use an external person database without directly managing the person information.

When the server 100 receives the image extracted from the web document and the image description, the server 100 extracts a candidate from the image description (S320). If it is not necessary to check the person in the image from the image description, or if it is certain that the person is a specific person, the following person identification procedure may be omitted and a procedure of tagging the image description tag directly on the extracted image may be performed.

The server 100 evaluates whether the candidate person is a person in the image based on the keywords included in the image description content and the person information of the candidate person (S330). The server 100 may mash up various APIs such as a person database, a movie database, and a TV program database to evaluate whether a candidate is a person in the image.

The server 100 confirms the person in the image based on the evaluation score of the candidate character (S340). When there are a plurality of candidate persons, the server 100 can determine the person in the image based on the evaluation score of each candidate person. The server 100 can finally determine the person by comparing the image feature value of the identified person with the feature value of the image extracted from the web document to improve the accuracy.

The server 100 adds the identified person ID to the image description tag of the image (S350).

For example, when the server 100 receives image description contents such as 'Kim Beom-soo and Yoo Hee-yul in a sketch book' together with a portrait image extracted from a web document, a person name 'Kim Beom-su' Based on the candidates (singer Kim Bum-soo, announcer Kim Bum-soo, executive Kim Bum-soo) is extracted.

The server 100 evaluates the degree of matching with the image description content among the candidates based on the keyword included in the image description content (e.g., a sketchbook of 'Yu Hee Yeol', which is a music broadcast program name) and the character information of the candidate character After that, you can see the highest number of 'singer Kim Bum Soo' in the image. At this time, the evaluation standard and the evaluation method can be set variously.

When the server 100 stores the images extracted from the web document in the image repository described with reference to FIG. 2, the server 100 can store the accurate person information by tagging. When the server 100 provides the clustered image to the user terminal 200, the server 100 may cluster the images with accurate person information. Accordingly, when the user inputs a person name tag (e.g., # Kim Bum-soo) and requests a search, the server 100 can separately provide an image clustered by Kim Bum-su and an image clustered by an announcer Kim Bum-soo.

Referring to FIG. 8, the person identification method in the image of the server 100 described above can be applied to the person identification service included in the image.

For example, the server 100 may be requested to confirm the person in the image. The way in which a user requests a person confirmation may vary. According to one embodiment, when a user browses a web document containing an image and wonders who the image is included in the web document, the user transmits link information (URL) of the web document or the web document to the server 100, You can request confirmation of the person in the image included in the document. Then, the server 100 may add the person identification information as shown in FIG. 8 to the image of the web document and display the same on the user terminal 200. According to another embodiment, the user may send an image to the server 100 to request identification of the image.

When a plurality of persons are included in the image, the server 100 can evaluate who each person is and can confirm the person's name based on the evaluation result.

When the image description content is input together with the image, the server 100 determines a candidate character from the image description content, and compares the image feature value of the narrowed candidate character with the feature value of the input image to evaluate each of the plurality of characters . For example, when image description content called 'Twice OO Stage' is input together with the image, the server 100 determines the candidate person as a 'Twice member' based on the image description content, The possibility of each member can be evaluated with a score.

Even if only the image is input, the server 100 searches the image storage 300 for the same stored image as the input image. Then, the server 100 determines a candidate character based on the image description tag or image description contents corresponding to the stored image, and compares the image feature value of the narrowed candidate character with the feature value of the input image to evaluate each of the plurality of characters have.

If there is no stored image identical to the input image in the image repository 300, the server 100 compares the characteristic values of the person images stored in the image repository 300 with the characteristic values of the input image to evaluate each of the plurality of persons .

The user terminal 200 may display the rating score together with the character name as shown in FIG. Each of the person names is generated as a tag, and when the user selects a corresponding person name or a person name tag on the interface screen, the user terminal 200 can display a search result related to the selected person name tag.

9 and 10 are views for explaining a cluster image sharing method according to an embodiment of the present invention.

Referring to FIGs. 9 and 10, the server 100 generates clustering images in the form of content that can be shared by users, and provides them to the user terminal 200. FIG. The server 100 may provide a 'moving image content' or 'summary content' to the user terminal 200 in which the clustering images are continuously played back. Although the service in which users share contents may be various, here, an example of sharing crowded images with chatters on the chat room screen of the messenger application will be described.

First, a description will be given of a method of generating summary type content using the clustered images of the server 100.

Referring to FIG. 9, the server 100 fetches a clustered image included in the cluster event (S410).

The server 100 identifies the characters in each of the crowded images and arranges them by characters (S420).

The server 100 confirms the movement among the same person images among the same person images sorted by the characters (S430). The server 100 can track the optical flow of the image coordinates to find the motion.

The server 100 calculates a continuous motion score for all image pairs from the optical flow of coordinates for all image pairs in the image in which the same person appears (S440). When many coordinates of two images move in succession in the same direction, the motion score is calculated high. The server 100 may store the relationship of all image pairs obtainable from the N images as an NxN matrix.

The server 100 estimates the motion direction in the image based on the motion scores of all the image pairs, and sequentially arranges the images in a continuous sequence of motion (S450). For example, when the images of the person # 1 included in the cluster event are from IMG1 to IMG5, the server 100 transmits IMG1, IMG2, IMG1 and IMG3, IMG1 and IMG4, IMG1 and IMG5, IMG2 and IMG3, IMG1 and IMG4, IMG2, IMG3, IMG1, IMG5, and IMG4, based on the motion scores of IMG2 and IMG5, IMG3 and IMG4, IMG3 and IMG5, IMG4 and IMG5, Place the images in order.

The server 100 generates summary content (moving image content) that is sequentially displayed (sequentially or sequentially) sequentially arranged images (S460). Because the images are arranged in a sequence that makes the movement of figures based on the optical flow, the characters seem to move naturally when the images are moved consecutively (randomly or sequentially). The summary type content may be transmitted to the chat room of the messenger application and shared as shown in FIG. 6 (b).

Summary type content can be selected and created from various templates. For example, the abstracted content may be generated as a template with images of a single character in sequence. The summary type content can be generated by connecting summary characters of a plurality of characters when a plurality of characters are present in a cluster event.

The server 100 may select a plurality of images from the clustered images to generate summary content or the server 100 may generate a plurality of images selected from the user terminal 200, You can generate summary content based on a person.

Next, a method of displaying the content related to the cluster event received from the server 100 on the screen will be described.

Referring to FIG. 10 (a), the user terminal 200 receives a plurality of images clustered from cluster events from the server 100 and displays the images on the search result screen 500a. The user terminal 200 may request the server 100 for a cluster event for the search keyword. At this time, the user can request a cluster event by directly inputting the search keyword, or can request a cluster event by selecting a search keyword displayed on the arbitrary interface screen. In the user interface screen 500a, if there are search results related to search keywords in other collections in addition to the clustering event, another collection can be displayed in the card view format.

The search result screen 500a may display representative images 510 of a plurality of images. At this time, the search result screen 500a may display a tag 520 (for example, #TWICE, # #, # stage, #M countdown, etc.) related to the cluster event. The user interface screen 500a may display the sharing button 530. [ The user interface screen 500a may display the image expansion view button 540. [

Referring to FIG. 10B, when the image unfolding view button 540 of the search result screen 500a is selected, the images of the clustering events hidden in the search result screen 500b are expanded and displayed. At this time, the images are sorted by the persons, and the images of the same person are displayed according to the movement direction (copper line). On the other hand, in FIG. 10 (a), when a representative image is selected, only the images linked to the representative image can be expanded and displayed.

When the sharing button 530 is selected, the server 100 may generate a plurality of images as summary contents and transmit them to the user terminal 200. Alternatively, when the user selects a button (not shown) that can request summary content generation for the images included in the cluster event on the search result screen 500a or 500b, the server 100 may summarize the plurality of images Type content and transmit the generated content to the user terminal 200. The search result screen 500a or 500b may further include an image selection button (not shown) so that the user can select the images included in the cluster event and generate the summary type content.

Referring to FIG. 10C, the user terminal 200 receives the summary content 610 of the clustering event from the server 100 and displays the summary content 610 on the shared screen (for example, the chatting screen) 600 have. When the user makes a selection, such as tapping the summary content 610, the images included in the summary content 610 are displayed sequentially (randomly or sequentially) sequentially over the images. The sharing screen 600 may display the tags 620 (e.g., # TWO, #, #, etc.) associated with the summary content with the summary content 610. [

When the recommendation tag or the representative tag is selected, the user terminal 200 can display the corresponding search result on the sharing screen 600 or a separate interface screen.

11 is a diagram illustrating a system for sharing web documents in a messenger service according to an embodiment of the present invention.

Referring to FIG. 11, it is assumed that the server 100 shares content through a messenger application of the user terminal 200. FIG. The server 100 can be divided into a messaging server 10, an image analysis server 12, a scrap server 14 and the like according to the functions provided by the messenger service, and the messaging server 10, the image analysis server 12, And the scrap server 14 and how they communicate with the user terminal 200 can be variously modified according to the interworking rule or the service scenario. The server 100 includes at least one of a messaging server 10, an image analysis server 12 and a scrap server 14. A server function not included in the server 100 includes an external server system A system operated by a third party), and can be provided from an external server system.

The user terminal 200 transmits the specific link (URL) information to the scrap server 14 and the scrap server 14 transmits the specific link (URL) information to the specific link (URL) URL) information, and scrapes the corresponding content. The scraped content includes an image and image description information (e.g., a web document title) included in the web document.

The scrap server 14 sends the scraped content to the image analysis server 12 to request the shared content.

The image analysis server 12 analyzes the information such as the appearance of a person in a scraped image, the position of a face, the position of an object, and generates a thumbnail. The image analysis server 12 then extracts the same / similar image as the scraped image in the image repository 300. Here, as shown in Table 1, the image storage 300 stores image description tags and image description contents included in web documents for each image. Accordingly, the image analysis server 12 generates the tag list associated with the scraped image based on the image description tags mapped to the extracted same / similar images or the image description contents included in the web documents in the image storage 300 can do.

The thumbnail and tag list of the scraped content may be delivered to the user terminal 200 via the scrap server 14.

The chat room screen 700 displays a thumbnail 720 of the scraped content in the link (URL) information 710 and at least one tag 730 included in the tag list. When the tag 730 is selected, the user terminal 200 can display the search result corresponding to the tag 730 on the chat window screen 700 or a separate interface screen.

As described above, according to the embodiment of the present invention, various images of various sources can be clustered according to attributes such as time, event, and person. According to the embodiment of the present invention, it is possible to generate a content that can be processed and shared by a large amount of images pouring into the web. According to the embodiment of the present invention, the user can obtain meaningful content from the keyword of interest, and visually and quickly confirm the image search result through the sorted images.

The embodiments of the present invention described above are not implemented only by the apparatus and method, but may be implemented through a program for realizing the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (20)

A method of image clustering of a server operated by at least one processor,
Extracting at least one image included in each web document and contents describing the image in the plurality of web documents,
Determining a representative image representative of the redundant images based on a web document creation date or image quality if redundant images are present among the images extracted from the plurality of web documents, Connecting corresponding image description contents,
Determining image information including an image description tag and a time stamp of the representative image from a plurality of image description contents connected to the representative image, mapping the image information and the representative image, and storing the image information in an image storage, and
Extracting images associated with the clustering keyword in the image repository if there is an image clustering request according to clustering keywords, and generating at least one clustering event by clustering at least some of the extracted images based on the similarity of the extracted images Lt; / RTI >
The step of storing in the image repository
Determining the image description tag including keywords appearing more than a predetermined number of times in the plurality of image description contents, determining a creation date of the web document having the shortest creation date among the web documents including the redundant images as a time stamp of the representative image,
Wherein only images determined as the representative image among the redundant images are stored in the image repository. delete The method of claim 1,
The step of generating the cluster event
And extracting images associated with the clustering keyword in the image repository based on an image description tag of images stored in the image repository. 4. The method of claim 3,
The step of generating the cluster event
And extracting the images created in the search period as search images based on the time stamps of the images stored in the image storage. The method of claim 1,
The step of generating the cluster event
Wherein the name of the cluster event and the tags associated with the cluster event are extracted from the image description tags or image description contents of the images grouped in the cluster event. The method of claim 1,
The step of extracting contents describing the corresponding image
Extracting at least one candidate character from the specific image description content when the specific image description content extracted from an arbitrary web document among the plurality of web documents includes the character information of the same name or ambiguous expression;
The degree of matching of the specific image description with the information included in the person information of each candidate person is compared based on the person information of each candidate person to determine whether each candidate person is a person in a specific image extracted from the arbitrary web document ; And
Determining a person in the specific image based on the evaluation score of each candidate person, and tagging the determined person identification information on the specific image
Wherein the image clustering method further comprises: The method of claim 1,
Further comprising transmitting to the user terminal the aggregated images into the at least one cluster event,
Wherein the grouped images are displayed in a grouped manner according to a grouping event on the interface screen of the user terminal, and when a plurality of characters are present in the images belonging to the same grouping event, Way. 8. The method of claim 7,
The step of transmitting the clustered images to the cluster event
And generating content to be displayed on the interface screen, and transmitting the content to the user terminal. delete delete delete delete delete delete delete delete 14. An image sharing method of a user terminal operating by at least one processor,
Transmitting a search keyword requested to be searched to a server,
Receiving from the server at least one cluster event associated with the search keyword and at least one tag associated with each cluster event, and
Displaying on the first interface screen the images clustered in each cluster event and at least one tag by cluster event,
Clusters of images at each cluster event and at least one tag
Wherein the information is information obtained from images extracted from a plurality of web documents and image description information describing the extracted images. The method of claim 17,
Transmitting a sharing request for an arbitrary cluster event received from the first interface screen to the server,
Receiving from the server a plurality of images clustered at the random clustering event, and
Displaying the received plurality of images on a second interface screen
The image sharing method further comprising: The method of claim 18,
Wherein the plurality of images are arranged in order to continuously make a movement of a character, and when a plurality of characters appear in the plurality of images, the plurality of images are arranged by characters. The method of claim 18,
The step of displaying the plurality of images
And displaying the generated content on the second interface screen in such a manner that the plurality of images are randomly displayed or sequentially displayed in accordance with the display order.

Images