JP7652480B1 - Information processing system, information processing method, and program - Google Patents

Abstract

To enable effective selection of videos.
[Solution] An information processing system comprising: a video data storage unit that stores multiple video data; a question acquisition unit that acquires questions from users; and a playback video determination unit that provides a prompt to a large-scale language model including information about the video data, the question, and an instruction to select video data to provide in response to the question, thereby determining from the video data the video data to answer the question.
[Selected Figure] Figure 1

Description

The present invention relates to an information processing system, an information processing method, and a program.

A system is provided that answers questions from users (see, for example, Patent Document 1).

JP 2006-039881 A

In recent years, there has been a demand to provide information through video rather than text, but it is difficult to provide appropriate answers to user questions through video.

The present invention was made in light of this background, and aims to provide a technology that enables effective video selection.

The main invention of the present invention for solving the above problem is an information processing system comprising a video data storage unit that stores multiple video data, a question acquisition unit that acquires a question from a user, and a playback video determination unit that provides a prompt to a large-scale language model, the prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, and determines the video data to be provided from the video data to answer the question.

Other problems and solutions disclosed in this application will be made clear in the description of the embodiments of the invention and the drawings.

The present invention allows for effective video selection.

FIG. 1 is a diagram illustrating an example of an overall configuration of an information processing system. FIG. 2 illustrates an example of a hardware configuration of a management server 2. FIG. 2 illustrates an example of the software configuration of a management server 2. FIG. 2 is a diagram illustrating the operation of the management server 2.

<System Overview>
An information processing system according to an embodiment of the present invention will be described below. The information processing system of this embodiment is configured to provide answers to questions from a user using videos, and uses a large language model (LLM) to determine which videos from a plurality of prepared videos should be provided to the user and in what order.

Figure 1 is a diagram showing an example of the overall configuration of an information processing system. The information processing system of this embodiment is configured to include a management server 2. The management server 2 is communicatively connected to a user terminal 1 via a communication network. The communication network is, for example, the Internet, and is constructed using a public telephone line network, a mobile phone line network, a wireless communication path, Ethernet (registered trademark), etc.

The user terminal 1 is a computer operated by a user. The user terminal 1 may be, for example, a smartphone, a tablet computer, a personal computer, etc.

The management server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

<Management Server>
FIG. 2 is a diagram showing an example of the hardware configuration of the management server 2. Note that the illustrated configuration is an example, and other configurations may be used. The management server 2 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 is, for example, a hard disk drive, a solid state drive, or a flash memory that stores various data and programs. The communication interface 204 is an interface for connecting to a communication network, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or an RS232C connector for serial communication, etc. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, etc. that input data. The output device 206 is, for example, a display, a printer, a speaker, etc. that output data. Each functional unit of the management server 2 described below is realized by the CPU 201 reading a program stored in the storage device 203 into the memory 202 and executing it, and each storage unit of the management server 2 is realized as part of the storage area provided by the memory 202 and the storage device 203.

Figure 3 is a diagram showing an example of the software configuration of the management server 2. The management server 2 includes a video data storage unit 231, a question acquisition unit 211, a playback video determination unit 212, and a playback processing unit 213.

<Memory Unit>
The video data storage unit 231 stores video data. In this embodiment, the video data storage unit 231 stores information about the video data (hereinafter, video information) together with the video data. The video information can include any information (metadata) that represents the contents and attributes of the video data, and can include, for example, information that identifies the video (for example, a video ID), a video title, a video description, video data, and the like. The metadata of the video information can also include the playback time of the video, the creation date and time of the video, the creator of the video, the attributes of the viewers targeted by the video, the video category, and keywords related to the video. The metadata of the video information can also include thumbnail images that represent the contents of the video, subtitle data of the video, and transcribed text data generated from the audio data of the video. Note that the video information may not include the contents of the video data, and may store the video data as, for example, a file, and store information for acquiring the file (hereinafter, referred to as file acquisition information. For example, it may be a path on a file system or a URL that identifies data in cloud storage), and acquire the video data based on the file acquisition information. The video data storage unit 231 stores a plurality of video data (and video information related to the video data). The title of the video information may be set in the file name. The description of the video is assumed to be text data, but may be video, audio, or the like.

In this embodiment, it is assumed that a large number of short video data (e.g., about 15 seconds to 1 minute) are registered in the video data storage unit 231. For example, one video data can be registered for one topic or subtopic. In this embodiment, as described below, these short video data are linked together to dynamically generate answers.

<Functional department>
The question acquisition unit 211 acquires a question from a user. The question from the user can be received from, for example, the user terminal 1. Also, an input of the question from the user together with the user's designation may be accepted from the user terminal 1 of the staff member who received the question from the user. The question acquisition unit 211 can acquire, for example, a message input by the user to a BOT or the like as a question.

The playback video determination unit 212 determines the video data to be played as a response to the user's question. The playback video determination unit 212 provides a prompt to the large-scale language model, the prompt including information about the video data, the question, and an instruction to select video data to provide in response to the question, and determines the video data to be used to respond to the question from among the video data.

The information about the video data may be at least one of the title and the description of the video included in the video information. The information about the video data may include the various metadata described above. The information about the video data included in the prompt may be, for example, information about all video data stored in the video data storage unit 231.

The large-scale language model may be provided, for example, in the management server 2. In this case, the management server 2 stores a program for executing the large-scale language model in the storage device 203, and the CPU 201 executes the program to realize the large-scale language model. The large-scale language model may also be provided in a server other than the management server 2. In this case, the management server 2 can access the large-scale language model using an external API (Application Programming Interface). For example, the management server 2 can send an HTTP (Hypertext Transfer Protocol) request to a specific URL (Uniform Resource Locator) and obtain the processing result of the large-scale language model as a response. As the large-scale language model, GPT-3 (Generative Pre-trained Transformer 3), PaLM (Pathways Language Model), Chinchilla, OPT (Open Pre-trained Transformer), etc. may be used. Fine-tuned versions of these large-scale language models may also be used.

The playback video determination unit 212 can determine the video data to be played as well as the order in which they should be played.

The playback video determination unit 212 can use various methods when determining the playback order of video data provided in response to a question. For example, the playback video determination unit 212 can determine the playback order by comparing the content of the selected video data with the content of the question and arranging the video data in the order in which matters related to the content of the question are explained. The playback video determination unit 212 can also determine the playback order by comparing the contents of the selected video data with each other and arranging video data with related contents so that they are consecutive. Furthermore, the playback video determination unit 212 can determine the playback order based on the priority or importance that is preset for the video data.

In this embodiment, the playback video determination unit 212 determines the playback order using a large-scale language model. For example, the playback video determination unit 212 can provide the large-scale language model with a prompt including information about the video data, a question, and instructions to select video data to be provided in response to the question and to determine the playback order, and can determine the video data from the video data to answer the question and the playback order thereof.

The playback order determined by the playback video determination unit 212 can be expressed in various formats. For example, the playback order can be expressed as the order of numbers assigned to the video data. In this case, the playback video determination unit 212 can assign numbers to the selected video data in the order in which they should be played. The playback order can also be expressed as a list of the file names of the video data. In this case, the playback video determination unit 212 can generate a list in which the file names of the selected video data are arranged in the order in which they should be played. Furthermore, the playback order can also be expressed as a list indicating the playback start time and playback end time of the video data. In this case, the playback video determination unit 212 can determine the playback start time and playback end time for each selected video data, and generate a list in which they are arranged in the order in which they should be played.

The playback video determination unit 212 may select the video data to be played and generate a reason for selecting the video (a reason for viewing the video in response to the question). The playback video determination unit 212 may provide a prompt including an instruction to explain the reason for viewing the video to the large-scale language model to generate the reason. In this case, the playback video determination unit 212 may provide a prompt including information on the video data, a question, and an instruction to consider the video data to be provided in response to the question and the reason for it to the large-scale language model. After the video is selected, the reason may be generated by another prompt. In this case, the playback video determination unit 212 may provide a prompt including a question, information on the selected video data, and an instruction to explain the reason for selecting the video data to the large-scale language model. The playback video determination unit 212 may select video data and include an instruction to consider the reason for the selection and the playback order in the prompt.

The playback video determination unit 212 may also instruct the large-scale language model to analyze the relevance between keywords included in the title or description of the selected video data and keywords included in the question, and generate a sentence including highly relevant keywords as part of a reason. The playback video determination unit 212 may also instruct the large-scale language model to generate text data summarizing the contents of the selected video data, compare the text data with the question, extract a portion suitable for explaining the contents of the question, and generate the extracted portion as part of a reason. The playback video determination unit 212 may also instruct the large-scale language model to generate a reason why the selected video data is appropriate as an answer to the question, by referring to the titles and descriptions of other video data related to the selected video data and analyzing the relevance of these with the question.

The playback processing unit 213 performs processing to play back the video data in the playback order. The playback processing unit 213 can output (e.g., transmit to the user terminal 1) a list of the video data determined by the playback video determination unit 212. The playback processing unit 213 may output the list of determined video data in the determined playback order. The playback processing unit 213 may arrange the list of determined video data in the determined playback order and attach to the video data a reason why the video data should be viewed.

The playback processing unit 213 can control the video data determined by the playback video determination unit 212 to be played back continuously in the determined playback order. The playback processing unit 213 can, for example, transmit the determined video data to the user terminal 1 continuously in the determined playback order so that the data is played back continuously in the user terminal 1. For example, the playback processing unit 213 can stream the video data. As a streaming delivery method, for example, various streaming protocols such as HTTP Live Streaming (HLS), Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), and MPEG-DASH can be used. The playback processing unit 213 can also download and distribute the video data. In this case, the playback processing unit 213 can control the video data stored in the local storage of the user terminal 1 to be played back after the video data is downloaded to the user terminal 1. The playback processing unit 213 may transmit the determined video data to the user terminal 1 with a playback order attached, so that the video data is played back in the playback order in the user terminal 1.

The playback processing unit 213 may also transmit, together with the video data, a reason why the video should be viewed, and cause the user terminal 1 to display the reason before, during, and/or after playback of the video data.

<Operation>
FIG. 4 is a diagram illustrating the operation of the management server 2.

The management server 2 acquires a question from the user (S301), provides the question and the titles/descriptions of the videos to be played to the LLM to generate the videos to be played and their order (S302), and controls the playback of the selected videos in the playback order (S303).

As described above, the information processing system of this embodiment can play back video data that is appropriate as an answer to a question from a user from among the video data prepared in advance. Furthermore, the information processing system of this embodiment can also play back appropriate answers to a question in an appropriate order. Furthermore, the information processing system of this embodiment can also display reasons why the suggested video data should be viewed.

The present embodiment has been described above, but the above embodiment is intended to facilitate understanding of the present invention and is not intended to limit the interpretation of the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents thereof are also included in the present invention.

For example, the processing by each functional unit of the management server 2 described above may be executed by any of the functional units. Also, a different functional unit that executes part of the processing by each functional unit described above may be added. Also, the functional units of the management server 2 may be distributed across multiple computers.

In addition, the information stored in each storage unit of the management server 2 may be stored by any storage unit. That is, the information stored in the multiple storage units described above may be stored by one storage unit, or part of the information stored in one storage unit described above may be stored by another storage unit.

<Modification 1>
Information about video data can include a wider variety of information, such as the date and time the video was created, the creator, the playback time, and the attributes of the target audience.

For example, the video information can include timestamp data indicating the date and time the video was created. In response to a question, videos can be selected based on the date and time the video was created, such as by preferentially selecting the most recent video or, conversely, by selecting an older video.

For example, the video information may include creator information that indicates the creator of the video. Creator information may include, for example, the creator's name, affiliation, title, and area of expertise. In response to a question, videos may be selected based on the video creator, such as preferentially selecting videos by a specific creator or selecting videos by multiple creators in a balanced manner.

For example, the video information can include playback time information that indicates the playback time of the video. In response to a question, videos can be selected based on the video playback time, such as preferentially selecting videos with long or short playback times, or selecting videos whose total playback time falls within a specified time.

For example, the video information may include viewer attribute information indicating the attributes of the target viewer. The viewer attribute information may include, for example, the expected age group, gender, occupation, and interests of the viewer. A video may be selected according to the attributes of the user who asked the question, and video information about this selected video data may be provided in a prompt to select video data to suggest to the user from the selected video data.

<Modification 2>
The playback video determination unit 212 may select video data without using a large-scale language model. For example, the playback video determination unit 212 generates data obtained by vectorizing the title and description of the video data in advance, and calculates the cosine distance between the data and the vectorized data of the question acquired by the question acquisition unit 211, thereby selecting video data related to the question.

For example, the play video determination unit 212 can perform morphological analysis on the title and description of each video data, and vectorize each video data based on the frequency of occurrence of each morpheme. The play video determination unit 212 can also perform morphological analysis and vectorize the question acquired by the question acquisition unit 211 in the same way. The play video determination unit 212 can then calculate the cosine distance between the vector of the question and the vector of each video data, and select the video data in order of smallest cosine distance. This allows the play video determination unit 212 to appropriately select video data related to the content of the question.

In addition, the playback video determination unit 212 may use information other than the title and description when vectorizing the video data. For example, the playback video determination unit 212 may vectorize the video data using metadata (keywords, category, etc.) assigned to the video data, or user ratings of the video data (number of views, number of likes, etc.). This allows the playback video determination unit 212 to select video data that is more relevant to the question.

The play video determination unit 212 may also select video data related to a question using a method other than cosine distance. For example, the play video determination unit 212 may calculate the Euclidean distance between the vector of the question and the vector of each video data, and select video data in order of smallest Euclidean distance. The play video determination unit 212 may also calculate the inner product between the vector of the question and the vector of each video data, and select video data in order of largest inner product. By appropriately selecting one of these methods, the play video determination unit 212 can select video data that is more related to the question.

<Modification 3>
The playback video determination unit 212 can determine the playback order of video data provided in response to a question based on the relevance of the contents between the video data. For example, the playback video determination unit 212 can extract keywords contained in the titles and descriptions of the video data, and determine a playback order such that video data containing many common keywords are played consecutively. The playback video determination unit 212 can also refer to metadata (e.g., tags representing objects and scenes shown in the video) that represent the contents of the video data, and determine a playback order such that video data having high similarity in metadata are played consecutively. This allows video data having high relevance in content to be played consecutively.

The playback video determination unit 212 can also determine the playback order of video data provided in response to a question based on the degree of relevance between the question and the content of the video data. For example, the playback video determination unit 212 can compare keywords contained in the question with keywords contained in the title or description of the video data, and determine a playback order in which video data with higher keyword matches are played first. The playback video determination unit 212 can also calculate the similarity between the content of the question and metadata representing the content of the video data, and determine a playback order in which video data with higher similarity are played first. This allows video data with higher relevance to the content of the question to be played preferentially.

Furthermore, the playback video determination unit 212 can determine the playback order by combining the relevance of the content between the video data and the degree of relevance between the question and the content of the video data. For example, the playback video determination unit 212 can first select video data that is highly relevant to the question, and then determine the playback order based on the relevance between the selected video data.

<Modification 4>
When generating a reason for selecting a video, the playback video determination unit 212 may generate the reason by comparing the content of the selected video with the content of the question and extracting a highly related part. For example, the playback video determination unit 212 can generate a reason for selecting a video by comparing text data such as the description and subtitle data of the selected video with the text data of the question, extracting words that appear in common between the two and words that are synonymous or similar, extracting sentences, paragraphs, etc. that include the extracted words from the text data of the video and the question, and summarizing these using a text summarization algorithm.

Furthermore, the playback video determination unit 212 may add the playback start time and end time of the part of the selected video that is highly relevant to the content of the question to the reason for selecting the video. Based on this playback start time and end time information, the playback processing unit 213 may play only the relevant part of the video, or may play the entire video before or after playing the relevant part.

<Variation 5>

The playback processing unit 213 can play back the video data in various ways. For example, the playback processing unit 213 can stream the video data. In this case, the playback processing unit 213 can transmit the video data to the user terminal 1 in real time and play the video data as it is received by the user terminal 1.

The playback processing unit 213 can also download the video data and then play it. In this case, the playback processing unit 213 transmits the video data to the user terminal 1, and the user terminal 1 receives and stores the video data. The stored video data can be played back in the user terminal 1.

The playback processing unit 213 can also perform playback by combining streaming and downloading of video data. For example, the playback processing unit 213 can download part of the video data, start playback, and stream the remaining part.

<Disclosures>
The present disclosure also includes the following configurations.
[Item 1]
a video data storage unit for storing a plurality of video data;
a question acquisition unit for acquiring a question from a user;
a playback video determination unit that provides a prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question to a large-scale language model, and determines the video data to be provided in response to the question from among the video data;
An information processing system comprising:
[Item 2]
Item 1 is an information processing system according to the present invention,
the playback video determination unit includes in the prompt an instruction for creating the video data and a playback order to be provided in response to the question, selects the video data to answer the question from among the video data, and determines the playback order of the selected video data;
An information processing system comprising:
[Item 3]
Item 2: An information processing system according to the present invention,
the playback video determination unit further provides the prompt, including an instruction to explain a reason for selecting the video data, to the large-scale language model to create the reason;
An information processing system comprising:
[Item 4]
Item 2: An information processing system according to the present invention,
a playback processing unit that plays back the video data in the playback order;
An information processing system comprising:
[Item 5]
storing a plurality of video data;
obtaining a question from a user;
providing a prompt to a large-scale language model, the prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, to determine the video data to be provided from among the video data to answer the question;
An information processing method characterized by being executed by a computer.
[Item 6]
storing a plurality of video data;
obtaining a question from a user;
providing a prompt to a large-scale language model, the prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, to determine the video data to be provided from among the video data to answer the question;
A program for causing a computer to execute the following.

1 User terminal 2 Management server

Claims (6)

a video data storage unit for storing a plurality of video data;
a question acquisition unit for acquiring a question from a user;
a playback video determination unit that provides a prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question to a large-scale language model, and determines the video data to be provided in response to the question from among the video data;
An information processing system comprising: 2. The information processing system according to claim 1,
the playback video determination unit includes in the prompt an instruction for creating the video data and a playback order to be provided in response to the question, selects the video data to answer the question from among the video data, and determines the playback order of the selected video data;
An information processing system comprising: 3. The information processing system according to claim 2,
the playback video determination unit further provides the prompt, which includes an instruction to explain a reason for selecting the video data, to the large-scale language model to create the reason;
An information processing system comprising: 3. The information processing system according to claim 2,
a playback processing unit that plays back the video data in the playback order;
An information processing system comprising: storing a plurality of video data;
obtaining a question from a user;
providing a prompt to a large-scale language model, the prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, to determine the video data to be provided from among the video data to answer the question;
An information processing method characterized by being executed by a computer. storing a plurality of video data;
obtaining a question from a user;
providing a prompt to a large-scale language model, the prompt including information about the video data, the question, and an instruction to select the video data to be provided in response to the question, to determine the video data to be provided from among the video data to answer the question;
A program for causing a computer to execute the following.

Images