KR102762246B1 - The Method Of Producing Automatic Video Content Based On User Input - Google Patents

Abstract

The present invention relates to a method for automatically producing video content based on user input, and more specifically, to a method for automatically producing video content based on user input, in which a service server inputs user text received from a user terminal into a macro-language model to extract a title, a plurality of subtitles, and a plurality of subtexts, and generates one or more images corresponding to each of the plurality of subtitles and the plurality of subtexts, and then generates a plurality of slides including the title, the subtitles, and the generated images, and when the layout and AI human characters of the AI human video are set through the user terminal, the AI human utters the subtexts and provides the user terminal with an AI human video including the plurality of slides and an editing interface for generating the AI human video.

Description

{The Method Of Producing Automatic Video Content Based On User Input}

The present invention relates to a method for automatically producing video content based on user input, and more specifically, to a method for automatically producing video content based on user input, in which a service server inputs user text received from a user terminal into a macro-language model to extract a title, a plurality of subtitles, and a plurality of subtexts, and generates one or more images corresponding to each of the plurality of subtitles and the plurality of subtexts, and then generates a plurality of slides including the title, the subtitles, and the generated images, and when the layout and AI human characters of the AI human video are set through the user terminal, the AI human utters the subtexts and provides the user terminal with an AI human video including the plurality of slides and an editing interface for generating the AI human video.

According to a survey by the mobile big data platform 'Mobile Index', YouTube was the most used app by Koreans from December 2023 to March 2024, and YouTube's monthly unique users in December 2023 were 45.65 million. Recently, video content has been gaining popularity on SNS, and video content is also produced and uploaded to SNS for corporate advertisements or election promotions. In this way, video content can effectively convey messages or leave them in the memory by arousing the concentration and interest of viewers.

Meanwhile, the current video production process can be time-consuming and require specialized knowledge and practice. Therefore, non-experts may find video production very difficult. In this situation, a service that automatically produces professional video content when the user inputs simple content is needed. Therefore, when the user inputs text, AI automatically generates related images, and automatically produces a video in which an AI human speaks a script, including the images, and then provides it to the user, allowing the user to freely adjust the layout of the video or each element included in the video. This is a technology that can provide a service.

As a prior art in this field, there is a technology regarding a method and system for automatically producing user-experience type video content, such as Korean Patent No. 10-2267673. However, in the case of the above-mentioned prior art, when the user selects a template, a method is suggested for producing a video by receiving photos, text, and music from the user and providing the video to the user, but as mentioned above, there is no disclosure at all about a method for automatically producing a video including an image generated through AI and an AI human speaking a script and providing a service that allows the user to freely edit it. Therefore, a technology that can solve this problem is in demand.

Republic of Korea Patent No. 10-2267673 (June 16, 2021)

The present invention relates to a method for automatically producing video content based on user input, and more specifically, to a method for automatically producing video content based on user input, wherein a service server inputs user text received from a user terminal into a macro language model to extract a title, a plurality of subtitles, and a plurality of subtexts, generates one or more images corresponding to each of the plurality of subtitles and the plurality of subtexts, and then generates a plurality of slides including the title, the subtitles, and the generated images, and when the layout and AI human characters of the AI human video are set through the user terminal, the AI human utters the subtexts and provides the AI human video including the plurality of slides and an editing interface for generating the AI human video to the user terminal.

In order to solve the above-described problem, in one embodiment of the present invention, a user input-based automatic video content production method performed in a computing system including one or more processors and one or more memories is provided, the method including: a data automatic generation step of extracting a title, a plurality of subtitles and a plurality of subtexts based on user text input from a user terminal and generating one or more generated images related to the plurality of subtitles and the plurality of subtexts; a slide automatic generation step of generating a plurality of slides including the title, the subtitles, and the generated images; and an AI human image generation step of generating an AI human image including an AI human uttering the subtext and the slides, and providing an editing interface to the user terminal.

In one embodiment of the present invention, the AI human image generation step may include a layout setting step of providing a layout adjustment interface for adjusting the size and position of a title, subtitle, and generated image included in the AI human image to a user terminal; an AI human character setting step of setting information for an AI human character to be implemented as a plurality of images according to a user input; and an editing interface providing step of providing an editing interface for generating the AI human image to the user terminal.

In one embodiment of the present invention, the layout adjustment interface includes a title layer that displays a title for the entire video; a subtitle layer that displays a subtitle for each slide; a generated image layer that displays a generated image automatically placed on each slide; an AI human layer that displays an AI human that utters a subtext of each slide; and a subtitle layer that displays the subtext as a subtitle of an AI human video; and the positions and sizes of the title layer, the subtitle layer, the generated image layer, the AI human layer, and the subtitle layer can be adjusted through the layout adjustment interface.

In one embodiment of the present invention, the data automatic generation step may include a user text reception step of receiving user text related to a title or subject of an AI human image from a user terminal; a subtext extraction step of inputting the user text into a large language model inside or outside a service server to extract a title, a plurality of subtitles and a plurality of subtexts to be included in a slide of the AI human image; and an image generation step of inputting each of the plurality of subtitles and the plurality of subtexts into a deep learning-based image generation model inside or outside the service server to generate one or more generated images corresponding to each of the plurality of subtitles and the plurality of subtexts.

In one embodiment of the present invention, the automatic slide generation step can provide a plurality of layout templates in which the title, the subtitle, and the generated image are arranged in different layouts to a user terminal, and generate a plurality of slides by arranging the title, the subtitle, and the generated image in the form of a layout template selected from the user terminal.

In one embodiment of the present invention, the subtext is a script to be spoken by the AI human, the editing interface includes a preview layer that provides a preview of the generated AI human image; and a script layer that provides an editing function for the subtext; and the script layer includes a plurality of subtext layers corresponding to each of the subtexts, each of which displays a plurality of subtexts and provides an editing function for the corresponding subtexts; and in the subtext layer, when the subtext included in the corresponding subtext layer is converted into a voice using TTS, the playback time of the voice can be displayed.

In one embodiment of the present invention, a gesture input layer can be displayed in the subtext layer, and the gesture input layer includes a gesture setting interface that can set the gesture type and expression location of the AI human character in the AI human image for the subtext included in the corresponding subtext layer, and when the gesture type and expression location for the subtext included in the corresponding subtext layer are set by the gesture input layer, in the summary block for the corresponding subtext, information about the corresponding gesture can be overlaid and displayed at a detailed location of the summary block corresponding to the set expression location.

In one embodiment of the present invention, the preview layer includes a preview playback layer that displays icons related to playback operations including playback and stopping of AI human images; a time series summary layer that summarizes and displays information according to the time series of the AI human images displayed in the preview playback layer; and the time series summary layer may include a timeline layer in which the viewpoint of the image of the preview playback layer moves according to the movement of the included manipulation axis element.

In one embodiment of the present invention, the editing interface further includes an AI human character selection layer capable of selecting the number and model of AI human characters in the AI human image; and an AI human character setting layer capable of setting the style, angle, pose, size, and position of each AI human character; and the AI human character setting layer can be displayed according to a selection input of an AI human character displayed in an AI human image played in the preview playback layer or a selection input in a summary block corresponding to a subtext.

According to one embodiment of the present invention, a user can effectively receive an AI human image by simply inputting text.

According to one embodiment of the present invention, it is possible to achieve the effect of generating subtitles and subtexts by inputting user text received from a user terminal into a large language model.

According to one embodiment of the present invention, an effect of automatically generating an image corresponding to each subtitle and subtext can be achieved through a deep learning-based image generation model.

According to one embodiment of the present invention, a user can have the effect of setting a layout for a title, subtitle, and generated image included in an AI human image through a layout template provided to a user terminal.

According to one embodiment of the present invention, a user can effectively edit the position and size of a title, subtitle, generated image, AI human, and subtitle included in an AI human video through a layout adjustment interface.

According to one embodiment of the present invention, a user can effectively edit each layer included in an AI human image through an editing interface.

According to one embodiment of the present invention, it is possible to achieve the effect of generating an AI human image in which an AI human speaks the subtext by converting the subtext into TTS.

According to one embodiment of the present invention, a plurality of slides including a title, a subtitle, and a generated image are automatically generated, and an effect can be achieved where a user can edit each slide through an editing interface.

According to one embodiment of the present invention, it is possible to achieve the effect of automatically generating an AI human image including a title, a subtitle, a subtitle, a generated image, and an AI human.

According to one embodiment of the present invention, it is possible to achieve the effect of generating an AI human according to a user's input regarding information on the AI human character.

According to one embodiment of the present invention, a user can set a gesture for each of a plurality of AI human characters in an AI human image, and information about the gesture set by the user can be displayed in a summary block for the corresponding AI human image.

According to one embodiment of the present invention, a user can change the playback time of an AI human image by moving an element displayed on a time axis.

According to one embodiment of the present invention, a script for each subtext and information on an AI human character can be input and modified, and an effect of checking in real time a regenerated AI human image reflecting the input and modified information can be achieved.

According to one embodiment of the present invention, a user can set a background image, a background video, and a background sound source in an AI human video, and can achieve the effect of checking an AI human video regenerated with the set background reflected in real time.

Figure 1 schematically illustrates the internal configuration of a service server according to one embodiment of the present invention.
Figure 2 schematically illustrates the internal configuration of an automatic data generation unit according to one embodiment of the present invention.
Figure 3 schematically illustrates the execution process of the automatic data generation step according to one embodiment of the present invention.
FIG. 4 schematically illustrates the execution process of the data automatic generation step, the slide automatic generation step, and the AI human image generation step according to one embodiment of the present invention.
FIG. 5 schematically illustrates the execution process of a method for automatically producing video content based on user input according to one embodiment of the present invention.
Figure 6 schematically illustrates a layout template according to one embodiment of the present invention.
Figures 7 to 14 illustrate an editing interface capable of editing AI human images according to one embodiment of the present invention.
FIG. 15 exemplarily illustrates the internal configuration of a computing device according to one embodiment of the present invention.

Hereinafter, various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a general understanding of one or more aspects. It will be recognized, however, by one skilled in the art that such aspect(s) may be practiced without these specific details. The following description and the annexed drawings set forth specific exemplary aspects of one or more aspects in detail. It should be understood, however, that these aspects are exemplary and that any of the various methods of the principles of the various aspects may be utilized, and the description is intended to encompass all such aspects and their equivalents.

Also, terms including ordinal numbers such as first, second, etc. may be used to describe various components, but the components are not limited by the terms. The terms are only used to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and/or includes a combination of a plurality of related described items or any item among a plurality of related described items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant technology, and shall not be interpreted in an ideal or overly formal meaning unless explicitly defined in the embodiments of the present invention.

The "user terminal" mentioned below can be implemented as a computer or portable terminal that can access a server or other terminal via a network. Here, the computer includes, for example, a notebook, desktop, laptop, etc. equipped with a WEB Browser, and the portable terminal can include, for example, all kinds of handheld-based wireless communication devices such as a wireless communication device that ensures portability and mobility, a smart phone, a PCS (Personal Communication System), a GSM (Global System for Mobile communications), a PDC (Personal Digital Cellular), a PHS (Personal Handyphone System), a PDA (Personal Digital Assistant), an IMT (International Mobile Telecommunication)-2000, a CDMA (Code Division Multiple Access)-2000, a W-CDMA (W-Code Division Multiple Access), a Wibro (Wireless Broadband Internet), a BLE beacon (Bluetooth Low Energy Beacon) terminal, etc. Additionally, the “network” may be implemented as a wired network such as a Local Area Network (LAN), a Wide Area Network (WAN), or a Value Added Network (VAN), or any type of wireless network such as a mobile radio communication network or a satellite communication network.

1. Method for automatic video content creation based on user input

Below, the composition of the automatic video content production method based on user input and the execution steps performed by each composition are described.

Figure 1 schematically illustrates the internal configuration of a service server (1) according to one embodiment of the present invention.

As illustrated in FIG. 1, the service server (1) includes: a data automatic generation unit (100) that performs a data automatic generation step of extracting a title, a plurality of subtitles and a plurality of subtexts based on a user text input from a user terminal and generating one or more generated images related to the plurality of subtitles and the plurality of subtexts; a slide automatic generation unit (200) that performs a slide automatic generation step of generating a plurality of slides including the title, the subtitles, and the generated images; and an AI human image generation unit (300) that performs an AI human image generation step of generating an AI human image including an AI human uttering the subtext and the slide, and providing an editing interface to the user terminal.

Specifically, each component included in the service server (1) illustrated in FIG. 1 serves to control the operation of the service server (1) that performs a user input-based automatic video content production method performed in a computing system including one or more processors and one or more memories of the present invention.

More specifically, the data automatic generation unit (100) of the service server (1) can extract a title, a plurality of subtitles, and a plurality of subtexts based on a user text input from a user terminal, and can generate one or more generated images related to the plurality of subtitles and the plurality of subtexts. In one embodiment of the present invention, the user text includes a text received from the user terminal in relation to a title or subject of an AI human image, the title includes a title for the entire AI human image, the subtitle includes a subtitle that summarizes each slide included in the AI human image, the subtext includes a script to be uttered by the AI human in each of the slides or the AI human image and content displayed in the subtitles of the AI human image, and the generated image can include an image related to each of the plurality of subtitles and the plurality of subtexts. Preferably, one generated image can be derived for each subtitle and subtext. In addition, the data automatic generation step includes a user text reception step of receiving user text related to the title or subject of the AI human image from a user terminal; a subtext extraction step of inputting the user text into a large language model inside or outside the service server (1) to extract a title, a plurality of subtitles and a plurality of subtexts to be included in a slide of the AI human image; and an image generation step of inputting each of the plurality of subtitles and the plurality of subtexts into a deep learning-based image generation model inside or outside the service server (1) to generate one or more generated images corresponding to each of the plurality of subtitles and the plurality of subtexts.

The slide automatic generation unit (200) of the above service server (1) can generate a plurality of slides including the title, the subtitle, and the generated image. In one embodiment of the present invention, the slide automatic generation step can generate a plurality of slides by providing a plurality of layout templates in which the title, the subtitle, and the generated image are arranged in different layouts to a user terminal, and arranging the title, the subtitle, and the generated image in the form of a layout template selected from the user terminal.

The AI human image generation unit (300) of the above service server (1) can generate an AI human image including an AI human uttering the subtext and the above slide, and provide an editing interface to a user terminal. The AI human image generation unit (300) includes: a layout setting unit (11) that performs a layout setting step of providing a layout adjustment interface for adjusting the size and position of a title, subtitle, and generated image included in the AI human image to the user terminal; an AI human character setting unit (12) that performs an AI human character setting step of setting information for an AI human character to be implemented as a plurality of images according to a user's input; an editing interface providing unit (13) that performs an editing interface providing step of providing an editing interface for generating an AI human image to the user terminal; And it includes a video content production unit (14) that performs a video content production step of producing an AI human video edited from a user terminal through the above editing interface into video content and providing the final AI human video to the user terminal.

In one embodiment of the present invention, the layout adjustment interface may include a title layer that displays a title for the entire video; a subtitle layer that displays a subtitle for each slide; a generated image layer that displays a generated image automatically placed on each slide; an AI human layer that displays an AI human that speaks a subtext of each slide; and a subtitle layer that displays the subtext as a subtitle of the AI human video. In addition, the editing interface may include a preview layer that provides a preview of the generated AI human video; and a script layer that provides an editing function for the subtext; and the script layer may include a plurality of subtext layers corresponding to each of the subtexts, each of which displays a plurality of subtexts and provides an editing function for the corresponding subtexts; and in the subtext layer, when the subtext included in the corresponding subtext layer is converted into a voice using TTS, the playback time of the voice may be displayed.

In one embodiment of the present invention, the user terminal may correspond to a terminal used by a user who uses a function related to the user input-based automatic video content production method, and the user may access the service server (1) through the user terminal and use a function or service provided by the service server (1). At this time, the user terminal is a computing device including one or more processors and one or more memories, and may include one or more of a laptop, a tablet, a smartphone, and a desktop. In addition, the components of the service server (1) described above are essential elements for implementing the present invention, and may further include a database, etc., without being limited thereto.

Figure 2 schematically illustrates the internal configuration of an automatic data generation unit (100) according to one embodiment of the present invention.

As illustrated in FIG. 2, the data automatic generation unit (100) includes a user text reception unit (110) that performs a user text reception step of receiving user text related to a title or subject of an AI human image from a user terminal; a subtext extraction unit (120) that inputs the user text into a large language model inside or outside the service server (1) and performs a subtext extraction step of extracting a title, a plurality of subtitles, and a plurality of subtexts to be included in a slide of the AI human image; and an image generation unit (130) that inputs each of the plurality of subtitles and the plurality of subtexts into a deep learning-based image generation model inside or outside the service server (1) and performs an image generation step of generating one or more generated images corresponding to each of the plurality of subtitles and the plurality of subtexts.

Preferably, in the user text receiving step, when a user transmits a user text including a title or subject of a video to be produced as an AI human video through a user terminal to the service server (1), the service server (1) receives the user text, automatically generates an AI human video related to the user text, and provides the AI human video and an editing interface to the user terminal, and the user can edit the AI human video through the editing interface to generate a final AI human video.

The above subtext extraction unit (120) can input the user text received from the user terminal into a large language model inside or outside the service server (1) to extract a title, a plurality of subtitles, and a plurality of subtexts. The above title and the plurality of subtitles may correspond to a title and subtitles to be included in a slide of an AI human image, and the plurality of subtexts may correspond to a script to be spoken by an AI human included in the AI human image, and in one embodiment of the present invention, the plurality of subtexts may be used as subtitles displayed on the AI human image.

The image generation unit (130) can input each of the plurality of subtitles and the plurality of subtexts extracted from the large language model into a deep learning-based image generation model inside or outside the service server (1) to generate one or more generated images. The generated images correspond to images corresponding to each of the plurality of subtitles and the plurality of subtexts, and preferably, one generated image can be derived per subtitle and subtext, and one subtitle, one subtext, and one generated image for one content can be included in each slide in the AI human image.

Figure 3 schematically illustrates the execution process of the automatic data generation step according to one embodiment of the present invention.

As illustrated in Fig. 3, user text can be input into a macro language model to extract subtitles and subtexts, and the subtitles and subtexts can be input into an image generation model to generate generated images.

Specifically, the user text received from the user terminal can be input into a macro language model inside or outside the service server (1) to extract a plurality of subtitles and a plurality of subtexts, and preferably, a title, a plurality of subtitles, and a plurality of subtexts can be extracted. If the user text is one sentence, the content can be expanded from the user text to extract the content to be included in the AI human image, and if the user text corresponds to a plurality of sentences or documents, the content included in the user text can be summarized or organized to extract the content to be included in the AI human image. In one embodiment of the present invention, when the service server (1) inputs the user text into the macro language model, it can request extraction of the title of the AI human image, the subtitle of each slide included in the AI human image, and the subtext to be uttered by the AI human for each slide.

In addition, each of the plurality of subtitles and each of the plurality of subtexts extracted through the macro language model may be input into a deep learning-based image generation model inside or outside the service server (1) to generate one or more generated images corresponding to each of the plurality of subtitles and each of the plurality of subtexts. Preferably, one generated image may be derived per subtitle and each of the subtexts, and in one embodiment of the present invention, when the service server (1) inputs each of the subtitles and each of the subtexts into the image generation model, it may request generation of one image per subtitle and each of the subtexts. The title, the plurality of subtitles, and the plurality of subtexts extracted through the macro language model and the one or more generated images generated through the image generation model may be included in each of a plurality of slides and displayed in the AI human image. Therefore, the data automatic generation unit (100) may automatically generate data through the macro language model and the image generation model.

If the data included in the AI human video cannot be automatically generated, the user may have difficulty in directly inputting the title, script, and image to be included in the video content. However, in the automatic video content production method based on user input of the present invention, since the title, multiple subtitles, multiple subtexts, and one or more images of the AI human video can be automatically generated through the data automatic generation step, the user can be provided with higher quality video content even if he or she inputs only a simple topic regarding the video content to be produced.

FIG. 4 schematically illustrates the execution process of the data automatic generation step, the slide automatic generation step, and the AI human image generation step according to one embodiment of the present invention.

In outline, (a) of FIG. 4 illustrates the execution process of the data automatic generation unit (100) that performs the data automatic generation step, (b) of FIG. 4 illustrates the execution process of the slide automatic generation unit (200) that performs the slide automatic generation step, and (c) of FIG. 4 illustrates the execution process of the AI human image generation unit (300) that performs the AI human image generation step.

Specifically, the data automatic generation unit (100) illustrated in (a) of FIG. 4 performs a data automatic generation step of extracting a title, a plurality of subtitles, and a plurality of subtexts based on a user text input from a user terminal, and generating one or more generated images related to the plurality of subtitles and the plurality of subtexts. When receiving a user text from a user terminal, the data automatic generation unit (100) can automatically generate a title, a plurality of subtitles, a plurality of subtexts, and one or more generated images to be included in an AI human image.

More specifically, the user text can be input into a large language model inside or outside the service server (1) to extract the title, the plurality of subtitles and the plurality of subtexts, and each of the plurality of subtitles and the plurality of subtexts can be input into a deep learning-based image generation model inside or outside the service server (1) to generate one or more generated images corresponding to each of the plurality of subtitles and the plurality of subtexts.

The slide automatic generation unit (200) illustrated in (b) of Fig. 4 performs a slide automatic generation step of generating a plurality of slides including the title, the subtitle, and the generated image. The slide automatic generation unit (200) can automatically generate a plurality of slides including the title generated by the data automatic generation unit (100), the plurality of subtitles, and the one or more generated images.

More specifically, a plurality of layout templates in which the title, the subtitle, and the generated image are arranged in different layouts are provided to the user terminal, and the title, the subtitle, and the generated image are arranged in the form of the layout template selected from the user terminal to generate a plurality of slides. Preferably, the title, the subtitle, and the generated image arranged in the slide through the layout template in the slide automatic generation unit (200) can be directly adjusted by the user through a layout adjustment interface provided to the user terminal in the layout setting step included in the subsequent AI human image generation step.

In one embodiment of the present invention, the layout adjustment interface includes a title layer that displays a title for the entire video; a subtitle layer that displays a subtitle for each slide; a generated image layer that displays a generated image automatically placed on each slide; an AI human layer that displays an AI human that utters a subtext of each slide; and a subtitle layer that displays the subtext as a subtitle of an AI human video; and the positions and sizes of the title layer, the subtitle layer, the generated image layer, the AI human layer, and the subtitle layer can be adjusted through the layout adjustment interface.

Preferably, the slide may include each slide in which the subtitle and generated image displayed on the screen change as the screen changes in the AI human video, and each slide may include a subtitle, a generated image, and an AI human, so that the generated image related to the subtitle may be displayed, and the AI human may speak a script related to the subtitle. In addition, the content of the script spoken by the AI human may be displayed as is on the screen and utilized as a subtitle of the AI human video.

The AI human image generation unit (300) illustrated in (c) of Fig. 4 generates an AI human image including an AI human uttering the subtext and the slide, and performs an AI human image generation step of providing an editing interface to a user terminal. The AI human image generation unit (300) can generate an AI human image including a plurality of slides generated by the slide automatic generation unit (200).

More specifically, by providing a layout adjustment interface for adjusting the size and position of the title, subtitle, and generated image included in the AI human image to the user terminal, the user can adjust the size and position of the title, subtitle, and generated image of the slide arranged according to the layout template. In addition, information on the AI human character is set according to the user's input, so that an AI human to be included in the AI human image can be created, and an editing interface for creating the AI human image is provided to the user terminal, so that the user can edit each layer of the AI human image, and a final edited AI human image can be created from the user terminal through the editing interface.

In one embodiment of the present invention, the editing interface includes a preview layer that provides a preview of the generated AI human image; and a script layer that provides an editing function for the subtext. The script layer includes a plurality of subtext layers corresponding to each of the subtexts, each of which displays a plurality of subtexts and provides an editing function for the corresponding subtexts; and in the subtext layer, when the subtext included in the corresponding subtext layer is converted into a voice using TTS, the playback time of the voice can be displayed. In addition, a gesture input layer may be displayed in the subtext layer, and the gesture input layer may include a gesture setting interface capable of setting the gesture type and expression location of the AI human character in the AI human image for the subtext included in the corresponding subtext layer, and when the gesture type and expression location for the subtext included in the corresponding subtext layer are set by the gesture input layer, information about the gesture may be overlaid and displayed in the detailed location of the summary block corresponding to the set expression location in the summary block for the corresponding subtext. The preview layer may include a preview playback layer in which icons related to playback operations including playback and stop of the AI human image are displayed; a time series summary layer in which information according to the time series of the AI human image displayed in the preview playback layer is summarized and displayed; and the time series summary layer may include a timeline layer in which the viewpoint of the image of the preview playback layer moves as the included manipulation axis element is moved. In addition, the editing interface may further include an AI human character selection layer capable of selecting the number and model of AI human characters in the AI human video; and an AI human character setting layer capable of setting the style, angle, pose, size, and position of each AI human character. The AI human character setting layer may be displayed according to a selection input of an AI human character displayed in an AI human video played in the preview playback layer, or a selection input in a summary block corresponding to a subtext.

FIG. 5 schematically illustrates the execution process of a method for automatically producing video content based on user input according to one embodiment of the present invention.

As shown in Fig. 5, the service server (1) can receive user text from a user terminal, generate an AI human image, and then provide the AI human image to the user terminal again.

Specifically, the service server (1) can input the user text received from the user terminal into a large language model inside or outside the service server (1) to extract a subtitle and a subtext. At this time, in one embodiment of the present invention, the title of the AI human image can also be extracted. Thereafter, the subtitle and the subtext can be input again into an image generation model to generate one or more generated images corresponding to each of the subtitle and the subtext. At this time, in one embodiment of the present invention, the image generation model can correspond to a deep learning-based image generation model inside or outside the service server (1). Thereafter, a plurality of slides including the subtitle and the generated images can be generated, and an AI human image based on the plurality of slides can be generated. At this time, in the step of generating the plurality of slides, a plurality of layout templates in which the title, the subtitle, and the generated image are arranged in different layouts may be provided to the user terminal, and the title, the subtitle, and the generated image may be arranged in the form of the layout template selected from the user terminal to generate the plurality of slides. In addition, in the step of generating the AI human image, a layout adjustment interface for adjusting the size and position of the title, subtitle, and generated image included in the AI human image may be provided to the user terminal, and the plurality of slides may be modified in the form of the adjusted layout from the user terminal, and when information of the AI human character is set according to the input of the user terminal, the corresponding AI human may be generated and included in each slide. After an AI human video is created based on a plurality of modified slides and the AI human, the AI human video and an editing interface for creating the AI human video are provided to a user terminal, so that editing of the AI human video can be performed on the user terminal, and the edited AI human video can be produced as video content from the user terminal through the editing interface, so that the final AI human video can be provided to the user terminal.

Figure 6 schematically illustrates a layout template according to one embodiment of the present invention.

Schematically, (a), (b), (c), and (d) of FIG. 6 illustrate examples of layout templates.

Specifically, the layout template includes a plurality of layout templates in which the title, the subtitle, and the generated image are arranged in different layouts, and after the layout template is provided to a user terminal, the title, the subtitle, and the generated image can be arranged in the form of a layout template selected from the user terminal to generate a plurality of slides.

As illustrated in FIG. 6, (a), (b), and (c) of FIG. 6 correspond to layout templates in which the screen is long in the horizontal direction, and (d) of FIG. 6 corresponds to a layout template in which the screen is long in the vertical direction. Preferably, the layout template may include a title, a subtitle, a generated image, an AI human, and a subtitle. However, in one embodiment of the present invention, as in FIG. 6 (c), the AI human may not be displayed and the generated image may be displayed as a background image of the AI human video, and as in FIG. 6 (d), the generated image may not be displayed and an AI human speaking a script may act as a presenter in the center of the AI human video. In addition, in the AI human video, not only the AI human and the generated image, but also the title, subtitle, or subtitle may not be displayed.

In one embodiment of the present invention, a layout adjustment interface may be provided to a user terminal in a form similar to the layout template illustrated in FIG. 6. The layout adjustment interface includes a title layer for displaying a title for the entire video; a subtitle layer for displaying a subtitle for each slide; a generated image layer for displaying a generated image automatically arranged for each slide; an AI human layer for displaying an AI human that utters a subtext of each slide; and a subtitle layer for displaying the subtext as a subtitle of an AI human video; and the positions and sizes of the title layer, the subtitle layer, the generated image layer, the AI human layer, and the subtitle layer may be adjusted through the layout adjustment interface. For example, referring to FIG. 6 (a), in the layout adjustment interface, a title may correspond to a title layer, a subtitle may correspond to a subtitle layer, a generated image may correspond to a generated image layer, an AI human may correspond to an AI human layer, and a subtitle may correspond to a subtitle layer.

Preferably, a layout template of an AI human image in which a title, a subtitle, a generated image, an AI human, and subtitles are displayed can be produced in various forms as well as in (a), (b), (c), and (d) of Fig. 6 and provided to a user terminal, and on the user terminal, after setting the arrangement of the title, subtitle, generated image, AI human, and subtitle through the layout template, the size and form of the title, subtitle, generated image, AI human, and subtitle can be adjusted again through the provided layout adjustment interface.

2. Method for providing AI human image generation interface

In the description of the above-mentioned drawings 1 to 6, the configuration of the automatic video content production method based on user input and the execution steps performed by each configuration are described. Hereinafter, an embodiment of an editing interface provided to a user terminal by the editing interface provision unit (13) included in the service server (1) through the AI human video production interface provision method will be described in detail.

Figures 7 to 14 illustrate an editing interface capable of editing AI human images according to one embodiment of the present invention.

As illustrated in FIGS. 7 to 14, the editing interface includes a preview layer (L1) that provides a preview of the generated AI human image; and a script layer (L2) that provides an editing function for the subtext; and the script layer (L2) includes a plurality of subtext layers (L3) corresponding to each of the subtexts, in which each of the plurality of divided subtexts is displayed and provides an editing function for the corresponding subtext; and in the subtext layer (L3), when the subtext included in the corresponding subtext layer (L3) is converted into voice by TTS, the playback time of the voice can be displayed.

In addition, the above editing interface further includes an AI human character selection layer (L6) capable of selecting the number and model of AI human characters in the AI human video; and an AI human character setting layer (L7) capable of setting the style, angle, pose, size, and position of each AI human character; and the AI human character setting layer (L7) can be displayed according to a selection input of an AI human character displayed in an AI human video played in the preview playback layer (L8) or a selection input in a summary block corresponding to a subtext.

In addition, a gesture input layer may be displayed in the subtext layer (L3), and the gesture input layer may include a gesture setting interface that can set the gesture type and expression location of the AI human character in the AI human image for the subtext included in the subtext layer (L3).

In addition, the preview layer (L1) includes a preview playback layer (L8) that displays icons related to playback operations including playback and stop of AI human images; a time series summary layer (L9) that summarizes and displays information according to the time series of the AI human images displayed in the preview playback layer (L8); and the time series summary layer (L9) may include a timeline layer (L10) in which the viewpoint of the image of the preview playback layer (L8) moves as the included manipulation axis element moves.

In addition, the time series summary layer (L9) includes a script summary information layer that displays information about the AI human image at a corresponding point in time according to the time series, and the script summary information layer includes each summary block corresponding to each subtext, and each summary block may include information about the AI human character and script for the corresponding subtext.

In addition, when the gesture type and expression location for the subtext included in the corresponding subtext layer (L3) are set by the gesture input layer, information about the corresponding gesture can be overlaid and displayed in the summary block for the corresponding subtext at the detailed location of the summary block corresponding to the set expression location.

In addition, the above editing interface further includes a background setting layer capable of setting background information for at least one of a background image, a background video, and a background sound source of the AI human video, and in the preview playback layer (L8), an AI human video with background information set by the background setting layer reflected can be displayed.

As illustrated in Fig. 7, the editing interface may include a preview layer (L1) corresponding to the L1 layer and a script layer (L2) corresponding to the L2 layer. Specifically, the preview layer (L1) may provide a preview of the AI human image, and the script layer (L2) may provide an editing function while displaying information on each of a plurality of subtexts that generate the AI human image.

In addition, the AI human image may include a title, a subtitle, and a generated image, and the layout of the title, the subtitle, and the generated image may be set based on a selection of a user terminal for a plurality of layout templates in the slide automatic generation step. The title, the subtitle, and the generated image are included in the title, the plurality of subtitles, and the plurality of generated images generated through the data automatic generation step, and preferably, one title, one subtitle, and one generated image may be displayed in one slide. In addition, the AI human image may further display a subtitle, and the layout of the title, the subtitle, the generated image, and the subtitle may be set based on a selection of a user terminal for a plurality of layout templates in the slide automatic generation step. Preferably, each subtext extracted in the above data automatic generation step can be displayed in the form of a subtitle within the AI human image, and the subtext displayed in the subtext layer (L3) illustrated in FIG. 7 can correspond to a script spoken by the AI human in the AI human image, and at the same time, can correspond to a subtitle displayed at the bottom of the AI human image.

The L2 layer of Fig. 8 illustrates a script layer (L2). As illustrated, the script layer (L2) may include a plurality of subtext layers (L3) that display each of a plurality of subtexts into which text is divided, while providing editing functions for the corresponding subtexts.

The subtext layer (L3) may include a subtext display layer (L4) that displays subtext for the corresponding subtext, and the user may modify the AI human image for the corresponding subtext by modifying the subtext displayed in the subtext display layer (L4). For example, the subtext may correspond to a script spoken by an AI human character in the AI human image, and the user may modify the content of the script spoken by the AI human character by modifying the subtext.

Meanwhile, when a subtext is modified by a user terminal, the service server (1) can regenerate the voice of the modified subtext converted to TTS and regenerate and display an AI human image based on the voice.

Additionally, the service server (1) can display information about the playback time of the regenerated voice in the subtext layer (L3) for the modified subtext.

In one embodiment of the present invention, when the voice playback time for a subtext modified by a user exceeds a preset time interval, the service server (1) can divide the modified subtext into multiple pieces and regenerate an AI human image for each of the divided subtexts.

Meanwhile, the subtext layer (L3) for each of the plurality of subtexts may further include an E2 element capable of playing a voice for the corresponding subtext. Specifically, the user can listen to a voice for the subtext by selecting and inputting the E2 element.

As described above, the subtext can be generated by dividing the playback time of the voice when the text extracted from the target document file is converted into TTS into preset time intervals, and the E2 element can be an element that indicates the playback time of the voice when the subtext generated in this manner is converted into TTS.

In other words, the information about the playback time of the voice displayed by the E2 element can correspond to the playback time of the AI human video when the AI human video is generated based on the subtext.

In this way, users can intuitively and easily check the playback time of AI human images for each of multiple subtexts.

In one embodiment of the present invention, the preset time interval for dividing text into subtexts may correspond to the maximum playback time for one AI human image that the service server (1) can generate. That is, when the service server (1) converts text into voice using TTS and divides the playback time of the voice into 1-minute intervals to generate multiple subtexts, the service server (1) can generate an AI human image with a maximum length of 1 minute.

The service server (1) can display the voice playback time for a subtext on the user terminal, thereby preventing the user from allowing a single subtext to exceed a preset time interval (1 minute).

Meanwhile, the script layer (L2) may further include an L5 layer capable of listening to the voice of the generated AI human image based on subtext for each of a plurality of subtext layers (L3) included in the script.

As described above, when text extracted from a target document file is converted into voice, an AI human image can be generated for each of a plurality of subtexts having a playback time less than a preset interval, and the voice for each subtext can be heard through the E2 element included in the subtext layer (L3) for the corresponding subtext.

The service server (1) can implement a function of listening to the voice for each of the multiple subtexts implemented through the E2 element described above, as well as a function of listening to the entire voice when the voices for each of the multiple subtexts are merged, through the L5 layer.

Fig. 9 (a) illustrates an AI human character selection layer (L6) that can select the number and model of AI human characters in an AI human image. As illustrated, in the AI human character selection layer (L6), a user can select the number and model type of AI human characters applied in an AI human image.

Fig. 9 (b) illustrates an AI human character setting layer (L7) that can set the style, angle, pose, size, and position of an AI human character in an AI human image. As illustrated, the AI human character setting layer (L7) may be a layer provided to set information on one or more of whether to hide (cover), style, angle, pose, size, and position of an AI human character selected in an AI human character selection layer (L6).

The AI human character setting layer (L7) further includes an E3 element that can change the AI human character for the AI human image for the corresponding subtext, and when the E3 element is selected and input, the AI human character selection layer (L6) can be displayed on the user terminal.

The AI human character setting layer (L7) can be displayed when selecting and inputting an AI human character displayed in an AI human video played in the preview playback layer (L8) or when selecting and inputting a summary block corresponding to a subtext.

As specifically described above, the AI human video can be played in the preview playback layer (L8) as a video in which the AI human character utters a subtext. In this way, when selecting and inputting an AI human character displayed in the AI human video played in the preview playback layer (L8), an AI human character setting layer (L7) can be displayed on the user terminal, and the user can set and change detailed information about the corresponding AI human character.

In addition, AI human images can be generated for each of a plurality of subtexts, and a summary block corresponding to each of a plurality of subtexts can be displayed, and an AI human character setting layer (L7) can be displayed according to a selection input in the summary block for the subtext.

For example, when a user selects and inputs the name of an AI human character displayed in a summary block, an AI human character setting layer (L7) for the corresponding AI human character may be displayed on the user terminal, and the user may set and change detailed information about the corresponding AI human character.

Fig. 10 illustrates a preview layer (L1) that provides a preview of an AI human image. Specifically, the preview layer (L1) may include a preview playback layer (L8) that displays icons related to playback operations, including playback and stopping of the AI human image.

As described above, the user terminal can be provided with an interface for uploading a target document file from the service server (1) and inputting detailed setting information for the AI human character, and the service server (1) can generate an AI human image for the text of the target document file based on the information input by the user terminal.

The preview playback layer (L8) may be a layer in which the AI human image generated in this manner is displayed. Specifically, the preview playback layer (L8) may include an E4 element including an icon that can input playback actions such as play and stop of the AI human image.

The preview layer (L1) may include a time series summary layer (L9) that displays information about the AI human image displayed in the preview playback layer (L8) in a time series summary manner. Specifically, the time series summary layer (L9) may include a timeline layer (L10) for changing the viewpoint of the AI human image displayed in the preview playback layer (L8). Preferably, the timeline layer (L10) may be displayed as a time axis.

As illustrated, the user can change the viewpoint of the AI human image displayed in the preview playback layer (L8) by moving the manipulation axis element (E5) displayed in the timeline layer (L10) left and right. For example, when the manipulation axis element (E5) moves to the right, the playback viewpoint of the AI human image can move backwards.

The time series summary layer (L9) of the preview layer (L1) may include a script summary information layer (L11) that displays information about the AI human image at a given point in time according to the time series. Specifically, the script summary information layer (L11) may include each summary block (B1) corresponding to each subtext.

Preferably, each summary block (B1) may include information about the AI human character and script for the corresponding subtext.

As described above, the service server (1) can divide the text extracted from the target document file into multiple subtexts and generate an AI human image for each of the multiple divided subtexts. The summary block can display information about the AI human image for each of the multiple subtexts generated in this manner.

Specifically, the summary block may include information about the name (model name) of the AI human character in the AI human video for the corresponding subtext, a script for the corresponding subtext, and the playback time of the voice when the corresponding subtext is converted to TTS.

For example, in Fig. 10, a user can check information about an AI human video played from 00:00 to 00:17 seconds through the first summary block (B1), and can check information about an AI human video played from 00:17 to 00:36 seconds through the second summary block (B1).

Meanwhile, when selecting and entering the name of the AI human character displayed in the summary block (B1), the AI human character setting layer (L7) for the corresponding AI human character may be displayed on the user terminal.

In one embodiment of the present invention, the form of each of a plurality of summary blocks can be determined by reflecting information about the playback time of the voice when the subtext is converted into voice as TTS.

For example, the size of each of the multiple summary blocks may be determined in proportion to the playback time of the voice when the subtext is converted into speech via TTS.

Preferably, the timeline layer (L10) may be a time axis displayed horizontally, and the horizontal length of each of the plurality of summary blocks is determined in proportion to the playback time of the voice when the subtext is converted into voice as TTS, so that the user can intuitively recognize the playback length of the AI human video for the subtext corresponding to each summary block through the horizontal length of each summary block.

In addition, the preview playback layer (L8) is a layer in which the AI human image is displayed, and the preview playback layer (L8) may include a title, a subtitle, a generated image, and an AI human. The positions and sizes of the title, the subtitle, the generated image, and the AI human may be adjusted according to an input of the user terminal through the editing interface, and at least one of the title, the subtitle, and the generated image may be deleted from the corresponding slide and AI human image. In addition, subtitles may be further displayed on the preview playback layer (L8) and the AI human image, and preferably, a plurality of subtexts extracted in the data automatic generation step may be displayed in the form of subtitles on the AI human image. In one embodiment of the present invention, the subtitles may be inserted and deleted according to an input of the user terminal through the editing interface, and when the subtitles are inserted, the positions and sizes of the corresponding subtitles may be adjusted.

Meanwhile, as described above, the user can set information for each of the plurality of AI human characters in each of the plurality of AI human images generated for each of the plurality of subtexts. For example, in Fig. 11, the user can create an AI human image in which two or more AI human characters converse by setting different models of the AI human characters in the AI human images for each of the plurality of subtexts.

In such a case, the preview playback layer (L8) may display an AI human image including multiple AI human characters C1 and C2.

In addition, the script summary information layer (L11) may display summary blocks B1 and B2 for each of multiple AI human characters C1 and C2. Specifically, summary block B1 may display information about AI human character C1, and summary block B2 may display information about AI human character C2.

In one embodiment of the present invention, in AI human images for each of a plurality of subtexts, the colors of summary blocks for subtexts for the same AI human character may be set to be the same, and the colors of summary blocks for subtexts for different AI human characters may be set to be different from each other.

For example, in Fig. 11, the colors of summary blocks B1 and B2 for different AI human characters C1 and C2 may be displayed differently. In this way, the service server (1) can display the colors of summary blocks B1 and B2 for different AI human characters C1 and C2 differently, thereby allowing the user to intuitively recognize which AI human character each summary block contains information about.

Meanwhile, as described above, when an AI human character displayed in an AI human video played in the preview playback layer (L8) is selected and input by the user, or a summary block for the corresponding AI human character is selected and input in the script summary information layer (L11), an AI human character setting layer (L7) for the corresponding AI human character may be displayed.

For example, a user can set detailed information about the AI human character C1 by selecting the AI human character C1 displayed in the preview playback layer (L8) or selecting and entering the summary block B1 (preferably the name of the AI human character C1 displayed in the summary block B1) for the AI human character C1 in the script summary information layer (L11).

As shown in Figure 12, a user can set the gestures of an AI human character in an AI human image.

As described above, the script layer (L2) can include a subtext layer (L3) for each of the plurality of subtexts, and the service server (1) can generate an AI human image for each of the plurality of subtexts.

Meanwhile, as illustrated in (a) of FIG. 12, each of the plurality of subtext layers (L3) may include an E6 element for displaying a gesture input layer (L12) that can set a gesture for an AI human character in an AI human image for the corresponding subtext.

When the E6 element is selected and input by the user, the gesture input layer (L12) can be displayed, and the user can set the gesture type and expression location for the AI human character for the corresponding subtext.

Specifically, the type of gesture may include information about which hand the AI human character will use, or what kind of action and facial expression it will make, and the location of gesture expression may refer to information about at what point in the AI human video for the corresponding subtext the AI human character will make the corresponding gesture.

For example, a user can set the AI human character to use the “right hand” in the “before sentence” for subtext.

In this way, by having the AI human character implement the gestures set by the user in the AI human video, a more realistic AI video that matches the user's intention can be implemented.

Meanwhile, as shown in (b) of Fig. 12, when a gesture is set for an AI human character in the gesture input layer (L12), information about the corresponding gesture can be displayed in the summary block (B1) for the corresponding subtext.

Specifically, an E7 element that displays information about the gesture may be overlaid and displayed at a detailed location of a summary block (B1) corresponding to the expression location set for the gesture.

For example, in (a) of Fig. 12, if the expression location of a gesture for a specific subtext is set to “before a sentence” and the type of gesture is set to “right hand,” information about the type of gesture “right hand” can be displayed in the summary block for the subtext at the detailed location (front or left) of the summary block corresponding to “before a sentence” in which the E7 element is located.

In this way, information about the gesture set by the user in the gesture input layer (L12) is displayed corresponding to the summary block (B1) for the corresponding subtext, so that the user can intuitively recognize which gesture was set for which subtext and at which expression location.

Meanwhile, in one embodiment of the present invention, a user can delete the E7 element of the summary block (B1) without reflecting information about the gesture set for the corresponding subtext.

In this way, if a user wants to delete a gesture, he or she can delete the gesture directly from the summary block (B1) for the corresponding subtext without having to reload the gesture input layer (L12) by selecting and inputting the E6 element.

As illustrated in Fig. 13, the service server (1) can provide a background setting layer that can set background information including at least one of a background image, background video, and background sound source of an AI human video to a user terminal.

Specifically, (a) of Fig. 13 is a layer displayed on a user terminal for setting a background image in an AI human image, (b) of Fig. 13 is a layer displayed on a user terminal for setting a background video in an AI human image, and (c) of Fig. 13 is a layer displayed on a user terminal for setting a background sound source in an AI human image.

In this way, users can insert one or more of a background image, background video, or background sound source into the AI human video.

As illustrated in Fig. 14, when background information for an AI human image is set in the background setting layer, an AI human image with the corresponding background information reflected can be displayed in the preview playback layer (L8).

In addition, the user can modify the background information reflected in the AI human image displayed in the preview playback layer (L8). For example, the user can select and input a background image that is overlaid on the AI human image displayed in the preview playback layer (L8) and duplicate, delete, or set the location, etc. of the background image.

FIG. 15 exemplarily illustrates the internal configuration of a computing device (11000) according to one embodiment of the present invention.

The service server (1) mentioned in the description of Fig. 1 may include components of the computing device (11000) illustrated in Fig. 15 described below.

As illustrated in FIG. 15, the computing device (11000) may include at least one processor (11100), memory (11200), peripheral interface (11300), input/output subsystem (I/O subsystem) (11400), power circuit (11500), and communication circuit (11600).

Specifically, the memory (11200) may include, for example, a high-speed random access memory, a magnetic disk, an SRAM, a DRAM, a ROM, a flash memory, or a nonvolatile memory. The memory (11200) may include a software module, a set of instructions, or other various data required for the operation of the computing device (11000).

At this time, access to the memory (11200) from other components such as the processor (11100) or the peripheral interface (11300) may be controlled by the processor (11100). The processor (11100) may be configured as a single or multiple processors, and may include processors in the form of GPUs and TPUs to improve the processing speed.

The peripheral interface (11300) may couple input and/or output peripherals of the computing device (11000) to the processor (11100) and the memory (11200). The processor (11100) may execute software modules or instruction sets stored in the memory (11200) to perform various functions for the computing device (11000) and process data.

The input/output subsystem (11400) can couple various input/output peripheral devices to the peripheral interface (11300). For example, the input/output subsystem (11400) can include a controller for coupling peripheral devices such as a monitor, a keyboard, a mouse, a printer, or a touch screen or a sensor as needed to the peripheral interface (11300). According to another aspect, the input/output peripheral devices can be coupled to the peripheral interface (11300) without going through the input/output subsystem (11400).

The power circuit (11500) may supply power to all or part of the components of the terminal. For example, the power circuit (11500) may include a power management system, one or more power sources such as a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or any other components for power generation, management, or distribution.

The above communication circuit (11600) may enable communication with another computing device by using at least one external port. Alternatively, as described above, the communication circuit (11600) may enable communication with another computing device by transmitting and receiving an RF signal, also known as an electromagnetic signal, including an RF circuit, if necessary.

This embodiment of FIG. 15 is only an example of the computing device (11000), and the computing device (11000) may have some of the components illustrated in FIG. 15 omitted, may further include additional components not illustrated in FIG. 15, or may have a configuration or arrangement that combines two or more components. For example, a computing device for a communication terminal in a mobile environment may further include a touchscreen or a sensor, in addition to the components illustrated in FIG. 15, and the communication circuit (1160) may include a circuit for RF communication of various communication methods (Wi-Fi, 3G, LTE, 5G, 6G, Bluetooth, NFC, Zigbee, etc.). The components that can be included in the computing device (11000) may be implemented as hardware including one or more signal processing or application-specific integrated circuits, software, or a combination of both hardware and software.

The methods according to the embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded on a computer-readable medium. In particular, the program according to the embodiments may be configured as a PC-based program or an application exclusively for mobile terminals. The application to which the present invention is applied may be installed on a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to a request from a user terminal.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone, but those skilled in the art will appreciate that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, a processing device may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

The software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and/or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processing device or for providing instructions or data to the processing device. The software may be standardized and stored or executed in a standardized manner on a network-connected computing device. The software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known to and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, flash memories, etc. Examples of the program commands include not only machine language codes generated by a compiler but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

According to one embodiment of the present invention, a user can effectively receive an AI human image by simply inputting text.

According to one embodiment of the present invention, it is possible to achieve the effect of generating subtitles and subtexts by inputting user text received from a user terminal into a large language model.

According to one embodiment of the present invention, an effect of automatically generating an image corresponding to each subtitle and subtext can be achieved through a deep learning-based image generation model.

According to one embodiment of the present invention, a user can have the effect of setting a layout for a title, subtitle, and generated image included in an AI human image through a layout template provided to a user terminal.

According to one embodiment of the present invention, a user can effectively edit the position and size of a title, subtitle, generated image, AI human, and subtitle included in an AI human video through a layout adjustment interface.

According to one embodiment of the present invention, a user can effectively edit each layer included in an AI human image through an editing interface.

According to one embodiment of the present invention, it is possible to achieve the effect of generating an AI human image in which an AI human speaks the subtext by converting the subtext into TTS.

According to one embodiment of the present invention, a plurality of slides including a title, a subtitle, and a generated image are automatically generated, and an effect can be achieved where a user can edit each slide through an editing interface.

According to one embodiment of the present invention, it is possible to achieve the effect of automatically generating an AI human image including a title, a subtitle, a subtitle, a generated image, and an AI human.

According to one embodiment of the present invention, it is possible to achieve the effect of generating an AI human according to a user's input regarding information on the AI human character.

According to one embodiment of the present invention, a user can set a gesture for each of a plurality of AI human characters in an AI human image, and information about the gesture set by the user can be displayed in a summary block for the corresponding AI human image.

According to one embodiment of the present invention, a user can change the playback time of an AI human image by moving an element displayed on a time axis.

According to one embodiment of the present invention, a script for each subtext and information on an AI human character can be input and modified, and an effect of checking in real time a regenerated AI human image reflecting the input and modified information can be achieved.

According to one embodiment of the present invention, a user can set a background image, a background video, and a background sound source in an AI human video, and can achieve the effect of checking an AI human video regenerated with the set background reflected in real time.

Although the embodiments have been described above by way of limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, appropriate results can be achieved even if the described techniques are performed in a different order from the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or are replaced or substituted by other components or equivalents. Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the following claims.

Claims (9)

A method for automatically producing video content based on user input performed in a computing system including one or more processors and one or more memories,
A data automatic generation step for extracting a title, a plurality of subtitles and a plurality of subtexts based on user text input from a user terminal and generating one or more generated images related to the plurality of subtitles and the plurality of subtexts;
A slide automatic generation step for generating a plurality of slides including the title, the subtitle, and the generated image; and
A method for automatically producing video content based on user input, comprising: an AI human video generation step of generating an AI human video including an AI human uttering the subtext and the slide, and providing an editing interface to a user terminal.
In claim 1,
The above AI human image generation step is,
A layout setting step that provides a layout adjustment interface to the user terminal to adjust the size and position of the title, subtitle, and generated image included in the AI human video;
An AI human character setting step in which information is set for an AI human character to be implemented as multiple images according to the user's input; and
A method for automatically producing video content based on user input, comprising: an editing interface providing step for providing an editing interface for creating AI human video using a user terminal;
In claim 2,
The above layout adjustment interface is,
A title layer that displays the title for the entire video;
A subtitle layer that displays subtitles for each slide;
A generated image layer that displays generated images automatically placed on each slide;
AI human layer showing AI human uttering subtext of each slide; and
Includes a subtitle layer that displays the above subtext as a subtitle of the AI human image;
A method for automatically producing video content based on user input, wherein the position and size of a title layer, a subtitle layer, a generated image layer, an AI human layer, and a subtitle layer can be adjusted through the above layout adjustment interface.
In claim 1,
The above data automatic generation step is,
A user text receiving step for receiving user text related to the title or topic of an AI human image from a user terminal;
A subtext extraction step for extracting a title, multiple subtitles, and multiple subtexts to be included in the slide of the AI human image by inputting the above user text into a large language model inside or outside the service server; and
A method for automatically producing video content based on user input, comprising: an image generation step of inputting each of the plurality of subtitles and the plurality of subtexts into a deep learning-based image generation model inside or outside a service server, and generating one or more generated images corresponding to each of the plurality of subtitles and the plurality of subtexts.
In claim 1,
The above slide auto-generation step is,
Providing a plurality of layout templates in which the title, the subtitle, and the generated image are arranged in different layouts to the user terminal,
A method for automatically producing video content based on user input, which generates a plurality of slides by arranging the title, the subtitle, and the generated image in the form of a layout template selected from the user terminal.
In claim 1,
The above subtext is a script for the AI human to speak.
The above editing interface is,
A preview layer that provides a preview of the generated AI human image; and
A script layer that provides editing functions for the above subtext;
The above script layer is,
A plurality of subtext layers corresponding to each of the subtexts, each of which is displayed and provides an editing function for the subtext;
A method for automatically producing video content based on user input, wherein, in the subtext layer above, when the subtext included in the subtext layer is converted into voice using TTS, the voice playback time is displayed. In claim 6,
The above subtext layer can display a gesture input layer,
The above gesture input layer is,
Includes a gesture setting interface that can set the type of gesture and the expression location of the AI human character in the AI human image for the subtext included in the corresponding subtext layer.
In the case where the gesture type and expression location for the subtext included in the corresponding subtext layer are set by the above gesture input layer,
In the summary block for that subtext,
A method for automatically producing video content based on user input, in which information about a corresponding gesture is overlaid and displayed on the detailed location of a summary block corresponding to a set expression location. In claim 6,
The above preview layer is,
A preview playback layer that displays icons related to playback actions, including playing and stopping AI human images; and
Includes a time series summary layer that summarizes and displays information according to the time series of the AI human image displayed in the above preview playback layer;
The above time series summary layer is,
A method for automatically producing video content based on user input, comprising a timeline layer in which the viewpoint of a video of a preview playback layer moves according to movement of an included manipulation axis element.
In claim 1,
The above editing interface is,
An AI human character selection layer that can select the number and model of AI human characters in the above AI human image; and
It further includes an AI human character setting layer that can set at least one of the style, angle, pose, size, and position for each AI human character;
The above AI human character setting layer is,
A method for producing automatic video content based on user input, which can be displayed according to selection input of an AI human character displayed in an AI human video played in a preview playback layer, or selection input in a summary block corresponding to a subtext.

Images