JP7187737B2 - Method, program and electronic device for providing a user interface - Google Patents

Description

The present invention relates to a method, a program, and an electronic device for providing a user interface in an electronic device operated by touching the screen with a finger.

Unlike text, videos and images displayed on electronic devices (eg, smartphones and tablet terminals) are effective means of visual communication. Especially in moving images, not only visual effects but also sound can be used to make the visual experience more realistic.

In general, tapping an image displayed on an electronic device to transition to another page is also used in an advertising method called banner advertisement. In addition, you can copy an image, mainly a photograph, by touching the target image for a certain period of time. Videos can be viewed on electronic devices on video sites/apps such as YouTube (registered trademark) (http://www.youtube.com), but video operations and page transitions are performed by tapping in the object tap area Generally, one action is executed by an operation.

On the other hand, Japanese text input on electronic devices uses a flick operation that touches and moves horizontally, providing a comfortable one-handed operation experience. However, although there are devices that operate by directly touching videos and images, those that operate by flicking have not been realized and do not exist in the world.

For videos and images displayed on conventional electronic devices, tap the target video or image itself (play button or pause button on the video, tap on the image to transition to another page) etc.), or by tapping an object displayed on the same page, most of which are operated by causing the terminal to perform an action.

The number of actions that can be performed is limited only by the technique of tapping and operating a displayed target video or image. In other words, there is a one-to-one relationship between a single tap area and a single action for operations on displayed videos and images. Also, it is required to provide a plurality of tappable objects for executing actions within the same page. As the number of objects that can be tapped increases, the amount of information increases for the user, which not only makes the operation difficult, but also may cause visual problems. In addition, especially with smartphones, it is necessary to use both hands to perform actions, creating situations that make it difficult for users to operate.

SUMMARY OF THE INVENTION An object of the present invention is to provide an excellent interface for a user who reproduces a moving image in an electronic device operated by touching the screen with a finger.

The present invention enables a plurality of actions to be performed in a limited touch/tap area by flicking a moving image displayed on an electronic device.

Specifically, it has the configuration described in the claims. Multiple objects can be displayed at the same time by directly touching a video or image displayed on an electronic device. By flicking a target object while multiple objects are displayed, the action of each object can be executed within the terminal.

The terms used in this specification will be briefly explained, and the present invention will be specifically explained.

By "object" throughout the specification is meant a user-selectable object. Icons, pop-up images, links, pictures, text, index items, etc. are examples of objects. "Page" means a space containing content, information or objects. For example, a web page, a content list, a thumbnail list, a tray in which photos are arranged, a tray in which icons are arranged, etc. are examples of pages.

"Touch" refers to the action of a user using a finger or a touch tool (stylus) to touch a touch-enabled screen.

A "flick operation" is when a user "touches" with a finger or touch tool, and then moves the touched finger or touch tool in a horizontal direction to the touch surface to give a specific instruction without leaving the screen. say operation. Therefore, the operation of simply sliding a finger on the screen quickly is also included in the flick operation. It should be noted that the action of releasing the finger after "touching" without moving the finger or the touch tool in the horizontal direction is a "tapping" and is distinguished from a "flicking operation". Note that "moving away from the screen" actually means moving away from a specific range from the screen.

According to the present invention, a plurality of actions can be performed in a single limited tap area of an object by enabling flick operations on moving images and images displayed on an electronic device. Reducing the amount of information on objects not only makes them look more organized, but also makes them easier for users to understand and allows them to perform actions with more intuitive operations. In particular, smartphones can provide a comfortable usage experience that can be operated with one hand. In addition, after flicking, it is possible to prepare more action patterns in a single object tap area, such as changing the effect depending on the finger that touches the screen or the time you release the touch tool. .

FIG. 1A is a front view showing an outline of an electronic device according to one embodiment of the invention. FIG. 1B is a block diagram showing how the electronic device communicates according to one embodiment of the present invention. FIG. 1C is a front view of the electronic device showing a situation in which a moving image is being reproduced in the moving image reproducing application.
FIG. 2A is a front view of the electronic device showing the situation when the user touches during video playback. FIG. 2B is a front view of the electronic device showing a situation in which the user moves the finger sideways after touching.
3A-3D, 4E-4H, and 5I-5K are flowcharts of methods for providing a user interface according to embodiments of the present invention.

FIG. 1A is a front view showing an outline of an electronic device according to one embodiment of the invention. FIG. 1B is a block diagram showing how the electronic device communicates according to one embodiment of the present invention. Electronic device 10 has touch-enabled screen 20 , speaker 30 , and physical buttons 50 . In this embodiment, the electronic device 10 is a user-portable electronic device with a communication function that further includes a microphone, a battery 88, a CPU 83, a memory 84, and a communication unit 86, and is a smart phone that also has a telephone function. Having a battery 88 and having a small size (1 kg or less, preferably 0.5 kg or less) makes it practical for the user to carry the electric device 10 . The electronic device 10 can use the communication function to access the external server 70 that distributes moving images to reproduce the moving images on the screen 20, and the moving image files stored in the internal memory of the electronic device 10. can be reproduced on the screen 20. The communication unit 86 supports communication standards such as W-CDMA, LTE, WiFi (eg IEEE802.11n).

In the electronic device 10, an operation system for mobile devices such as iOS or Android is running, and application software (app) is running on this or on an app such as a web browser running on it. Operate. FIG. 1A shows an initial screen of a video playback application, which is a program according to the present invention. This initial screen is displayed, for example, when a specific address is accessed. A moving image reproduction area 22 is provided on the screen 20 , and a reproduction button 40 , a current position display 44 , a progress bar 45 , and a time display 46 are arranged inside or near the moving image reproduction area 22 . In this situation, when the play button 40 is touched, playback of the moving image is started. Alternatively, when a specific address is accessed, playback of the video is automatically started.

FIG. 1C is a diagram showing a situation in which a moving image is being reproduced in this moving image reproduction application. Twelve seconds have passed since the video playback started, the current position display 44 on the progress bar 45 has moved horizontally, and the playback button 40 in FIG. 1A is not displayed. The sound of the moving image is output from another speaker (not shown) that is not the call speaker 30 .

FIG. 2A is a diagram showing the situation when the user touches with the finger 90 during video playback. In FIG. 2A, a pause button 43 is displayed at the center of the screen where the play button 40 was displayed in FIG. 1A, and four flick auxiliary displays 60 to 63 are displayed around it. "Flick assist display" is a display for assisting the user's flick operation, preferably each having a frame, and inside, characters, icons, images, and animations representing related actions. is displayed. The frame is preferably substantially quadrangular, but may be substantially pentagonal in the form of a balloon drawn from the central portion. In the latter case, it is possible to encourage the user to perform the flick operation. The flick assist display 60 corresponds to "play previous video", and the flick assist display 62 corresponds to "play next video". Although the inside of the flick assist display 61 is not drawn in the drawing, letters and the like are actually displayed. The number of flick assist displays displayed at the same time is preferably 1 or more and 8 or less, and preferably 4. This is because if there are four, they will be arranged vertically and horizontally, and it is possible to display many while preventing operation errors. Also, if the number is two, it is possible to further reduce the number of operation errors, and if the number is eight, it is possible to display more while moderately preventing operation errors. In addition, since the flick assist display at the lower position is difficult for the user to see, it is preferable to use 3 (top, left, right, top, left, right, top left, and top right), and 7 of the 8 directions other than the bottom.

From the situation in FIG. 2A, the user moves the finger 90 in the horizontal direction (in this case, up, down) corresponding to the flick assist indications 60 to 63 without releasing the finger 90 from the touch surface (or beyond a predetermined range). , left, right) and the user removes finger 90 from the screen. This completes the user's flick operation. It should be noted that, depending on the speed at which the finger 90 is slid at this time, it can be determined that it is a flick operation if it is fast and that it is not a flick operation if it is slow.

FIG. 2B illustrates the situation where the user has moved the finger 90 horizontally to the right after touching. After touching the screen with the finger 90 in FIG. 2A, the user slides the finger 90 horizontally to the right without removing it from the touch surface, and then removes the finger 90 from the screen to complete the flick operation called "right flick." ing. As a result, the flick assist display 62 corresponding to the "right flick" is selected, and the associated action "playback of next moving image" is executed.

In another embodiment, when the flick assistance displays 60 to 63 are displayed in FIG. 2A, the flick assistance display 60 corresponds to "playback related operation" and the flick assistance display 62 corresponds to "address related operation". Characters and icons are displayed inside, and if you complete the "left flick", you can also stop playback, select the playback position of the video, move to the next video playback, and move to the previous video. Another four "second auxiliary displays" corresponding to the action to move to playback, the action to move to the beginning of the moving image, etc. are displayed. The second auxiliary display is located at the same position as the flick auxiliary displays 60 to 63 as shown in FIG. Is displayed. In the former case, it is possible to give the user consistency in the position of the auxiliary display, making the operation easier to understand. On the other hand, in the latter case, it becomes easier to distinguish the second auxiliary display from the first flick auxiliary display. The user simply touches the second auxiliary display to complete the selection. Alternatively, the user does not release the finger 90 from the screen at the time of the previous flicking operation, or touches the finger 90 immediately after releasing it once, and completes the second flicking operation on the second auxiliary display, thereby making a selection. to complete.

3A-4K are flowcharts of a method for providing a user interface according to one embodiment of the invention. The operation will be described below based on the flow charts of FIGS. 3A to 4K. In S102 of FIG. 3A, a moving image is reproduced on the screen based on the user's operation or the like. After this, the user touches the moving image reproduction area 22 on the screen 20 with his/her finger. At S104, a touch by the user's finger is detected. In S106, the flick assist displays 60 to 63 are displayed on the screen 20 while continuing to reproduce the moving image. In the examples of FIGS. 2A and 5, the flick assist displays 60 and 62 respectively correspond to "play previous video" and "play next video". The flick assist displays 60 to 63 may be icons, character strings, regions that generate sounds, or combinations thereof. The flick assist displays 60 to 63 make it easier for the user to understand the content of the operation.

After that, the user wants to operate "play next video", and the user does not lift his finger from the screen 20 as shown in FIG. to the right and release the finger from the screen 20 to complete the "quick flick to the right" operation. In S108, the completion of this flick operation is detected.

Even at this point, the video that was being played continues to be played. At S110, the action "play next video" associated with the flick assist display 62 is executed. In S104 to S108, the moving image is continuously reproduced, so that even when the user is thinking about which flick auxiliary display 60 to 63 to perform an operation, motion reproduction can be provided to the user, If the user desires to continue watching the video playback, the video can be played back at the video playback position from S104 by not completing the flick operation. This improves user convenience.

FIG. 3B is a flowchart of a method for stopping video playback after S104 compared to the method of FIG. 3A. After the touch is detected in S104, the moving image reproduction is stopped and the flick assist displays 60 to 63 are displayed in S107. At this time, the pause button 43 was displayed in the method of FIG. 3A, but the play button 40 is displayed at this position. Then, in S109, the completion of the flick operation is detected while continuing to stop moving image reproduction.

By stopping the video playback in this way, it becomes easier for the user to concentrate on the flick auxiliary displays 60 to 63, and even if the user performs the above-described cancel operation, the touch in S104 Since the video playback can be resumed from the video playback position in , the user can be prevented from missing a desired scene. A method of continuing to play the video after the touch (S104), like the method of FIG. 3A, and a method of stopping the video playback after the touch (S104), like the method of FIG. 3B, can be set by the user. It is preferable to

In the method of FIG. 3C, after the touch, the playback speed of the moving image is changed in S202. Specifically, it is preferable to make the playback speed of the moving image slower than the normal playback speed. This makes it easy for the user to consider which operation to perform while looking at the flick auxiliary displays 60-63. Note that S202 may be performed after S106. Also, in the method of FIG. 3C, instead of continuing to play the video after the touch (S104), the video playback may be stopped after the touch (S104) as in the method of FIG. 3B.

In the method of FIG. 3D, after the touch, at S204, the brightness of the moving image area is changed. Specifically, it is preferable to darken the brightness of the moving image area. This makes it possible for the user to easily grasp that the user can perform operations related to the flick auxiliary displays 60 to 63, and to make the flick auxiliary displays 60 to 63 stand out. , has the effect of prompting the operation of the flick auxiliary displays 60-63. Note that S204 may be performed after S106. Also, in the method of FIG. 3D, instead of continuing to play the video after the touch (S104), the video playback may be stopped after the touch (S104) as in the method of FIG. 3B.

In the method of FIG. 4E, after detecting a touch in S104, in S208, a flick assist display corresponding to a fast-forwarding action and a rewinding action is displayed. Then, in S108, the completion of the flick operation is detected. Then, the user continues to touch the flick assist display corresponding to the action of fast-forwarding and the action of rewinding. At S<b>210 , the corresponding fast-forwarding or rewinding action is executed during the touch, and the moving image corresponding to the fast-forwarding or rewinding is displayed on the moving image reproduction area 22 . As a result, the user can grasp the state of fast-forwarding or rewinding, and can grasp the moving image playback position when the touch is stopped at that time. Note that, in the method of FIG. 4E, the moving image may be stopped after the touch (S104), as in the method of FIG. 3B, instead of continuing to reproduce the moving image after the touch (S104).

The action associated with the flick assist display in S110 preferably includes an action of saving or registering a moving image. FIG. 4F shows a method for this case. "Save a moving image" refers to an act of saving a moving image in the memory 84 of the electronic device 10 or on an external server. "Registering a moving image" refers to setting a favorite ( bookmark setting).

The actions associated with the flick assist display in S110 include an action to stop playing, an action to select the playing position of the video, an action to play the next video, an action to play the previous video, and the beginning of the video. It preferably includes play-related actions such as moving to. The method of FIG. 4G illustrates this case. There are many user operations related to moving images, and it is advantageous to display them only when touched in S104, instead of always displaying them on the screen, because the screen can be used effectively.

Further, the operations associated with the flick assist display in S110 include an operation of accessing a specific address, an operation of copying the address associated with the video, and an operation of executing a specific program using the address associated with the video. Preferably, address-related operations are included, such as those that cause specific functions to be performed. User operations related to moving images are not limited to the above-described playback-related operations. In particular, in the portable electronic device 10 with a communication function, it is very important to be able to easily perform address-related operations. Because it's convenient. The "operation of accessing a specific address" includes, for example, accessing a specific URL with an argument using the HTTP protocol with a browser. "Copy the address associated with the video" means that you can put the address such as the URL or contact information associated with the video into the virtual clipboard and paste it in the text editing area etc. It means to perform the copy function of the operating system to make it. "Action to execute a specific function of a specific program using an address associated with a video" means passing an address associated with a video to a specific program as information to execute a specific function. . For example, the operation of evaluating current status registration services (e.g., ``Like'' operation on facebook provided by http://facebook.com), automatic posting in posting services (e.g., http: There is an action that emits a tweet on twitter provided by //twitter.com.

In the method of FIG. 4H, after detecting a touch in S104, in S230, a representative image of another moving image related to the moving image being played is displayed in the flick assist display. This representative image is also called a thumbnail image, and may be a moving image or a still image. At S232, an operation of reproducing another moving image corresponding to the representative image in the flick assist display is performed. By this method, the user is provided with another moving image related to the moving image being reproduced, can know a part of the contents thereof, and can easily select reproduction of the other moving image.

In the method of FIG. 5I, after detecting a touch in S104, in S240, a screen shot image of a moving image at or near the time of the touch is temporarily acquired in memory. A screen shot image of a moving image is a still image in a specific scene of the moving image. At S242, a flick assist display corresponding to the action of acquiring a screenshot image is displayed. After detecting completion of the flick operation in S108, an operation of acquiring (saving) a screenshot image in the memory 84 is executed in S244. In another embodiment, S240 is omitted, and an operation of acquiring (saving) in the memory 84 the screen shot image at the time of detection of the completion of the flick operation is executed in S244. This method makes it easy for the user to take a screenshot of the video at the desired video playback position.

The method of FIG. 5J provides a user interface that allows the user to easily select the video playback position. After detecting the touch in S104, in S250, a flick assist display corresponding to the selection of the video playback position is displayed. The moving image playback position is the position from the beginning to the end within the moving image and is represented by time. After detecting the completion of the flick operation in S108, the user performs a predetermined move operation. A move action is an action of moving a finger on the screen. It is preferable to move the current position display 44 on the progress bar 45 according to this move operation. This makes it easier to grasp where it is in the whole. In addition, it is preferable to display the image at that time in the entire moving image reproduction area or in a smaller frame that moves according to the current position display 44 . This makes it possible to recognize the position even in the image. Then, in S252, an operation of accepting selection of a moving image playback position in accordance with the detected move operation of the user is executed. Then, in S254, the moving image is reproduced from the received reproduction position. This method makes it easier for the user to select the playback position.

The method of FIG. 5K provides a user interface that allows a user to select a plurality of second auxiliary displays newly displayed after a flick operation is performed by displaying the flick auxiliary display. In S306, a flick assist display corresponding to the action group is displayed in some (or all) of the plurality of flick assist displays. For example, the reproduction-related operation and the address-related operation described above. After the operation corresponding to the flick auxiliary display is executed in S110, a plurality of second auxiliary displays are newly displayed in S308. This state is, for example, as shown in FIG. 2A. Alternatively, the second auxiliary display can be displayed by being shifted so as to correspond to the performed flick operation. After that, the user selects one of the second auxiliary displays by touch or the like, and in S310, the user's selection operation of the second auxiliary display is detected, and in S312, the selected second auxiliary display is displayed. Execute the associated action. By this method, a nested user interface can be provided, and the user can be provided with many options.

It should be noted that although several embodiments have been shown and several features have been described in the drawings and specification, these may be combined, and these combinations are also incorporated herein. For example, after S104 and before S110 of the method of FIG. 3B, the animation is stopped, but this is replaced by "continue playing the animation" after S104 and before S110 of the method of FIGS. 3C-4J. , to "stop the animation" in this section.

Claims (12)

A method for providing a user interface in an electronic device operated by touching a screen with a finger, comprising the steps of playing back a moving image on the screen, detecting a user's touch in the area where the moving image is being played, and newly displaying a plurality of flick assist displays each associated with an action around the position of the touch on the moving image while continuing to play the moving image in response to the detection of the touch; detecting completion of a flick operation corresponding to the flick assist display while continuing to reproduce; and executing an action associated with the flick assist display; A method characterized by varying the playback speed of the. 2. The method of claim 1 , wherein in response to detecting the touch, the brightness of a region displaying a moving image is changed. The actions associated with the flick assist display include the action of stopping the playback, the action of selecting the playback position of the video, the action of moving on to playback of the next video, the action of moving to playback of the previous video, and the operation of moving to the beginning of the video. 2. The method of claim 1 , comprising a play-related action selected from the group consisting of moving actions. The action associated with the flick assist display includes a reproduction-related action selected from the group consisting of an action for fast-forwarding playback and an action for rewinding playback, and the fast-forwarding or rewinding is the flick operation. 2. The method according to claim 1 , wherein the screen is touched after and during the fast-forwarding or rewinding, the animation corresponding to the fast-forwarding or rewinding at that time is displayed on the screen. 2. The method of claim 1 , wherein the action associated with the flick assist display includes saving or registering a video being played. The actions associated with the flick assist display include an action of accessing a specific address, an action of copying an address associated with a video, and a specific function of a specific program using the address associated with a video. 2. The method of claim 1 , comprising an address-related operation selected from the group consisting of an operation that causes a . The flick auxiliary display displays a representative image of another moving image related to the moving image being played, and the operation associated with the flick auxiliary display is an operation of reproducing another moving image corresponding to the representative moving image. 2. The method of claim 1 , comprising: 2. The method of claim 1 , wherein the action associated with the flick assist display includes taking a screenshot image of the animation at a given stage of the flick operation. The operation associated with the flick assist display includes an operation of accepting selection of a video playback position in response to the detected user's move operation on the screen, and the method thereafter includes: 2. The method of claim 1 , further comprising playing a motion picture. The operation associated with the flick auxiliary display is an operation of displaying a further auxiliary display, and when the operation of displaying a further auxiliary display is executed upon completion of the flick operation, a plurality of new auxiliary displays are displayed on the screen. and performing an action associated with the second auxiliary display upon detecting a user selection operation on one of the second auxiliary displays. Item 1. The method according to item 1. A program configured to cause an electronic device to execute the method according to any one of claims 1 to 10 . An electronic device arranged to carry out the method according to any one of claims 1-10 .

Images