JP2025007851A - PROGRAM, INFORMATION PROCESSING APPARATUS, AND METHOD FOR CONTROLLING INFORMATION PROCESSING APPARATUS - Google Patents

Abstract

To provide a mechanism that can easily select an object that is difficult to be select by tapping.SOLUTION: An information processing device 1001 equipped with a touch display 14 displays a poster editing screen 201 on the touch display 14. If it is determined that an operation for selecting an object displayed on the poster editing screen 201 is not easy, the information processing device 1001 displays a selection editing assistance item 90 that assists in operation to edit the object and brings the object into a selected state.SELECTED DRAWING: Figure 9

Description

The present invention relates to a program, an information processing device, and a control method for an information processing device.

There is layout software (hereafter referred to as layout software) that creates designs by arranging objects such as shapes, images, and text boxes. In layout software, objects can be edited on a screen (hereafter referred to as editing screen) where designs are created and edited. To edit an object, an input device attached to a terminal such as a PC or smartphone is used to select the object (hereafter referred to as selection operation) and place the object in a selected state (hereafter referred to as selected state). To place an object in a selected state, an input device attached to the terminal is used to select an area (hereafter referred to as selection area) that accepts input for placing the object in a selected state. The selection area is generally a rectangle circumscribing the object or an area of the same shape as the object.

On a terminal such as a PC, an input device such as a mouse is used to operate the mouse cursor and click the selection area of an object to be selected, and the object is selected. On the other hand, on a terminal such as a smartphone, an input device such as a finger or a touch pen is used to tap the selection area of an object to be selected on a touch display (hereinafter referred to as a tap operation), and the object is selected. When an object is selected, information such as the fill, border, and size of the object (hereinafter referred to as attribute information) can be edited. When creating a design using multiple objects, the multiple objects may be placed in an overlapping state. In this state, when the editing screen is set to the XY plane, the overlapping order of the objects is managed by the Z axis. When objects overlap, the object in the lower layer is displayed as if it is hidden by the object in the upper layer. To select an object in the lower layer, if there is a selection area of the object in the lower layer that is visible from the top layer, that area is selected. If the selection area of a lower layer object is completely hidden by an upper layer object, or if the selection area of a lower layer object visible from the top layer is extremely small, the upper layer object is temporarily placed in a different location so that the lower layer object can be selected.

Patent Document 1 proposes a technique for selecting multiple overlapping objects. In Patent Document 1, when multiple objects overlap, performing a specified operation on the selection area of the overlapping objects switches the object to be selected from the multiple overlapping objects. This is a function that assists in selecting an object placed in a lower layer. Meanwhile, PowerPoint provided by Microsoft Corporation has a function that displays a list of placed objects and selects any object. Also, Illustrator provided by Adobe Corporation has a function that allows you to recognize the object to be selected by placing the mouse cursor over an object (mouse over) and highlighting the border of the object.

In addition, in object detection such as Convolutional Neural Network (hereafter referred to as CNN) and You Only Look Once (hereafter referred to as YOLO), when an object to be searched for within an image is detected, a technique is known in which the detected location is given a rectangular frame or the edge of the detected location is used as a boundary to perform a filling process, and the image is output. The detected location may be treated as an object and a selection operation may be performed.

JP 2021-197024 A

However, on devices such as smartphones, fine editing operations may be required depending on the object settings, making object selection difficult. One example of when object selection becomes difficult is when the size of the object is small. As the selection area becomes smaller with the size of the object, fine selection operations are required to select the object. On devices such as PCs, it is possible to select small objects by operating the mouse and making fine adjustments to the mouse cursor, but on devices such as smartphones where tapping is the main operation, fine selection operations become difficult. Furthermore, with the technology of Patent Document 1 described above, it is necessary to tap the selection area to select a small object, making fine selection operations difficult on devices such as smartphones.

In addition, as another example of a case where the object selection operation becomes difficult, the color setting of the attribute information such as the fill of a first object is set to a similar color to the color setting of the attribute information of an adjacent second object, and the visibility of the first object decreases (hereinafter referred to as object assimilation). When object assimilation occurs, it becomes difficult to recognize the placement position of the object. The above-mentioned PowerPoint provides a function to display all objects in a list, but since the display size of a terminal such as a smartphone is relatively small, it is not possible to provide a list display of objects. In addition, the above-mentioned Illustrator provides a function to highlight the frame line by hovering the mouse over an object, but since the mouse cursor cannot be operated on a terminal such as a smartphone, it is not possible to highlight the object by hovering the mouse. Furthermore, in the technology of the above-mentioned Patent Document 1, in order to select an object in which object assimilation has occurred, it is necessary to tap the selection area of the object. When object assimilation has occurred, the placement position of the object cannot be visually recognized, and therefore it becomes difficult to select the object.

Another example of a case where it becomes difficult to select an object is when the selection area of an object is largely hidden by an object above it. In a device such as a smartphone, the area of the selection area of a lower-layer object that can be seen from the top layer becomes small, making it difficult to perform a tap operation to select the lower-layer object. With the technology of Patent Document 1 described above, it is possible to select an object in a lower layer by performing a predetermined selection operation on the overlapping selection area, but if the object in the lower layer is located in a position that cannot be seen from the top layer, it is difficult to recognize the overlapping selection area, making it difficult to select the object. Thus, with the conventional technology, it is not possible to easily select an object that is difficult to select by tapping.

The present invention aims to provide a mechanism that makes it easy to select objects that are difficult to select by tapping.

To achieve the above object, the program of the present invention is characterized in that it causes an information processing device having a touch display that accepts user operations to execute the steps of: displaying an object on the touch display in accordance with the user operations accepted on the touch display; determining whether the object displayed on the touch display is easy to select; and, if it is determined that the object displayed on the touch display is not easy to select, displaying an auxiliary UI that assists in editing the object and setting the object to a selected state.

According to the present invention, it is possible to easily select objects that are difficult to select by tapping.

1 is a block diagram showing an example of the configuration of an information processing system including an information processing device according to an embodiment of the present invention. 1. FIG. 4 is a diagram showing an example of a poster editing screen displayed on a touch display of the information processing apparatus of FIG. 4 is a flowchart showing the steps of a poster creation process executed by the information processing apparatus of FIG. 1 . 1. FIG. 4 is a diagram showing an example of a poster editing screen displayed on a touch display of the information processing apparatus of FIG. 4 is a flowchart showing the procedure of the determination process in S305 of FIG. 3; 6 is a flowchart showing the procedure of a process for determining a color difference in step S503 of FIG. 5; FIG. 6 is a diagram for explaining the determination of color difference in S503 of FIG. 5 . FIG. 6 is a diagram for explaining the determination of the area of the visible region in S505 of FIG. 5 . 4 is a diagram showing an example of a selection editing support item displayed on a touch display of the information processing device of FIG. 1 . 12A to 12C are diagrams for explaining display control in the second edit control process of FIG. 11 . 10 is a flowchart showing the procedure of a second edit control process executed by the information processing apparatus of FIG. 1 . 13 is a flowchart showing the procedure of a third edit control process executed by the information processing apparatus of FIG. 1 . 11 is a diagram showing a state in which a user performs a swipe operation on the poster editing screen using a pointing device. FIG. 13 is a flowchart showing the procedure of a fourth edit control process executed by the information processing apparatus of FIG. 1 . 13 is a flowchart showing the procedure of a fifth edit control process executed by the information processing apparatus of FIG. 1 . 11A to 11C are diagrams for explaining display control of a selected editing support item in the present embodiment. 13 is a flowchart showing the procedure of a sixth edit control process executed by the information processing apparatus of FIG. 1 . FIG. 18 is a diagram for explaining display control in the sixth edit control process of FIG. 17 . 13 is a flowchart showing the procedure of a seventh edit control process executed by the information processing apparatus of FIG. 1 . FIG. 20 is a diagram for explaining display control in the seventh edit control process of FIG. 19 . 13 is a flowchart showing the procedure of an eighth edit control process executed by the information processing apparatus of FIG. 1 . FIG. 22 is a diagram for explaining the display control in S2102 of FIG. 21. 11A to 11C are diagrams for explaining display control of a selected editing support item in the present embodiment. 13 is a flowchart showing the procedure of a ninth edit control process executed by the information processing apparatus of FIG. 1 . FIG. 25 is a diagram for explaining display control in the ninth edit control process of FIG. 24 . 13 is a flowchart showing the procedure of a tenth edit control process executed by the information processing apparatus of FIG. 1 . FIG. 27 is a diagram for explaining the process of S2602 in FIG. 26; FIG. 2 is a diagram showing an example of a UI diagram of the object detection software according to the embodiment;

The preferred embodiment of the present invention will be described in detail below with reference to the attached drawings. Note that the following embodiment does not limit the scope of the present invention as claimed, and not all of the combinations of features described in the present embodiment are necessarily essential to the solution of the present invention. Furthermore, the present invention is applicable to applications that perform object selection operations, in addition to general layout software such as poster creation applications, photo album editing applications, and presentation creation applications. In this embodiment, a poster creation application will be described as an example of layout software.

FIG. 1 is a block diagram showing an example of the configuration of an information processing system 1000 including an information processing device 1001 according to this embodiment. In FIG. 1, the information processing system 1000 is composed of the information processing device 1001 and an external server 1002.

First, the configuration of the information processing device 1001 will be described. In FIG. 1, the information processing device 1001 includes a CPU 10, a ROM 11, a RAM 12, and a touch display 14 as a hardware configuration. The information processing device 1001 also includes an input/output interface (not shown) for connecting to a network 20 such as the Internet. The information processing device 1001 is, for example, a mobile terminal such as a smartphone or a tablet terminal. In the information processing device 1001, a selection area of an object to be selected is tapped using an input device such as a finger or a touch pen.

The CPU 10 is a central processing unit, and controls the entire information processing device 1001 by executing the OS 13 stored in the ROM 11 and the RAM 12. The CPU 10 also performs calculations based on input data by executing a web browser 15 stored in the ROM 11 and the RAM 12, a native application 16 executed directly on the information processing device 1001, and other programs. This allows data to be processed and output, and each piece of hardware to be controlled, thereby realizing each function of the information processing device 1001.

The ROM 11 is a read-only memory that stores programs for operating the layout software. The RAM 12 is a random access memory that is used as a work memory for the CPU 10. If the RAM 12 is a non-volatile RAM, it can also store programs.

The touch display 14 includes a display 141 that displays the screen and a touch sensor 142 that detects touch operations. On the touch display 14, the user performs input operations by directly touching the screen with a finger or the like, and the processing results are displayed on the screen. In the above-mentioned layout software, the user can select an object arranged on the screen by directly touching it with a finger, and can move the position of the object by sliding the finger while still touching. The user can also perform a zoom operation by placing two fingers on an object and spreading them apart to enlarge the object, and can input characters by touching a software keyboard displayed on the screen.

The web browser 15 browses web sites on the Internet and executes web applications. When executing a web application, the information processing device 1001 receives a client-side program from an external server 1002 connected via the Internet. The web browser 15 of the information processing device 1001 interprets scripting languages such as HTML and JavaScript to display the web site and run the web application.

The native application 16 is an application that becomes operational when installed in the OS 13. The native application 16 may operate without interfacing with the external server 1002, or may operate in interfacing with the external server 1002. Furthermore, the native application 16 has a function to assist in processing that is difficult to execute with a web application. Note that the above-mentioned layout software may be provided as a web application.

Next, the configuration of the external server 1002 will be described. In FIG. 1, the external server 1002 includes a program execution server 30, a print server 32, a storage server 33, and an SNS/cloud service 34. The external server 1002 also includes an input/output interface (not shown) for connecting to a network 20 such as the Internet.

The program execution server 30 executes a server-side program 301, which is a program that runs on the external server 1002. The print server 32 has a function of causing a design image created by the layout software on the information processing device 1001 to be printed on a printer 21 connected on the network.

The storage server 33 can be used as an auxiliary device for the RAM 12, and stores, for example, layout data output by the layout software in the information processing device 1001 and processed images.

The SNS/cloud service 34 accesses the provider of the SNS or cloud service, and controls the downloading of images of layout materials and the uploading of layout data to a storage service on the cloud. Note that, in the present embodiment, a configuration has been described in which the external server 1002 includes all of the program execution server 30, print server 32, storage server 33, and SNS/cloud service 34, but this configuration is not limited thereto. For example, the program execution server 30, print server 32, storage server 33, and SNS/cloud service 34 may each be realized as a physically separate device.

The printer 21 is connected to the information processing device 1001 via a wired connection, via the network 20, or via a wireless interface (not shown) such as BlueTooth or IrDA, and prints a design image created by the layout software on the information processing device 1001.

FIG. 2 is a diagram showing an example of a poster editing screen 201 displayed on the touch display 14 of the information processing device 1001 of FIG. 1. The poster editing screen 201 is a screen on which a user creates a poster. A background image 202 of the poster is arranged on the poster editing screen 201. The background image 202 is graphics data or image data. In addition to the background image 202, objects such as a graphics data object 203, a text data object 204, and an image data object 205 are arranged at arbitrary positions on the poster editing screen 201.

On the poster editing screen 201, the user performs an editing operation of the object 203 by, for example, inputting text data 210, selecting a fill color 211 for filling the object 203, and selecting a line type and a line color for the frame line 212 of the object 203. When editing the object 203, the user needs to select the object 203. When the user taps the selection area of the object that he or she wants to select with a finger or touches it with a pointing device such as a touch pen, the touch display 14 detects the coordinates and selects the object at the top layer on the coordinates. For example, when the object 203 transitions to the selected state, the object 203 can be edited by pressing the button 209, and the object 203 is deleted by pressing the button 206. Also, a new object is added by pressing the button 208. When the button 207 is pressed, the design image of the poster created on the poster editing screen 201 is transferred to the print server 32 of the external server 1002 via the OS 13.

Fig. 3 is a flowchart showing the steps of the poster creation process executed by the information processing device 1001 of Fig. 1. Fig. 3(a) is a diagram showing the steps of the poster creation process executed by the information processing device 1001. The poster creation process is realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The poster creation process is executed when the user performs a predetermined operation such as an operation for adding a new object to the poster editing screen 201 displayed on the touch display 14 or an operation for editing an object already placed on the poster editing screen 201. In this embodiment, an example will be described in which the poster creation process is started in response to the user pressing the button 208 for adding a new object to the poster editing screen 201.

In FIG. 3(a), first, the CPU 10 causes a new object to be added and displayed on the poster editing screen 201 in response to pressing of the button 208 (S301).

Next, in S302, the CPU 10 accepts a selection operation of an object displayed on the poster editing screen 201 from the user. Specifically, the CPU 10 accepts a tap operation on this object from the user. The CPU 10 puts this object into a selected state. Note that the tap operation may be not only a tap with a finger, but also a touch or click with a pointing device such as a touch pen.

Next, in S303, the CPU 10 accepts an editing operation of the selected object from the user. This editing operation is, for example, an operation of moving, resizing, changing the fill color of the selected object, etc. For example, when an editing operation of moving the selected object is accepted, the CPU 10 acquires the coordinates of the selected object after it has been moved. In addition, when an editing operation of the text data (e.g., text data 210) or fill color (e.g., fill color 211) constituting the selected object is accepted, the CPU 10 acquires the changes made to the selected object.

Next, in S304, the CPU 10 causes the touch display 14 to display the poster editing screen 201 in which the editing operation accepted in S303 is reflected in the selected object. For example, if an editing operation to move the selected object is accepted in S303, the CPU 10 causes the touch display 14 to display the poster editing screen 201 in which the position of the selected object has been changed in accordance with the editing operation accepted in S303. The poster creation process then ends.

In such object editing operations, if multiple objects are placed on top of each other, the user cannot easily select the desired object. For example, as in Figure 4, if object 401 is placed on top of object 203, the selection area for object 203 becomes narrower, making it difficult for the user to select object 203.

In contrast, in this embodiment, if it is determined that the selection operation of an object displayed on the poster editing screen 201 is not easy, a selection editing support item 90 (support UI) described below is displayed to support the editing operation of the object.

Fig. 3(b) is a flowchart showing the procedure of the first editing control process executed by the information processing device 1001 of Fig. 1. As with the above-mentioned poster creation process, the first editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The first editing control process is executed when a new object is placed on the poster editing screen 201 in response to the user pressing the button 208.

In FIG. 3(b), first, the CPU 10 executes the determination process of FIG. 5 to determine whether the object arranged on the poster editing screen 201 is easy to select (S305).

Figure 5 is a flowchart showing the procedure for the determination process of S305 in Figure 3.

In FIG. 5, the CPU 10 executes the processes of S502 to S507 for an object placed on the poster editing screen 201. Note that if multiple objects are placed on the poster editing screen 201, the processes of S502 to S507 are executed for each of these multiple objects. Hereinafter, the object for which processing is being performed is referred to as the object being processed.

In S502, the CPU 10 determines whether the size of the object to be processed is equal to or greater than a predetermined size. Specifically, the CPU 10 determines whether the height of the object to be processed is equal to or greater than a height threshold and the width of the object to be processed is equal to or greater than a width threshold. For example, the CPU 10 sets the height threshold and width threshold based on the pixel values of the object to be processed drawn on the touch display 14 and the actual dimensions of the object to be processed when printed by the printer 21, and performs the determination in S502. If it is determined that the height of the object to be processed is equal to or greater than the height threshold and the width of the object to be processed is equal to or greater than the width threshold, the determination process in FIG. 5 proceeds to S503.

In S503, the CPU 10 determines whether the color difference in the boundary area between the object to be processed and other areas is equal to or greater than a predetermined value. The method for determining this color difference will be described with reference to Figures 6 and 7.

Figure 6 is a flowchart showing the steps of the process for determining color difference in S503 of Figure 5.

In FIG. 6, the CPU 10 obtains the boundary area between the object to be processed and other areas. Here, an example will be described in which a background image 71, an object 72, and an object to be processed 73 are arranged on the poster editing screen 201, as shown in FIG. 7(a). In this case, there are two boundary areas between the object to be processed 73 and other areas, as shown in FIG. 7(b). The first area is the boundary area 74 between the object 72 and the object to be processed 73, and the second area is the boundary area 75 between the background image 71 and the object to be processed 73.

Next, the CPU 10 executes the processes of S603 to S605 for the boundary regions 74 and 75 acquired in S601. In S603, the CPU 10 acquires the configuration of the boundary region. For example, FIG. 7C is an enlarged view of the boundary region 74. In S603, for example, the CPU 10 acquires that the configuration of the boundary region 74 is the fill color of the object 72 and the fill color of the processing target object 73.

Next, in S604, the CPU 10 calculates the color difference from the configuration of the boundary area acquired in S603. For example, the CPU 10 calculates the color difference of the boundary area 74 from the fill color of the object 72 and the fill color of the object 73 to be processed. Note that, in calculating the color difference of the boundary area 74, for example, the CPU 10 may acquire from the OS 13 the RGB values that display the object 72 on the touch display 14 and the RGB values that display the object 73 to be processed, and calculate the Euclidean distance between these RGB values.

Next, in S605, the CPU 10 determines whether the color difference calculated in S604 is equal to or greater than a predetermined value. Data indicating "true" as the determination result for a boundary region in which the color difference calculated in S604 is determined to be equal to or greater than the predetermined value is stored in the ROM 11 or RAM 12. Also, data indicating "false" as the determination result for a boundary region in which the color difference calculated in S604 is determined to be less than the predetermined value is stored in the ROM 11 or RAM 12. When the CPU 10 has completed the processes of S603 to S605 for all boundary regions, it performs the process of S606.

In S606, the CPU 10 obtains the logical product of all the judgment results stored in the ROM 11 or RAM 12. Next, in S607, the CPU 10 judges whether the logical product of S606 is "true". If the logical product of S606 is judged to be "true", that is, if the color difference calculated in all boundary regions is equal to or greater than a predetermined value, the CPU 10 judges that the color difference in the boundary region of the object to be processed is equal to or greater than a predetermined value (S608). On the other hand, if the logical product of S606 is judged to be "false", that is, if the color difference calculated in any boundary region is less than the predetermined value, the CPU 10 judges that the color difference in the boundary region of the object to be processed is not equal to or greater than a predetermined value (S609).

Returning to the explanation of S503 in FIG. 5, if it is determined that the color difference in the boundary region of the object to be processed is equal to or greater than a predetermined value, the determination process in FIG. 5 proceeds to S504.

In S504, the CPU 10 determines whether the object to be processed is located in the topmost layer. If it is determined that the object to be processed is not located in the topmost layer, but in a layer other than the topmost layer, the determination process in FIG. 5 proceeds to S505.

In S505, the CPU 10 determines whether the area of the visible area of the object being processed is equal to or larger than a predetermined value. The visible area is the area of the object being processed that is visible and does not overlap with any object above the object being processed. The determination of the area of the visible area will be explained using Figures 8(a) to (d).

8A is a diagram showing a case where a processing target object 81 and an object 82 are arranged on the poster editing screen 201. In such a case, the visible area of the processing target object 81 is the area 84 in FIG. 8C where there is no object 82 above the processing target object 81. FIG. 8B is a diagram showing a case where a processing target object 81 and an object 83 are arranged on the poster editing screen 201. In such a case, the visible area of the processing target object 81 is the area 85 in FIG. 8D where there is no object 83 above the processing target object 81. In S505, the CPU 10 acquires the area of the area 84 or the area 85 as the area of the visible area, and determines whether the area of the acquired visible area is equal to or larger than a predetermined value. Here, the CPU 10 may acquire the proportion of the visible area in the processing target object, and determine whether the proportion of the acquired visible area is equal to or larger than a predetermined proportion. In addition, the size and resolution of the touch display 14, the display magnification of the poster editing screen 201, etc. may be used as threshold values such as a predetermined value or a predetermined ratio used in the determination of S505.

If it is determined in S505 that the area of the visible region of the object to be processed is equal to or greater than a predetermined value, the determination process in FIG. 5 proceeds to S506. Also, if it is determined in S504 that the object to be processed is located in the top layer, the determination process in FIG. 5 proceeds to S506. In S506, the CPU 10 determines that the selection operation of the object to be processed is easy.

On the other hand, if it is determined in S505 that the area of the visible region of the object to be processed is less than a predetermined value, the determination process in FIG. 5 proceeds to S507. Also, if it is determined in S502 that the size of the object to be processed is less than a predetermined size, that is, the height of the object to be processed is not equal to or greater than the height threshold or the width of the object to be processed is not equal to or greater than the width threshold, the determination process in FIG. 5 proceeds to S507. Furthermore, if it is determined in S503 that the color difference in the boundary region of the object to be processed is not equal to or greater than a predetermined value, the determination process in FIG. 5 proceeds to S507. In S507, the CPU 10 determines that the selection operation of the object to be processed is not easy. In this way, in this embodiment, the determination process in FIG. 5 is performed for all objects arranged on the poster editing screen 201.

Returning to the explanation of S305 in FIG. 3B, if the CPU 10 determines in S305 that the selection operation of the object placed on the poster editing screen 201 is easy, the first editing control process ends. On the other hand, if the CPU 10 determines in S305 that the selection operation of the object placed on the poster editing screen 201 is not easy, the first editing control process proceeds to S306.

In S306, the CPU 10 displays items (hereinafter referred to as selection and editing support items) that support the selection and editing of objects determined to be difficult to select. For example, of the objects 203 and 401 arranged on the poster editing screen 201 shown in FIG. 4, the selection and editing support item 90 in FIG. 9(a) corresponding to the object 203 determined to be difficult to select in S305 is displayed. The selection and editing support item 90 not only indicates that the selection and editing support item 90 has put the object 203 in a selected state, but also has some of the functions of editing the object 203.

The selection editing support item 90 includes items 91, 92, and 93. The item 91 has a function of resizing the object 203. For example, when the item 91 is dragged toward the outside of the object 203, the object 203 is enlarged. When the item 91 is dragged toward the inside of the object 203, the object 203 is reduced. The item 92 has a function of rotating the object 203. For example, when a circle is drawn on the item 92, the object 203 is rotated in the direction of the circle. The item 93 has a function of moving the object 203. For example, when an arbitrary direction is dragged on the item 93, the object 203 is moved in the dragged direction. Note that the configuration of the selection editing support item 90 shown in FIG. 9(a) is one example, and other configurations may be used. FIG. 9(b) is a diagram showing an example of the poster editing screen 201 on which the selection editing support item 90 of the object 203 determined to be difficult to select in S305 is displayed.

When the process of S306 is completed, the CPU 10 selects the object 203 on which the selected editing support item 90 is displayed (S307). This allows the user to perform an editing operation on the object 203 using the selected editing support item 90. Thereafter, the first editing control process ends.

According to the embodiment described above, when it is determined that an object displayed on the poster editing screen 201 is not easy to select, a selection editing assistance item 90 that assists in the editing operation of the object is displayed, and the object is placed in a selected state. This makes it possible to easily select an object that is difficult to select by a tap operation.

In addition, in the above-described embodiment, if it is determined that the size of an object displayed on the poster editing screen 201 is less than a predetermined size, it is determined that the object is not easy to select. This makes it easy to select an object that is difficult to select by tapping due to its small size.

In addition, in the above-described embodiment, if it is determined that the color difference in the boundary area of an object displayed on the poster editing screen 201 is not equal to or greater than a predetermined value, it is determined that the object is not easy to select. This makes it easy to select an object that is difficult to select by tapping because the color difference with other areas is small and the placement position is difficult to visually recognize.

In addition, in the above-described embodiment, if it is determined that an object displayed on the poster editing screen 201 is not the top layer and the area of the visible region of the object is less than a predetermined value, it is determined that the object is not easy to select. This makes it possible to easily select an object that is difficult to select by tapping because it overlaps with an object in an upper layer.

In addition, in the above-described embodiment, the selection editing support item 90 has an object rotation function, a movement function, and a resize function. This allows the user to easily perform editing operations such as rotation, movement, and resize on objects that are difficult to select by tapping.

In the embodiment described above, when a new object is placed on the poster editing screen 201 in response to pressing of the button 208, it is determined whether or not the object placed on the poster editing screen 201 is easy to select. This makes it possible to easily select an object that has become difficult to select by tapping due to the placement of the new object.

In the present embodiment, a configuration has been described in which the first editing control process is executed when a new object is placed on the poster editing screen 201 in response to the user pressing the button 208, but the present invention is not limited to this configuration. For example, the first editing control process may be executed in response to the content of an editing operation performed by the user being reflected in an object placed on the poster editing screen 201. This makes it possible to easily select an object that has become difficult to select by a tap operation due to the content of an editing operation performed by the user being reflected in an object placed on the poster editing screen 201.

In addition, in this embodiment, the selected editing support item 90 may be controlled to be hidden at a predetermined timing. Here, for example, when the user moves the object 31 from the position in FIG. 9(b) to the position in FIG. 10(a), the area of the visible region of the object 203 to which the selected editing support item 90 is added becomes equal to or larger than a predetermined value. In such a case, the object 203 can be easily selected and operated, and the selected editing support item 90 becomes unnecessary.

In contrast, in this embodiment, when it is determined that the object to which the selection editing support item 90 is assigned is easy to select as a result of the content of the editing operation performed by the user being reflected in the object arranged on the poster editing screen 201, the selection editing support item 90 assigned to the object is hidden.

Figure 11 is a flowchart showing the procedure of the second editing control process executed by the information processing device 1001 of Figure 1. Like the first editing control process described above, the second editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The second editing control process is executed when S307 of the first editing control process described above is completed, the selection editing support item 90 of any object is displayed on the poster editing screen 201, and editing of the object is performed according to an editing operation by the user.

In FIG. 11, first, the CPU 10 determines whether the object to which the selected editing support item 90 was added in S306 described above is easy to select (S1101). For example, the CPU 10 performs the process of S1101 on the object 203 to which the selected editing support item 90 is added in FIG. 10(a). Specifically, the CPU 10 performs the same process as S502 to S507 described above on the object 203 to determine whether the object 203 is easy to select. If it is determined in S1101 that the object 203 is not easy to select, the second editing control process ends, and the display of the selected editing support item 90 for the object 203 continues.

If it is determined in S1101 that the object 203 can be easily selected, the CPU 10 hides the selected editing support item 90 (S1102), for example, as shown in FIG. 10(b). The CPU 10 also cancels the selection state of the object 203. The second editing control process then ends.

In the second editing control process described above, when it is determined that the object 203 to which the selection editing support item 90 is assigned is easy to select as a result of the content of the editing operation by the user being reflected in the object arranged on the poster editing screen 201, the selection editing support item 90 of the object 203 is hidden. This makes it possible to prevent the selection editing support item 90 that has become unnecessary due to the editing operation by the user from continuing to be displayed, thereby reducing the operability for the user.

However, a user may wish to change the content displayed on the touch display 14 by performing a swipe operation on the poster editing screen 201. For example, when an object is enlarged and displayed over the entirety of the poster editing screen 201, the user may perform a swipe operation on the poster editing screen 201 to display another object on the touch display 14. Also, when editing a poster with multiple pages on the poster editing screen 201, the user may perform a swipe operation on the poster editing screen 201 to display another page on the touch display 14. If the selection editing support item 90 continues to be displayed while such a swipe operation is being performed, the user may feel bothered because he or she must be conscious of not accidentally tapping the selection editing support item 90 when performing the operation.

In contrast, in this embodiment, when the start of a swipe operation is detected while the selection editing support item 90 is displayed on the poster editing screen 201, the selection editing support item 90 is hidden.

Figure 12 is a flowchart showing the procedure of the third editing control process executed by the information processing device 1001 of Figure 1. Like the first and second editing control processes described above, the third editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The third editing control process is executed when the CPU 10 receives a signal corresponding to a user operation from the touch sensor 142 after S307 of the first editing control process described above is completed and the selection editing support item 90 of any object is displayed on the poster editing screen 201.

In FIG. 12, the CPU 10 determines whether or not the signal received from the touch sensor 142 is a signal indicating a swipe operation (S1201). FIG. 13 is a diagram showing how a user performs a swipe operation on the poster editing screen 201 using a pointing device such as the touch pen 131. Note that the user may perform the swipe operation with something other than the touch pen 131, for example, a finger.

If it is determined in S1201 that the signal received from the touch sensor 142 is not a signal indicating a swipe operation, the third edit control process ends, and the display of the selected edit support item 90 continues. If it is determined in S1201 that the signal received from the touch sensor 142 is a signal indicating a swipe operation, the CPU 10 hides the selected edit support item 90 (S1202). As a result, for example, the selected edit support item 90 that was assigned to the object 203 as shown in FIG. 9B becomes hidden as shown in FIG. 13. Thereafter, the third edit control process ends.

In the third editing control process described above, when the start of a swipe operation is detected while the selected editing support item 90 is displayed on the poster editing screen 201, the selected editing support item 90 is hidden. In other words, when the start of a swipe operation is detected, the selected editing support item 90 is immediately hidden. This can reduce the annoyance of the selected editing support item 90 during a swipe operation.

In addition, in this embodiment, when an object other than the object to which the selection editing support item 90 is assigned is selected by a user operation, the selection editing support item 90 may be hidden.

Figure 14 is a flowchart showing the procedure of the fourth editing control process executed by the information processing device 1001 of Figure 1. Like the first, second, and third editing control processes described above, the fourth editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The fourth editing control process is executed when the information processing device 1001 receives a user operation after S307 of the first editing control process described above is completed and the selection editing support item 90 of any object is displayed on the poster editing screen 201. For example, when a selection operation of an object other than the object to which the selection editing support item 90 was assigned in S306 is received from the user, the object selected by the user is selected on the poster editing screen 201.

In FIG. 14, the CPU 10 determines whether the selected object is an object other than the object to which the selected editing support item 90 was assigned in S306 (S1401).

If it is determined in S1401 that the selected object is an object to which the selected editing support item 90 was assigned in S306, the fourth editing control process ends and the display of the selected editing support item 90 continues.

If it is determined in S1401 that the selected object is an object other than the object to which the selected editing support item 90 was assigned in S306, the CPU 10 makes the selected editing support item 90 invisible (S1402) and ends this processing.

In the fourth editing control process described above, when an object other than the object to which the selection editing support item 90 is assigned is selected by a user operation, the selection editing support item 90 is hidden. This makes it possible to prevent the selection editing support item 90 of an object whose selection state has been released from continuing to be displayed, thereby preventing a decrease in operability for the user.

In addition, in this embodiment, if it is determined that the selection operation of an object to which a selection editing support item 90 is assigned is not easy, the selection editing support item 90 may be temporarily hidden.

Figure 15 is a flowchart showing the procedure of the fifth editing control process executed by the information processing device 1001 of Figure 1. Like the first, second, third and fourth editing control processes described above, the fifth editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The fifth editing control process is executed after S307 of the first editing control process described above is completed and the selection editing support item 90 of any object is displayed on the poster editing screen 201.

In FIG. 15, the CPU 10 determines whether the object to which the selected editing support item 90 was added in S306 described above is easy to select (S1501). In S1501, the same processing as S502 to S507 described above is performed on the object to which the selected editing support item 90 was added in S306. If it is determined in S306 that the object to which the selected editing support item 90 was added is not easy to select, the fifth editing control process proceeds to S1502.

In S1502, the CPU 10 temporarily hides the selected editing support item 90. Next, the CPU 10 determines whether or not a predetermined condition has been met (S1503). The CPU 10 waits until the predetermined condition is met. For example, if a predetermined time has passed since the selected editing support item 90 was hidden in S1502, or if the CPU 10 detects pressing of the button 207, which is a print instruction, it is determined that the predetermined condition has been met. In this case, the CPU 10 redisplays the selected editing support item 90 that was hidden in S1502 (S1504), and ends this process.

If it is determined in S1501 that the object to which the selection editing support item 90 was assigned in S306 is easy to select, the CPU 10 makes the selection editing support item 90 invisible (S1505) and ends this process.

In the fifth editing control process described above, if it is determined that the selection operation of the object to which the selection editing support item 90 is assigned is not easy, the selection editing support item 90 is temporarily hidden. This prevents the user from mistaking the selection editing support item 90 for part of the design, and thus allows the user to properly check the finished design without compromising the operability of the editing operation.

In addition, in this embodiment, the configuration may be such that it is possible to switch whether or not to execute the first, second, third, fourth, and fifth editing control processes by setting the web browser 15 or the native application 16.

In addition, in this embodiment, if other objects are placed so as to cover the selected editing support item 90, the selected editing support item 90 may be controlled to be displayed in the top layer.

Figure 16(a) is a diagram showing a case where object 161 is placed so as to cover object 203. When objects 161 and 203 are placed in this manner, the above-mentioned editing control process causes selected editing support items 90 for object 203 to be displayed as shown in Figure 16(b). However, some items of selected editing support items 90, for example item 91, are hidden by object 161, and the user cannot see item 91.

In contrast, in this embodiment, when other objects are placed so as to cover at least a portion of the selected editing support item 90, the selected editing support item 90 is controlled to be displayed in the top layer.

Figure 17 is a flowchart showing the procedure of the sixth edit control process executed by the information processing device 1001 of Figure 1. Like the first, second, third, fourth and fifth edit control processes described above, the sixth edit control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The sixth edit control process is executed after S307 of the first edit control process described above is completed and the selected edit support item of any object is displayed on the poster editing screen 201.

In FIG. 17, the CPU 10 determines whether at least some of the items included in the selected editing support item 90 displayed in S306 overlap with an object above the object corresponding to the selected editing support item 90 by a predetermined percentage or more (S1701).
If it is determined in S306 that at least some of the items included in the selected editing support item 90 displayed are not overlapped by a predetermined ratio or more with the object above the object corresponding to the selected editing support item 90, the sixth editing control process ends. In other words, the display of the selected editing support item 90 continues as is.

If it is determined in S1701 that at least some of the items included in the selected editing support item 90 displayed in S306 overlap with an object above the object corresponding to the selected editing support item 90 by a predetermined percentage or more, the sixth editing control process proceeds to S1702.

In S1702, the CPU 10 displays the selected editing support item 90 in the top layer of the poster editing screen 201. As a result, for example, the item 93 hidden by the object 181 as shown in FIG. 18(a) is displayed on the poster editing screen 201 as shown in FIG. 18(b). Thereafter, the sixth editing control process ends.

In the sixth editing control process described above, if other objects are placed so as to cover at least a portion of the selected editing support item 90, the selected editing support item 90 is displayed in the top layer. This allows the user to see all items that make up the selected editing support item 90, regardless of the placement positions of the other objects.

In addition, in this embodiment, if another object is positioned so as to cover at least a portion of the selected editing support item 90, the selected editing support item 90 may be controlled to be displayed in a position that does not overlap with the other object.

Figure 19 is a flowchart showing the procedure of the seventh editing control process executed by the information processing device 1001 of Figure 1. Like the first, second, third, fourth, fifth and sixth editing control processes described above, the seventh editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The seventh editing control process is executed after S307 of the first editing control process described above is completed and the selection editing support item 90 of any object is displayed on the poster editing screen 201.

In FIG. 19, the CPU 10 determines whether at least some of the items included in the selected editing support item 90 displayed in S306 overlap with an object above the object corresponding to the selected editing support item 90 by a predetermined percentage or more (S1901).
If it is determined in S306 that at least some of the items included in the selected editing support item 90 displayed are not overlapped by a predetermined ratio or more with the object above the object corresponding to the selected editing support item 90, the seventh editing control process ends. In other words, the display of the selected editing support item 90 continues as is.

If it is determined in S1901 that at least some of the items included in the selected editing support item 90 displayed in S306 overlap with an object above the object corresponding to the selected editing support item 90 by a predetermined percentage or more, the seventh editing control process proceeds to S1902.

In S1902, the CPU 10 corrects the display position of the selected editing support item 90 to a position that is visible on the poster editing screen 201. Specifically, the CPU 10 corrects the display position of the selected editing support item 90 to a position where no object is located above the object corresponding to this selected editing support item 90. Thereafter, the seventh editing control process ends.

In the seventh editing control process described above, when another object is placed so as to cover at least a portion of the selected editing support item 90, the selected editing support item 90 is displayed in a position where it does not overlap with the other object. This makes it possible to suppress a decrease in the operability of the selected editing support item 90 caused by the placement position of the other object.

In addition, in the above-mentioned S1902, the display position of the selected editing support item 90 may be corrected based on the user's handedness information. The user's handedness information is information indicating the user's handedness (right-handed or left-handed), and is stored in the ROM 11 or the RAM 12. For example, if the user's handedness information indicates right-handedness, items 902 and 903 corresponding to items 92 and 93 included in the selected editing support item 90 are displayed at the right end of the poster editing screen 201, as shown in FIG. 20(a). Also, if the user's handedness information indicates left-handedness, items 902 and 903 corresponding to items 92 and 93 included in the selected editing support item 90 are displayed at the left end of the poster editing screen 201, as shown in FIG. 20(b).

By controlling in this manner, the operability of the selected editing support item 90 can be improved.

In addition, in this embodiment, if there are multiple objects that are determined to be difficult to select in S305 described above, a selection editing support item 90 for the lowest object among the multiple objects that are determined to be difficult to select may be displayed.

Figure 21 is a flowchart showing the procedure of the eighth editing control process executed by the information processing device 1001 of Figure 1. Like the first, second, third, fourth, fifth, sixth and seventh editing control processes described above, the eighth editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The eighth editing control process is executed when a new object is placed on the poster editing screen 201 in response to the user pressing button 208 after S307 of the first editing control process described above is completed and the selection editing support item 90 of any object is displayed on the poster editing screen 201.

In FIG. 21, the CPU 10 performs the determination process of FIG. 5 to determine whether there are multiple objects on the poster editing screen 201 that are difficult to select (S2101). If it is determined that there are not multiple objects on the poster editing screen 201 that are difficult to select, the eighth editing control process ends.

If it is determined in S2101 that there are multiple objects on the poster editing screen 201 that are difficult to select, the eighth editing control process proceeds to S2102.

In S2102, the CPU 10 displays the selection editing support item 90 for the bottom object among the multiple objects determined to be difficult to select. For example, in FIG. 22, the selection editing support item 90 for the bottom object 203 among the objects 203 and 221 determined to be difficult to select is displayed.

Next, the CPU 10 selects the object corresponding to the selected editing support item 90 displayed in S2102 (S2103) and ends this process.

In the eighth editing control process described above, when there are multiple objects that are determined to be difficult to select, a selection editing assistance item 90 is displayed for the bottommost object among the multiple objects that are determined to be difficult to select. The bottommost object is also placed in a selected state. This makes it possible to easily select the bottommost object that is the most difficult to select when multiple objects that are difficult to select are placed on the poster editing screen 201.

In addition, in this embodiment, when there are multiple objects that are determined to be difficult to select, an operation button for switching the object to which the selection editing support item 90 is assigned, for example, the button 231 in FIG. 23(a) may be displayed. For example, in FIG. 23(a), the selection editing support item 90 of the object 203 is displayed by the above-mentioned S2102. When the user presses the button 231 from this state, the CPU 10 displays the selection editing support item 90 of the object 221 out of the objects 203 and 221 that are determined to be difficult to select in S305, as shown in FIG. 23(b). In this way, in this embodiment, the object to which the selection editing support item 90 is assigned can be switched in accordance with the pressing of the button 231. Note that the switching of the object to which the selection editing support item 90 is assigned may be switched, for example, from the bottom layer according to the stacking order.

By controlling in this manner, when multiple objects that are difficult to select are placed on the poster editing screen 201, the user can easily switch between these objects to which the selection editing support item 90 is to be applied.

In addition, in this embodiment, a selection editing support item 90 may be displayed for each object that is determined to be difficult to select.

Figure 24 is a flowchart showing the procedure of a ninth editing control process executed by the information processing device 1001 of Figure 1. Like the first, second, third, fourth, fifth, sixth, seventh, and eighth editing control processes described above, the ninth editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12. The ninth editing control process is executed when a new object is placed on the poster editing screen 201 in response to the user pressing button 208 after S307 of the first editing control process described above is completed and the selection editing support item 90 of any object is displayed on the poster editing screen 201.

In FIG. 24, the CPU 10 determines whether there are multiple objects on the poster editing screen 201 that are difficult to select (S2401), similar to S2101 described above. If it is determined that there are not multiple objects on the poster editing screen 201 that are difficult to select, the ninth editing control process ends.

If it is determined in S2401 that there are multiple objects on the poster editing screen 201 that are difficult to select, the ninth editing control process proceeds to S2402. For example, in FIG. 25(a), objects 251 and 252 of the same color as the background image 202 are placed on the background image 202. Note that although the borders of objects 251 and 252 are shown as dotted lines in FIG. 25(a), these borders are not actually displayed. In such a case, in S2401, it is determined that there are multiple objects on the poster editing screen 201 that are difficult to select.

In S2402, the CPU 10 displays a selection editing support item for each of all objects determined to be difficult to select. For example, as shown in FIG. 25(b), items 911, 912, and 913 are displayed as selection editing support items 90 for object 251 determined to be difficult to select. Also, items 921, 922, and 923 are displayed as selection editing support items 90 for object 252 determined to be difficult to select.

Next, the CPU 10 selects the lowest object among the objects determined to be difficult to select (S2403). For example, in FIG. 25(b), the lowest object 251 among the objects 251 and 252 determined to be difficult to select is selected. At this time, the colors of the items 911, 912, and 913 may be changed to items 931, 932, and 933 in FIG. 25(c), respectively, so that it is clear that the object 251 is selected. In addition, the CPU 10 may switch the selected object from object 251 to object 252 in response to the user pressing button 231. At this time, the colors of the items 931, 932, and 933 are changed to their original colors (the colors of the items 911, 912, and 913), respectively, and the colors of the items 921, 922, and 923 are changed to items 931, 932, and 933. When processing of S2403 is completed, the ninth editing control process ends.

In the ninth editing control process described above, a selection editing assistance item 90 is displayed for each of all objects determined to be difficult to select, and the object at the bottom of the objects determined to be difficult to select is selected. This allows the user to easily edit the object at the bottom, which is the most difficult to select, and also allows the user to easily grasp the location of other objects that are difficult to select.

In addition, in this embodiment, if multiple objects that are determined to be difficult to select are included in a specified area, a selection assistance UI 272 (described later) may be displayed to assist in the selection of these multiple objects.

Figure 26 is a flowchart showing the procedure of a tenth editing control process executed by the information processing device 1001 of Figure 1. The tenth editing control process is also realized by the CPU 10 executing a program stored in the ROM 11 or RAM 12, similar to the first, second, third, fourth, fifth, sixth, seventh, eighth, and ninth editing control processes described above. The tenth editing control process is executed when a new object is placed on the poster editing screen 201 in response to the user pressing button 208 after S307 of the first editing control process described above is completed and the selection editing support item 90 of any object is displayed on the poster editing screen 201.

In FIG. 26, the CPU 10 determines whether there are multiple objects on the poster editing screen 201 that are difficult to select (S2601), similar to S2101 and S2401 described above. If it is determined that there are not multiple objects on the poster editing screen 201 that are difficult to select, the tenth editing control process ends.

If it is determined in S2601 that there are multiple objects on the poster editing screen 201 that are difficult to select, the 10th editing control process proceeds to S2602.

In S2602, the CPU 10 determines whether there are multiple objects that are difficult to select within a predetermined area corresponding to the user's touch position. For example, as shown in FIG. 27, assume that objects 203 and 221 are placed in positions that are determined to be difficult to select, and the user taps area 271 as an editing operation. In S2602, the CPU 10 determines whether there are multiple objects that are difficult to select in area 271.

If it is determined in S2602 that there are not multiple objects in the specified area that are difficult to select, the tenth edit control process ends. On the other hand, if it is determined in S2602 that there are multiple objects in the specified area that are difficult to select, the tenth edit control process proceeds to S2603.

In S2603, the CPU 10 displays the selection assistance UI 272 for selecting an object. Specifically, the CPU 10 acquires information on all objects in the area 271, and displays a list of buttons indicating the attribute information of each object (such as the fill color of the object and text information associated with the object) in the assistance UI 272. For example, if the user has input text data "Summer Sale" into the object 203, the CPU 10 displays "Summer Sale" as the button 274 corresponding to the object 203. Also, if the user has set the fill color of the object 222 to blue, the CPU 10 displays the color name as the button 275 corresponding to the object 222. When the user taps the button 274, the CPU 10 selects the object 203. Also, when the user taps the button 275, the CPU 10 selects the object 222. When the process of S2603 is completed, the tenth editing control process ends.

In the above-mentioned tenth editing control process, when multiple objects that are determined to be difficult to select are included in a predetermined area, a selection assistance UI 272 that assists in the selection of the multiple objects is displayed. This allows the user to easily select the multiple objects that are determined to be difficult to select.

In the above embodiment, the layout software is described as an example, but the present invention may be applied to software using object detection technology such as CNN or YOLO (hereinafter referred to as object detection software). Specifically, the UI diagram in FIG. 28 is used for explanation. When the object detection software detects the pressing of the button 286, it starts the process of detecting the search target 285 from the source image 281. The object detection software uses the object detection technology to display the detection areas 282 to 284 in a rectangle on the source image 281, but the user may select the detection area 282 using the object detection software. If the object detection software determines in the above-mentioned S305 that the selection operation of the detection area 282 is not easy, a selection editing support item is displayed. As a result, the object detection software can also achieve the same effect as the above-mentioned embodiment.

Needless to say, the object of the present invention can also be achieved by supplying a recording medium on which is recorded software program code that realizes the functions of the above-mentioned embodiments to a system or device, and having the computer (or CPU or MPU) of that system or device read and execute the program code stored on the recording medium. In this case, the program code read from the storage medium itself realizes the functions of the above-mentioned embodiments, and the storage medium on which the program code is stored constitutes the present invention.

Storage media for supplying program code may include, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, a ROM, a DVD, etc.

Furthermore, it goes without saying that not only are the functions of the above-mentioned embodiments realized by the computer executing the program code it has read, but also that the functions of the above-mentioned embodiments can be realized by an Operating System (OS) or the like running on the computer performing some or all of the actual processing based on the instructions of the program code.

Furthermore, it goes without saying that this also includes cases where the program code read from the storage medium is written into the memory of an expansion board inserted into a computer or an expansion unit connected to a computer, and then a CPU or the like of the expansion board or expansion unit performs some or all of the actual processing based on the instructions of the program code, thereby realizing the functions of the above-mentioned embodiments.

The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that realizes one or more of the functions.

The disclosure of this embodiment includes the following configurations and methods.
(Configuration 1) A program that causes an information processing device having a touch display that accepts user operations to execute the following steps: displaying an object on the touch display in accordance with the user operations accepted by the touch display; determining whether the object displayed on the touch display is easy to select; and, if it is determined that the object displayed on the touch display is not easy to select, displaying an auxiliary UI that assists in editing the object and setting the object to a selected state.
(Configuration 2) The program described in configuration 1, characterized in that in the judgment process, if the size of an object displayed on the touch display is less than a predetermined size, it is judged that the selection operation of the object is not easy.
(Configuration 3) A program described in configuration 1 or 2, characterized in that in the judgment process, if the color difference in the boundary area between an object displayed on the touch display and an area other than the object is not a predetermined value or more, it is judged that the selection operation of the object is not easy.
(Configuration 4) A program described in any one of configurations 1 to 3, characterized in that in the judgment process, if the object displayed on the touch display is not in the top layer and the area of the visible area of the object displayed on the touch display is less than a predetermined value, it is judged that the selection operation of the object is not easy.
(Configuration 5) The program according to any one of configurations 1 to 4, wherein the auxiliary UI has a function of rotating, moving, and resizing the object.
(Configuration 6) The program described in any one of configurations 1 to 5, characterized in that the program causes the information processing device to perform the determination step when a new object is displayed on the touch display in accordance with the user operation.
(Configuration 7) A program described in any one of configurations 1 to 5, characterized in that the information processing device performs the judgment process in accordance with the editing content corresponding to the user operation being reflected in an object displayed on the touch display.
(Configuration 8) The program described in Configuration 7, characterized in that when it is determined that the object for which the auxiliary UI was displayed is easy to select in accordance with the editing content corresponding to the user operation being reflected in the object displayed on the touch display, the program causes the information processing device to execute a process of hiding the auxiliary UI and canceling the selection state of the object.
(Configuration 9) A program described in any one of configurations 1 to 8, characterized in that after the auxiliary UI is displayed on the touch display, if it is determined that it is not easy to select an object corresponding to the auxiliary UI, the program causes the information processing device to execute a process of temporarily hiding the auxiliary UI.
(Configuration 10) A program described in any one of configurations 1 to 9, characterized in that when the start of a swipe operation for changing the display content of the touch display is detected while the auxiliary UI is displayed on the touch display, the program causes the information processing device to execute a process of hiding the auxiliary UI.
(Configuration 11) A program described in any one of configurations 1 to 10, characterized in that when an object other than an object corresponding to the auxiliary UI is selected in accordance with the user operation, the program causes the information processing device to execute a process of hiding the auxiliary UI.
(Configuration 12) A program described in any one of configurations 1 to 11, characterized in that when another object is placed so as to cover at least a portion of the auxiliary UI, the program causes the information processing device to execute a process of displaying the auxiliary UI in the top layer.
(Configuration 13) A program described in any one of configurations 1 to 11, characterized in that, when another object is placed so as to cover at least a portion of the auxiliary UI, the program causes the information processing device to execute a process of displaying the auxiliary UI in a position that does not overlap the other object.
(Configuration 14) A program described in any one of configurations 1 to 13, characterized in that the program causes the information processing device to execute a process of controlling the display position of the auxiliary UI based on information indicating the dominant hand of a user operating the information processing device.
(Configuration 15) A program described in any one of configurations 1 to 14, characterized in that when it is determined that it is not easy to select multiple objects among the objects displayed on the touch display, the program causes the information processing device to execute a process of displaying the auxiliary UI of the lowest object among the multiple objects determined to be difficult to select, and setting the lowest object to a selected state.
(Configuration 16) The program described in Configuration 15, characterized in that the information processing device executes a process of switching an object to which the auxiliary UI is to be assigned from among a plurality of objects determined to be difficult to select in accordance with a user operation.
(Configuration 17) A program described in any one of configurations 1 to 14, characterized in that when it is determined that selection operation of multiple objects among the objects displayed on the touch display is not easy, the program causes the information processing device to execute a process of displaying the auxiliary UI for all objects determined to be difficult to select and selecting the lowest object among the multiple objects determined to be difficult to select.
(Configuration 18) A program described in any one of configurations 1 to 17, characterized in that when a predetermined area corresponding to the touch position of the touch display by the user includes multiple objects for which the selection operation is determined to be difficult, the program causes the information processing device to execute a process of displaying another auxiliary UI different from the auxiliary UI, which assists in the selection operation of the multiple objects.

10 CPU
14 Touch display 31 Object 71 Background image 72 Object 73 Processing target object 74 Boundary area 75 Boundary area 81 Processing target object 82 Object 83 Object 90 Selection editing support item 91 Item 92 Item 93 Item 161 Object 181 Object 203 Object 221 Object 222 Object 231 Button 251 Object 252 Object 272 Selection support UI
401 Object 902 Item 903 Item 911 Item 912 Item 913 Item 931 Item 932 Item 933 Item 1001 Information processing device

Claims (20)

An information processing device having a touch display that accepts user operations,
displaying an object on the touch display in accordance with a user operation received on the touch display;
a determination step of determining whether or not a selection operation of an object displayed on the touch display is easy;
When it is determined that the selection operation of an object displayed on the touch display is not easy, a program that executes a process of displaying an auxiliary UI that assists in editing operations of the object and setting the object in a selected state. The program according to claim 1, characterized in that in the determination step, if the size of an object displayed on the touch display is less than a predetermined size, it is determined that the object is not easy to select. The program according to claim 1, characterized in that in the determination step, if the color difference in the boundary area between the object displayed on the touch display and an area other than the object is not equal to or greater than a predetermined value, it is determined that the object is not easy to select. The program according to claim 1, characterized in that in the determination step, if an object displayed on the touch display is not in the top layer and the area of the visible region of the object displayed on the touch display is less than a predetermined value, it is determined that the object is not easy to select. The program according to claim 1, characterized in that the auxiliary UI has functions for rotating, moving, and resizing the object. The program according to claim 1, characterized in that the information processing device is caused to perform the determination step in response to a new object being displayed on the touch display in accordance with the user operation. The program according to claim 1, characterized in that the information processing device is caused to perform the determination step in response to the editing content corresponding to the user operation being reflected in the object displayed on the touch display. The program according to claim 7, further comprising: making the information processing device execute a process of hiding the auxiliary UI and canceling the selection state of the object when it is determined that the object for which the auxiliary UI is displayed is easy to select in accordance with the editing content corresponding to the user operation being reflected in the object displayed on the touch display. The program according to claim 1, further comprising causing the information processing device to execute a process of temporarily hiding the auxiliary UI if it is determined that the selection operation of an object corresponding to the auxiliary UI is not easy after the auxiliary UI is displayed on the touch display. The program according to claim 1, characterized in that, when the start of a swipe operation for changing the display content of the touch display is detected while the auxiliary UI is displayed on the touch display, the program causes the information processing device to execute a process of hiding the auxiliary UI. The program according to claim 1, further comprising causing the information processing device to execute a process of hiding the auxiliary UI when an object other than the object corresponding to the auxiliary UI is selected in accordance with the user operation. The program according to claim 1, characterized in that it causes the information processing device to execute a process of displaying the auxiliary UI in the top layer when another object is placed so as to cover at least a portion of the auxiliary UI. The program according to claim 1, characterized in that, when another object is arranged so as to cover at least a part of the auxiliary UI, the program causes the information processing device to execute a process of displaying the auxiliary UI at a position that does not overlap the other object. The program according to claim 1, characterized in that it causes the information processing device to execute a process of controlling the display position of the auxiliary UI based on information indicating the dominant hand of a user who operates the information processing device. The program according to claim 1, characterized in that, when it is determined that a selection operation of multiple objects among the objects displayed on the touch display is not easy, the program causes the information processing device to execute a process of displaying the auxiliary UI of the lowest object among the multiple objects determined to be difficult to select, and placing the lowest object in a selected state. The program according to claim 15, characterized in that the information processing device is caused to execute a process of switching the object to which the auxiliary UI is to be added from among a plurality of objects determined to be difficult to select in accordance with the user operation. The program according to claim 1, characterized in that, when it is determined that multiple objects among the objects displayed on the touch display are not easy to select, the program causes the information processing device to execute a process of displaying the auxiliary UI for all objects determined to be difficult to select and selecting the lowest object among the multiple objects determined to be difficult to select. The program according to claim 1, characterized in that, when a predetermined area corresponding to a touch position on the touch display by a user includes multiple objects for which the selection operation is determined to be difficult, the program causes the information processing device to execute a process of displaying another auxiliary UI different from the auxiliary UI, which assists in the selection operation of the multiple objects. An information processing device having a touch display that accepts user operations,
means for displaying an object on the touch display in accordance with a user operation received on the touch display;
a means for determining whether an object displayed on the touch display can be easily selected;
An information processing device characterized in that, when it is determined that the selection operation of an object displayed on the touch display is not easy, an auxiliary UI is displayed to assist in the editing operation of the object, and the object is placed in a selected state. A method for controlling an information processing device having a touch display that accepts user operations, comprising:
displaying an object on the touch display in accordance with a user operation received on the touch display;
determining whether an object displayed on the touch display can be easily selected;
A control method for an information processing device, comprising the steps of: when it is determined that the selection operation of an object displayed on the touch display is not easy, displaying an auxiliary UI that assists in editing operations of the object and setting the object in a selected state.