JP2018084792A - Image processing apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve convenience of a deformable display object compared with a case where an image processing apparatus does not change the display content according to deformation of the display object.SOLUTION: An image processing apparatus comprises: a drawing part that, when a deformable display object separated from the body of the image processing apparatus is deformed from a first shape before the display object is bent to a second shape after the display object is bent, adds an edition of an image corresponding to the bent position to a partial image itself displayed on the display object over the bent position.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image processing apparatus and a program.

  Patent Document 1 describes a flexible portable terminal device in which a folding part that can be bent or folded in the front direction or the back direction is arranged at the center in the long side direction of the terminal device.

JP 2014-161209 A

  Some display devices that display images have a deformable display surface. However, in these display devices, it is not assumed that display contents are changed by using deformation of the display surface.

  An object of the present invention is to improve the convenience of a display object that can be deformed as compared with the case where the display content is not changed according to the deformation of the display object.

According to the first aspect of the present invention, when the deformable display object separated from the main body of the image processing apparatus is deformed from the first shape before bending to the second shape after bending, the bending position is straddled. The image processing apparatus includes a drawing unit that adds the image corresponding to the bending position to the partial image displayed on the display target.
The invention according to claim 2 is the image processing device according to claim 1, wherein the drawing unit edits an image projected on the sheet-like display target.
The invention according to claim 3 is the image processing apparatus according to claim 1, wherein the display target is an aerial display on which an image projected in the air is displayed.
The invention according to claim 4 is the image according to claim 3, wherein the drawing unit edits an image displayed on the aerial display in accordance with an image obtained by imaging a user's gesture for folding the display target. It is a processing device.
In the invention according to claim 5, the drawing unit edits an image displayed on the aerial display according to an output of a sensor that detects a fluctuation of air accompanying a gesture of a user who bends the display target. An image processing apparatus according to claim 3.
The invention according to claim 6 is the image processing apparatus according to claim 1, wherein the editing of the image is addition of a line segment to an overlapping portion between the partial image itself and the bending position.
The invention according to claim 7 is the image processing apparatus according to claim 1, wherein in the editing of the image, one of the partial images is left and the other is erased with the bending position as a boundary. .
The invention according to claim 8 is the image processing according to claim 1, wherein the editing of the image is added to the partial image itself after the display object returns from the second shape to the first shape. Device.
According to the ninth aspect of the present invention, when the deformable display object separated from the main body of the image processing apparatus is deformed by the computer from the first shape before bending to the second shape after bending, the bending is performed. This is a program for executing a function of editing an image corresponding to the bending position to the partial image itself displayed on the display target across the position.

According to the first aspect of the present invention, it is possible to improve the convenience of a display object that can be deformed as compared with the case where the display content is not changed according to the deformation of the display object.
According to the second aspect of the present invention, it is possible to improve the convenience of the display object that can be deformed as compared with the case where the display content is not changed according to the deformation of the display object.
According to the third aspect of the present invention, the convenience of the deformable display object can be improved as compared with the case where the display content is not changed according to the deformation of the display object.
According to the fourth aspect of the present invention, it is possible to improve the convenience of the display object that can be deformed as compared with the case where the display content is not changed according to the deformation of the display object.
According to the fifth aspect of the present invention, it is possible to improve the convenience of the display object that can be deformed as compared with the case where the display content is not changed according to the deformation of the display object.
According to the sixth aspect of the present invention, it is possible to improve the convenience of the display object that can be deformed, compared to the case where the display content is not changed according to the deformation of the display object.
According to the seventh aspect of the present invention, it is possible to improve the convenience of the display object that can be deformed as compared with the case where the display content is not changed according to the deformation of the display object.
According to the eighth aspect of the present invention, it is possible to improve the convenience of the display object that can be deformed as compared with the case where the display content is not changed according to the deformation of the display object.
According to the ninth aspect of the present invention, it is possible to improve the convenience of the display object that can be deformed as compared with the case where the display content is not changed according to the deformation of the display object.

It is a figure which shows the external appearance structure of the display apparatus which can deform | transform. It is a figure explaining the hardware structural example of a display apparatus. It is a figure explaining the example of arrangement | positioning of a deformation | transformation detection part. It is a block diagram explaining the functional structural example of the control part regarding the processing function which changes a display content according to a deformation | transformation of a display part. It is a figure explaining the relationship between the detection position of a deformation | transformation, and the corresponding bending position. It is a figure explaining the method for inputting a bending position more correctly. It is a flowchart explaining the outline | summary of the process sequence performed with a control part. It is a figure explaining the transition of the display content when the display mode which changes the display content of a display part is selected only while the bending state is detected. It is a figure explaining the transition of the display content in case the display mode which changes the display content of a display part is selected until the detection of the next bending state during the detection of a bending state. It is a figure explaining the transition of the display content when the display mode which changes the display content of a display part is selected only from the end of the detection of a bending state to the detection of the next bending state. It is a figure explaining the example which uses the specified bending position for image editing. It is a figure explaining the example which uses the specified bending position for other image editing. When the display part pulled out from the storage container is bent in the same direction at two places in the pulling direction (when viewed from above, three small screens are formed in a U shape), the user can It is a figure which shows the display form in the case of observing three small screens arrange | positioned on the same surface. When the display part pulled out from the storage container is bent in the same direction at two places in the pulling direction (when viewed from above, three small screens are formed in a U shape), the user can It is a figure which shows the display form in the case of observing from the outer diagonal upper position of the far side bending position. When the display unit pulled out from the storage container is folded in the same direction at two places in the pulling direction (when viewed from above, three small screens are formed in a U shape), three users It is a figure which shows the display form in the case of observing from 3 directions. It is a figure explaining the switching control of the display content performed when a new bending position is detected in the position different from the small screen in front display. It is a flowchart explaining the outline | summary of the processing operation of the control part which implement | achieves switching control of a display position. It is a figure which shows the example of a screen for guiding a display form suitable for reproduction | regeneration of the display form and display image which the user specified to a user. It is a flowchart explaining the outline | summary of the processing operation performed when a control part guides a user's operation.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

<Embodiment 1>
<Configuration of display device>
FIG. 1 is a diagram showing an external configuration of a deformable display device 1. The deformation of the display device 1 in the present embodiment includes, for example, bending at an arbitrary position and bending at a predetermined position. FIG. 1 shows the appearance of the display device 1 before deformation. The display device 1 placed on a flat plate surface has a planar shape as shown in FIG. In the present embodiment, the display device 1 in a planar shape is referred to as a state before deformation.

  The display device 1 is configured by mounting a display unit 2 used for displaying an image, a control unit 3 for controlling the entire device, and the like on a flexible thin film substrate formed of plastic or the like. The display device 1 is also called a so-called flexible display. Note that in this embodiment, the display device 1 having a dedicated display function is described. However, the display device in this embodiment includes various electronic devices having a display function as long as they can be modified. . For example, the display device 1 includes a portable terminal device.

  The display unit 2 uses an organic EL (Electro Luminescence) element or a liquid crystal element as a light emitting element. The display unit 2 is driven by a drive circuit (not shown). When the light emitting element is a liquid crystal element, a light source (not shown) is also arranged.

  The display device 1 according to the present embodiment is also equipped with a speaker 4 and a device (not shown) used for sound output. In the case of FIG. 1, a pair of left and right speakers 4 are arranged near both ends of the lower side of the display device 1. However, the mounting of the speaker 4 is not essential, and an audio port or an earphone jack may be mounted instead of or together with the speaker 4. The sound data may be transmitted by wireless communication between the display device 1 and the external device.

  The display surface of the display device 1 (the surface on which the display unit 2 is arranged) has a rectangular shape. In the present embodiment, the direction along the short side of the display device 1 is referred to as the vertical direction V, and the direction along the long side is referred to as the horizontal direction H. Therefore, the display device 1 in the present embodiment has a shape that is long in the horizontal direction H. Of course, the display device 1 may have a shape that is long in the vertical direction V.

  FIG. 2 is a diagram illustrating a hardware configuration example of the display device 1. In addition to the devices described above, the display device 1 includes a storage unit 14 that stores various data, an operation reception unit 15 that receives user operations, and a plurality of deformation detection units 16 that are used to detect deformation of the display unit 2. A communication unit 17 used for communication with an external device, and a power supply unit 18 that supplies power to each unit. Each unit described above is connected to the bus 19, and data of each unit is transferred via the bus 19.

  The control unit 3 is a so-called computer, and includes a CPU (Central Processing Unit) 11 that executes programs, a ROM (Read Only Memory) 12 that stores programs such as BIOS (Basic Input / Output System) and firmware, and data, It is composed of a RAM (Random Access Memory) 13 that gives a work area to the program. The control unit 3 in the present embodiment functions as an example of an image processing device.

  The storage unit 14 includes a deformable storage device or a semiconductor memory formed by using a printing technique. The operation reception part 15 in this Embodiment is comprised with the touchscreen sensor which detects the position where the user's finger | toe touched. Accordingly, the operation reception unit 15 is arranged so as to overlap the surface side of the display unit 2.

  The deformation detection unit 16 is configured by a so-called strain sensor. The strain sensor outputs a sensor output having a magnitude corresponding to the bending amount (angle). Thus, the strain sensor is a device that can detect the deformation of the attached member. Therefore, the deformation detection unit 16 also detects deformation due to bending before reaching the folded state. FIG. 3 is a diagram for explaining an example of an arrangement position of the deformation detection unit 16. A plurality of deformation detection units 16 in FIG. 3 are arranged along each side of the display unit 2. The arrangement location and arrangement interval (arrangement density) of the deformation detection unit 16 are determined according to the size and specifications of the strain sensor. Note that the deformation detection unit 16 may be arranged so as to overlap the display unit 2.

  Although details will be described later, the control unit 3 estimates the folding position (that is, the folding position) of the display unit 2 based on the relationship between the positions where the bending detection is detected by the deformation detection unit 16. Moreover, in this Embodiment, the deformation | transformation detection part 16 is arrange | positioned on the single side | surface side (for example, display surface side) of the display part 2. FIG. But you may arrange | position the deformation | transformation detection part 16 on both surfaces of a thin film substrate. The communication unit 17 includes a communication interface. The power supply unit 18 is constituted by an integrated circuit for power supply.

<Functional configuration of control unit>
Next, the functional configuration of the control unit 3 will be described. FIG. 4 is a block diagram illustrating an example of a functional configuration of the control unit 3 related to a processing function that changes display contents according to deformation of the display unit 2. The processing functions shown in FIG. 4 are realized through execution of a program by the CPU 11.

  The control unit 3 acquires the sensor output from each of the plurality of deformation detection units 16 and specifies the bending position of the display unit 2 from the relationship between the deformation information acquisition unit 21 that detects the occurrence of the deformation and the position where the deformation is detected. A bending position specifying unit 22, a drawing unit 23 for changing display contents according to the specified bending position, a sound reproducing unit 24 for outputting voice and music from the speaker 4, and a bending of the display unit 2 by a user. And a guide unit 25 for guiding operations.

  In the case of the present embodiment, the deformation information acquisition unit 21 determines that deformation has occurred at the position of the deformation detection unit 16 corresponding to the sensor output when a change exceeding a predetermined threshold appears in the sensor output. On the other hand, the deformation information acquisition unit 21 determines that the deformation state at the position of the deformation detection unit 16 corresponding to the sensor output is released when a change that returns to a predetermined threshold value or less appears in the sensor output.

  In addition, when the deformation | transformation detection part 16 with a small sensor size is arrange | positioned densely and the arrangement | positioning of the deformation | transformation detection part 16 which detected the deformation | transformation continues, the deformation | transformation information acquisition part 21 is a plurality of continuous deformation | transformation. You may use the position where the largest sensor output is obtained in the detection part 16 as a deformation | transformation position.

  The bending position specifying unit 22 specifies the bending position of the display unit 2 based on the positional relationship of the deformation detecting unit 16 that has detected the deformation. 5 and 6 are diagrams for explaining the relationship between the deformation detection position and the corresponding bending position. Here, the bending position specifying unit 22 and the above-described deformation information acquiring unit 21 constitute an example of an acquiring unit.

  FIG. 5A illustrates a bending position specified by the bending position specifying unit 22 when deformation is detected by the two deformation detecting units 16 located near the midpoint of each short side of the display device 1. It is. In the case of FIG. 5A, the bending position specifying unit 22 determines that the display unit 2 is folded in two (laterally folded) so that the line L1 parallel to the long side of the display device 1 is a fold.

  FIG. 5B shows the bending position specified by the bending position specifying unit 22 when deformation is detected in the sensor outputs from the two deformation detecting units 16 located near the midpoint of each long side of the display device 1. FIG. In the case of FIG. 5B, the bending position specifying unit 22 determines that the display unit 2 is folded in two (vertically folded) so that the line L2 parallel to the short side of the display device 1 becomes a fold.

  FIG. 5C illustrates the bending position specified by the bending position specifying unit 22 when deformation is detected in the sensor outputs from the two deformation detecting units 16 located at the upper right corner and the lower left corner of the display device 1. It is a figure to do. In the case of FIG.5 (c), the bending position specific | specification part 22 determines with the display part 2 being folded in two (diagonally folded) so that the line L3 of the diagonal direction which goes up to the right may become a crease | fold.

  FIG. 5D shows a bending that the bending position specifying unit 22 specifies when deformation is detected in the sensor outputs from the two deformation detecting units 16 located second from the upper right corner and the upper left corner of the display device 1. It is a figure explaining a position. In the case of FIG. 5D, the bending position specifying unit 22 has the display unit 2 folded in half (diagonally folded) so that the line L4 forming the oblique side of the triangle having the upper left corner as a right angle becomes a fold. judge.

  FIG. 5 (e) shows the bending position when deformation is detected in the sensor outputs from the six deformation detectors 16 located at three places on the upper side (long side) and three places on the lower side (long side). It is a figure explaining the bending position which specific part 22 specifies. In the case of FIG.5 (e), the bending position specific | specification part 22 arrange | positions the line L5 which forms a valley fold, the line L6 which forms a mountain fold, and the line L7 which forms a valley fold in order from the left side to the right side. It is determined that the display unit 2 is folded in three.

  FIG. 6 is a diagram for explaining a method for more accurately inputting a bending position. When the bending position is specified using only the position of the deformation detection unit 16 where the deformation is detected, if the number of the deformation detection units 16 is small (the density is sparse), the specified bending position and the actual bending There may be a deviation from the position. Therefore, as shown in FIG. 6, the sensor output of the touch panel sensor, which is an example of the operation receiving unit 15, is used to input the bending position. The bending position specifying unit 22 specifies, as the bending position, the movement locus of the finger 30 that moves so as to trace the ridge line when there is a sensor input from the touch panel sensor while the deformation of the deformation detecting unit 16 is detected. To do.

  Returning to the description of FIG. The drawing unit 23 displays on the display unit 2 an image that has been changed according to the bending position specified by the bending position specifying unit 22. For example, the drawing unit 23 divides the display surface into a plurality of small screens with the specified bending position as a boundary, and displays different images on each screen. In addition, for example, the drawing unit 23 displays an image obtained by editing the region of the specified bending position among the images displayed on the display unit 2 on the display unit 2. The bending position here includes not only the bending position specified in real time but also the stored bending position (deformation history). Any of the bending positions is an example of information regarding deformation. Details of the processing operation will be described later.

  The sound reproducing unit 24 has a function of giving the speaker 4 a sound corresponding to the image displayed on the display unit 2. The guide unit 25 is a function for guiding how the display unit 2 should be bent in order to obtain a bending state necessary for realizing a display form desired by the user. As a guidance method, there are an image method and a sound method.

<Control action of control unit>
Subsequently, processing operations executed by the control unit 3 according to the present embodiment will be described. FIG. 7 is a flowchart illustrating an outline of a processing procedure executed by the control unit 3. First, the control unit 3 monitors the sensor output from the deformation detection unit 16 disposed on the display unit 2 and determines whether or not the display unit 2 has been deformed (step 101). The function of the deformation information acquisition unit 21 is used for monitoring here. While the negative result continues, the deformation information acquisition unit 21 repeats the determination operation of step 101. On the other hand, when a change exceeding the threshold appears in the sensor output, the deformation information acquisition unit 21 obtains a positive result and proceeds to step 102.

  Next, the control unit 3 specifies the bending position of the display unit 2 using the position information of the deformation detection unit 16 in which the change is detected (step 102). For this specification, the function of the bending position specifying unit 22 is used. Subsequently, the control unit 3 determines whether or not the specified bending position is used for the screen division process (step 103). In this determination, the function of the drawing unit 23 is used. Note that it is preferable that the user inputs in advance what kind of processing the specified bending position is used for.

  If a positive result is obtained, the drawing unit 23 outputs a number of images corresponding to the number of screen divisions determined by the specified bending position to each small screen (step 104). At this time, the drawing unit 23 determines the content of the image to be output to each small screen based on the parameters such as the size and shape of the small screen, the aspect ratio of the image, the number of pixels, and the like with the specified bending position as a boundary. .

Note that image switching by the drawing unit 23 is executed, for example, in the following period.
(1) Only during detection of bending state (2) Until detection of bending state and detection of next bending state (deformation history information is also used)
(3) From the end of detection of the bending state to the detection of the next bending state (only the deformation history information is used)

  Here, the end of the image switching is not limited to the detection of the next bending state. For example, the switching of images may be terminated after the display time predetermined by the user elapses or a display end instruction from the user.

  FIG. 8 is a diagram for explaining the transition of the display content when the display mode for changing the display content of the display unit 2 is selected only while the bending state is detected. In the case of FIG. 8, an image of an apple picture (hereinafter referred to as “A image”) is displayed on the display unit 2 at a time T1 before the display unit 2 is deformed.

  In the case of FIG. 8, at the time T2, the display unit 2 is folded so that two folds are formed in parallel to the short side. When the display unit 2 is bent inward at two locations on the long side, the screen of the display unit 2 is divided into three small screens. The drawing unit 23 determines an image to be displayed on each small screen according to the size and shape of the formed small screen. In the example of FIG. 8, an A image is displayed on the middle small screen, a square image (hereinafter referred to as “B image”) is displayed on the left small screen, and a triangular image (hereinafter referred to as “B image”) is displayed on the right small screen. "C image") is displayed.

  After time T3, the display unit 2 returns to the original planar state. In this state, the drawing unit 23 treats the display unit 2 as one screen and displays only the A image. In the case of this display method, the bending period by the user coincides with the change period of the display content.

  FIG. 9 is a diagram for explaining the transition of the display content when the display mode for changing the display content of the display unit 2 is selected during the detection of the bending state and until the detection of the next bending state. At time T1 before the change, only the A image is displayed on the display unit 2. Consider the case where the display unit 2 is bent in the same manner as in FIG. That is, consider a case where the screen of the display unit 2 is divided into three small screens at time T2. At this time, the drawing unit 23 displays the A image on the middle small screen, the B image on the left small screen, and the C image on the right small screen, as in the example of FIG.

  However, in the case of FIG. 9, the drawing unit 23 maintains the display contents up to immediately before the time T3 after the time when the display unit 2 returns to the flat state. In other words, even after the bending state is no longer detected, the information (history information) when deformation is detected is used to display the A image on the middle small screen and the B image on the left small screen. The C image is displayed on the small screen on the right side. This display method is characterized in that the same display contents as during folding are continued using the history information not only during the period when the user is bending the display unit 2 but also after the folding is finished.

  FIG. 10 is a diagram for explaining the transition of the display content when the display mode for changing the display content of the display unit 2 is selected only from the end of the detection of the bending state to the detection of the next bending state. In the case of FIG. 10 as well, the display contents at the time T1 are the same. However, in the case of FIG. 10, even after the display unit 2 is bent, the display surface of the display unit 2 is not divided into small screens, and only the A image is displayed.

  Then, for the first time after time T3 when the display unit 2 returns from the bent state to the original flat state, the screen of the display unit 2 is divided into three small screens, and display of an image corresponding to each is started. That is, the drawing unit 23 displays the A image on the middle small screen, the B image on the left small screen, and the C image on the right small screen. In the case of FIG. 10, the bending of the display unit 2 by the user is used only for inputting the boundary position of the small screen, and the image is visually recognized after the display unit 2 is returned to the flat state.

  Returning to the description of FIG. If a negative result is obtained in step 103, the drawing unit 23 shifts to a mode in which the specified bending position is used for image editing (step 105). That is, the drawing unit 23 shifts to a mode in which the image being displayed is edited using the specified bending position. FIG. 11 is a diagram illustrating an example in which the specified bending position is used for image editing. In FIG. 11, only the A image is displayed on the display unit 2 at time T1 before the display unit 2 is deformed. At the next time point T2, the display unit 2 is bent inward at a position slightly to the right of the screen center. At this time, there is no change in the display of the A image.

  At the subsequent time T3, the display unit 2 returns to the original flat state, but the line segment 41 is added to the image portion of the apple corresponding to the bending position (folding position) at the time T2 in the A image. In the case of FIG. 11, the input of the bending position is used as an operation for adding the line segment 41. In FIG. 11, the display of the line segment 41 is started after the detection of the bending state is completed. However, even if the line segment 41 is displayed only during the detection of the bending state as shown in FIG. Alternatively, the line segment 41 may be displayed during the detection of the bent state and after the end of the detection of the bent state as shown at time points T2 and T3 in FIG.

  FIG. 12 is a diagram illustrating an example in which the specified bending position is used for other image editing. Also in the case of FIG. 12, the same bending as in FIG. 11 is applied to the display unit 2 in the order of the time points T1, T2, and T3. However, in the case of FIG. 12, at time T3, an image obtained by cutting the apple at the specified bending position is displayed. That is, in the case of FIG. 12, the user's bending operation of the display unit 2 is used to input a part to which special effect processing (cutting processing) is applied.

  In FIG. 12, only the image after the cutting process is displayed, but the process from the start of cutting to after cutting may be displayed as a moving image. Although the cut surface is not displayed in FIG. 12, the cut surface may be displayed. By displaying the cut surface, the realism of image processing is enhanced. In the case of the present embodiment, the content of the special effect process to be executed is designated in advance by the user. Further, in the present embodiment, the cutting process is shown as an example of the special effect process, but the display unit 2 is deformed so as to be rounded by hand, and the deformed part at this time is bent through the sensor output of the deformation detecting unit 16. When the specific part 22 can be detected, the result of image processing may be displayed as if the displayed apple is crushed.

  By the way, when executing image editing, a sound effect according to the content of the image editing may be added. For example, in the example of FIG. 12, a sound effect such as a sound of cutting an apple through the speaker 4 (for example, “Zaku”) may be added. By adding sound effects, the realism of image processing is enhanced.

<Effect of Embodiment 1>
As described above, according to the present embodiment, different images can be displayed on a plurality of small screens or special effects can be given to the displayed image through the deformation operation of the display unit 2, and can be deformed. The convenience of the display device 1 having the display unit 2 is improved. In addition, by adding a sound according to the content of the special effect at the time of display, a sense of reality when displaying an image with the special effect is improved.

<Embodiment 2>
In this embodiment, a technique for switching an output position of an image according to a user's eyeball position (observation position) will be described with reference to FIGS. In the case of FIGS. 13 to 15, the deformable display device 1 is used by being pulled out of the state accommodated in the storage container 50. Here, FIGS. 13 to 15 are expressed in the XYZ coordinate system. That is, the Z axis is the height direction, and the X axis and the Y axis are the two directions giving a horizontal plane. In addition, in FIGS. 13 to 15, the three small screens to be formed are represented by the first (# 1), second (# 2), and third (# 3) in order from the far side from the storage container 50. .

  FIG. 13 shows a case where the display unit 2 pulled out from the storage container 50 is folded in the same direction at two places in the pulling direction (when viewed from above, three small screens 51, 52, 53 are formed in a U shape. Is a diagram showing a display form when the user observes three small screens 51, 52, and 53 arranged on the same surface of the thin film substrate. In FIG. 13, the user's observation position 54 is indicated by the notation of an eyeball. In FIG. 13, the display unit 2 is bent at a right angle twice in the drawing direction, but the angle formed by two adjacent surfaces at the bending position is not limited to a right angle. For example, an obtuse angle may be used.

  In the case of FIG. 13, different images are displayed on the small screens 51, 52, and 53 while the display unit 2 is bent. For example, the front landscape is displayed on the small screen 52 positioned in front of the user, the right landscape is displayed on the small screen 51 positioned on the right side, and the left landscape is displayed on the small screen 53 positioned on the left side. The observation position 54 shown in FIG. 13 is possible when the display unit 2 is mounted on one side of the thin film substrate constituting the display device 1 or when it is mounted on both sides. In the case of the present embodiment, the observation position 54 is set in the display device 1 in advance by the user. In addition, when the gaze detection sensor which detects a user's gaze direction is arrange | positioned in a part of the storage container 50 or the display apparatus 1, you may identify the observation position 54 using the detection result.

  FIG. 14 shows that when the display unit 2 pulled out from the storage container 50 is bent in the same direction at two places in the pulling direction (when viewed from above, three small screens 51, 52, 53 are formed in a U-shape. FIG. 6 is a diagram showing a display form when the user observes from a diagonally upper position outside the folding position far from the storage container 50.

  The observation position 54 shown in FIG. 14 is possible when the display unit 2 is mounted on both surfaces of the thin film substrate constituting the display device 1. In the case of FIG. 14, two small screens (small screens 51 and 52) among the display units 2 arranged outside the folded thin film substrate and one of the display units 2 arranged inside the thin film substrate. A small screen (small screen 53) is observed. Also in FIG. 14, the display unit 2 is bent at a right angle twice in the drawing direction, but the angle formed by two adjacent surfaces at the bending position is not limited to a right angle.

  Also in the case of FIG. 14, the observation position 54 is set in the display device 1 in advance by the user. In addition, when the gaze detection sensor which detects a user's gaze direction is arrange | positioned in a part of the storage container 50 or the display apparatus 1, you may identify the observation position 54 using the detection result. Also in the case of FIG. 14, different images are displayed on the small screens 51, 52, and 53 while the display unit 2 is bent.

  15 shows a case where the display unit 2 pulled out from the storage container 50 is bent in the same direction at two places in the pulling direction (when viewed from above, three small screens 51, 52, 53 are formed in a U-shape. FIG. 6 is a diagram showing a display form when three users observe from three directions.

  The observation position 54 shown in FIG. 15 is possible both when the display unit 2 is mounted on one side of the thin film substrate constituting the display device 1 and when it is mounted on both sides. In the case of FIG. 15, three small screens 51, 52, and 53 that divide the display unit 2 arranged on the same surface of the thin film substrate (the surface positioned outside in the bent state) are observed. Also in the case of FIG. 15, the display unit 2 is bent at a right angle twice in the drawing direction, but the angle formed by two adjacent surfaces at the bending position is not limited to a right angle.

  In the case of FIG. 15, on the three small screens 51, 52, 53, images dedicated to the user corresponding to each observation position 54 may be displayed. For example, the television image of channel A is displayed on the small screen 51 corresponding to the observation position 54A, the operation screen of the personal computer is displayed on the small screen 52 corresponding to the observation position 54B, and the small screen 53 corresponding to the observation position 54C. May display a television image of channel C.

  Also in the case of FIG. 15, the observation position 54 is set in the display device 1 in advance by the user. Moreover, when the gaze detection sensor which detects a user's gaze direction is arrange | positioned in a part of the storage container 50 or the display apparatus 1, you may identify the observation position 54 using the detection result. Also in the case of FIG. 15, different images are displayed on the small screens 51, 52, and 53 while the display unit 2 is bent.

  As described above, in the case of the present embodiment, the image output for the three small screens 51, 52, 53 that appear by the deformation operation of the display unit 2 is the positional relationship between each small screen and the observation position 54. Therefore, the convenience of the display device 1 having the deformable display unit 2 is improved.

<Embodiment 3>
In the present embodiment, a case will be described in which the deformation of the display unit 2 is used for switching display contents between small screens. FIG. 16 is a diagram for explaining display content switching control executed when a new bending position is detected at a position different from the small screen being displayed. In the upper part of FIG. 16, an A image is displayed on the left small screen, and a B image is displayed on the right small screen. In this case, the line L11 that gives the boundary between the two screens is parallel to the short side of the display unit 2. On the other hand, the lower part of FIG. 16 shows display contents when a line L12 corresponding to a new bending position is detected at a position different from the line L11. In the lower row, the B image is displayed on the left small screen, and the A image is displayed on the right small screen. In this way, the display contents are switched between the left and right before and after the detection of a new bending position.

  FIG. 17 is a flowchart for explaining an overview of the processing operation of the control unit 3 that realizes the display position switching control. First, the control unit 3 monitors the sensor output from the deformation detection unit 16 disposed on the display unit 2 and determines whether or not the display unit 2 has been deformed (step 201). The function of the deformation information acquisition unit 21 is used for monitoring here. While the negative result continues, the deformation information acquisition unit 21 repeats the determination operation in step 201. On the other hand, when a change exceeding the threshold appears in the sensor output, the deformation information acquisition unit 21 obtains a positive result and proceeds to step 202.

  Next, the control unit 3 specifies the bending position of the display unit 2 using the position information of the deformation detection unit 16 where the deformation is detected (step 202). For this specification, the function of the bending position specifying unit 22 is used. Subsequently, the control unit 3 determines whether or not the specified bending position is different from the previous bending position (step 203). For this determination, the function of the drawing unit 23 is used. If an affirmative result is obtained, the drawing unit 23 switches the display position of the image between the small screens with the new bending position as a boundary (step 204). On the other hand, when a negative result is obtained, the drawing unit 23 continues the current display.

  As described above, according to the present embodiment, the display position of the image displayed on the display unit 2 can be switched by an easy-to-understand operation such as deformation of the display unit 2, and the convenience of the display device 1 for the user is achieved. Is improved.

<Embodiment 4>
In the present embodiment, the display unit 2 deformation operation required for realizing a display mode desired by the user (for example, the number and arrangement of small screens used for displaying an image) and a display suitable for the image selected by the user. A function for guiding the user will be described. FIG. 18 is a diagram illustrating an example of a screen for guiding the shape of the display unit 2 suitable for the display form designated by the user to the user.

  The screen example shown in FIG. 18 is an operation example when different images are to be displayed above and below the display unit 2. In the case of the present embodiment, a dotted line 61 extending in parallel with the long side is displayed near the center of the display unit 2 and a guidance sentence 62 for instructing the user's work content is displayed. In the example of FIG. 18, it is not described whether it is mountain fold or valley fold. In addition, when displaying the guidance sentence 62, it is desirable that the content of the guidance sentence 62 is output by voice.

  FIG. 19 is a flowchart illustrating an outline of processing operations executed when the control unit 3 guides the user's operation. This function is executed through the function of the guide unit 25. First, the guide unit 25 determines whether or not guidance for a bending position is requested (step 301). While the negative result continues, the guide unit 25 repeats the determination operation in step 301.

  On the other hand, when a positive result is obtained, the guide unit 25 specifies a display form to be guided (step 302). For example, the guide unit 25 specifies a display form designated by the user from among a plurality of types of display forms displayed on the display unit 2. Next, the guide unit 25 displays a guide screen (see FIG. 18) indicating the bending position according to the specified display mode on the display unit 2 (step 303).

  The above guidance screen is an example that is easy to understand intuitively, but by guiding the bending position suitable for the content of the image to be displayed, the user can shape the shape suitable for image observation in a short time. The display unit 2 can be modified.

<Other embodiments>
In the above-described embodiment, the case where the image displayed on the display unit 2 is changed through the deformation operation of the display unit 2 has been described. However, the attached sound is reproduced only for the image displayed on the small screen at the specific position. May be. In addition, when the speaker 4 is arranged at a position corresponding to each small screen, the attached sound of the image displayed on each small screen may be reproduced through the speaker 4 at a position corresponding to each small screen. That is, a different attached sound may be reproduced for each small screen.

  In the above-described embodiment, only the output form of the display image has been described. However, among the plurality of small screens formed by the deformation of the display unit 2, a specific position (for example, three small screens on the left, middle, and right) The playback speed of the display image may be changed only for the small screen in the middle of the screen. For example, the playback speed may be doubled only for the image displayed on the middle small screen, or slow motion or frame advance may be used.

  In the above-described embodiment, the display unit 2 and the control unit 3 are integrally configured as shown in FIG. 1, but a projector type in which the display unit 2 (including the communication unit) is separated from the control unit 3. An apparatus configuration may be used. The display unit 2 in this embodiment includes an example configured as a so-called projector screen and an example configured as a so-called aerial display.

  When the display unit 2 is configured as a projector screen, the display unit 2 includes a plurality of deformation detection units 16 and a communication unit mounted on a sheet-like screen surface. The communication unit of the display unit 2 is used, for example, to transmit the sensor output of the deformation detection unit 16 to the control unit 3. On the other hand, when the display unit 2 is configured as an aerial display, the display unit 2 is realized as a mirror image. That is, the display unit 2 as an aerial display is a perceptual existence and not a physical existence. Therefore, the deformation detection unit 16 used in the display unit 2 as an aerial display detects an imaging camera that captures a gesture in which a user bends the display surface, a fluctuation in space (for example, change in air or wind) associated with the gesture, and the like. The sensor is disposed around the display unit 2. The sensor output of the deformation detection unit 16 is output to the control unit 3 through the communication unit.

  In the case of this form, the control unit 3 is included in an image processing apparatus configured as an independent casing. The drawing unit 23 in the above-described embodiment controls a light projecting unit (comprising a light source and an optical system) that is also included in the image processing apparatus, and projects an image projected on the surface (projection surface) of the display unit 2. Change the content. Even in the case of such a device configuration, the same effect as the above-described embodiment can be realized.

  Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear from the description of the scope of the claims that various modifications or improvements added to the above embodiment are also included in the technical scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Display apparatus, 2 ... Display part, 3 ... Control part, 4 ... Speaker, 14 ... Memory | storage part, 15 ... Operation reception part, 16 ... Deformation detection part, 17 ... Communication part, 18 ... Power supply part, 21 ... Deformation information Acquisition unit, 22 ... bending position specifying unit, 23 ... drawing unit, 24 ... sound reproduction unit, 25 ... guide unit, 41 ... line segment, 50 ... storage container, 51, 52, 53 ... small screen, 54, 54A, 54B 54C: Observation position

Claims (9)

When the deformable display object separated from the main body of the image processing apparatus is deformed from the first shape before bending to the second shape after bending, it is displayed on the display object across the bending position. An image processing apparatus comprising: a drawing unit for adding an image corresponding to the bending position to a partial image itself.   The image processing apparatus according to claim 1, wherein the drawing unit edits an image projected on the sheet-like display target.   The image processing apparatus according to claim 1, wherein the display target is an aerial display on which an image projected in the air is displayed.   The image processing apparatus according to claim 3, wherein the drawing unit edits an image displayed on the aerial display in accordance with an image obtained by capturing a user's gesture for folding the display target.   The image processing apparatus according to claim 3, wherein the drawing unit edits an image displayed on the aerial display in accordance with an output of a sensor that detects air fluctuation accompanying a gesture of a user who bends the display target. .   The image processing apparatus according to claim 1, wherein the editing of the image is addition of a line segment to an overlapping portion between the partial image itself and the bending position.   The image processing apparatus according to claim 1, wherein in editing the image, one of the partial images itself is left and the other is erased with the bending position as a boundary.   The image processing apparatus according to claim 1, wherein the editing of the image is added to the partial image itself after the display target returns from the second shape to the first shape. On the computer,
When the deformable display object separated from the main body of the image processing apparatus is deformed from the first shape before bending to the second shape after bending, it is displayed on the display object across the bending position. A program for executing the function of editing the image corresponding to the bending position on the partial image itself.

Images