JP2018073330A - Input device and virtual space display device - Google Patents

Abstract

Provided is an input device having a high function for input using an image display foot such that an operator is moving in a virtual space. A pressure distribution sensor 40 is disposed on the back side of a foot and detects loads at a plurality of points distributed in a planar shape. The detection unit 50 receives the pressure distribution detected by the pressure distribution sensor 40, creates pressure distribution data for the right foot and the left foot, and detects the right foot position and the left foot position. The processing unit 60 determines the front direction of the operator using the pressure distribution data used for detection by the detection unit 50. [Selection] Figure 1

Description

  The present invention relates to an input device that performs input by an operator's foot, and a virtual space display device that displays a virtual space using such an input device.

Conventionally, as described in Patent Document 1, there has been proposed an input device that inputs a display screen by movement of an operator's foot. The input device described in Patent Document 1 has a configuration in which track balls are arranged in a matrix, and has not only a pressure applied from an operator's foot, but also an input function by the operator rotating the track ball with the foot. ing.
The input device described in Patent Document 1 was developed exclusively for use in a running machine, and is intended to display an image on the assumption that the operator is always running straight and facing straight forward. Is suitable.

JP 2003-323246 A

  However, in the input device described in Patent Document 1, when an image is displayed as if the operator is moving vertically and horizontally in a virtual space that is a virtual three-dimensional space, such as a computer game. It cannot be said that it has enough functions to make a suitable input.

  An object of the present invention is to provide an input device with a high function for input using an image display foot such that an operator is moving in a virtual space.

Hereinafter, a plurality of modes will be described as means for solving the problems. These aspects can be arbitrarily combined as necessary.
An input device according to an aspect of the present invention includes a pressure distribution sensor, a detection unit, and a processing unit.
The pressure distribution sensor is disposed on the back side of the foot and detects loads at a plurality of points distributed in a planar shape.
The detection unit inputs the pressure distribution detected by the pressure distribution sensor, creates pressure distribution data for the right foot and the left foot, and detects the right foot position and the left foot position.
The processing unit determines the front direction of the operator using the pressure distribution data used for detection by the detection unit.
In this input device, since the processing unit determines the front direction of the operator using the pressure distribution data used for detection by the detection unit, the front of the virtual space and the front direction of the operator can be related, The function of inputting an image display as if the operator is moving in the virtual space is improved.

In the input device described above, the detection unit determines a first region where a pressure equal to or higher than a specific threshold is applied to each of the right foot and the left foot, and a second region where a pressure smaller than the threshold is applied. Detecting the right foot area center position, which is the center of the area of the right foot shape obtained from the shape, as the right foot position, detecting the center position of the left foot area, which is the area center of the shape of the left foot, obtained from the shape of the first region, The processing unit may determine the direction of the extending leg of the second straight line that intersects the first straight line passing through the right foot area center position and the left foot area center position at a specific angle as the front direction.
The detection unit detects the right foot load center of gravity position, which is the position of the right foot load center of gravity, from the pressing state of the right foot sole, and detects the left foot load center of gravity position, which is the position of the left foot load center of gravity, from the pressing state of the left foot sole. The processing unit uses the right foot area center position and the left foot area center position, the right foot load gravity center position, and the left foot load gravity center position detected by the detection unit to distribute the load in the longitudinal direction, which is the load distribution in the longitudinal direction of the entire body. May be determined.
In the input device described above, the detection unit determines a first region where a pressure equal to or higher than a specific threshold is applied to each of the right foot and the left foot, and a second region where a pressure smaller than the threshold is applied. The shape of the right foot obtained from the shape is detected, the shape of the left foot obtained from the shape of the first region is detected, and the processing unit fits the shape of the right foot and the specific right foot shape stored in advance and The front direction may be determined from the long-axis direction of the right foot and the long-axis direction of the left foot by fitting the shape and the previously stored specific left-foot shape.
The detection unit detects the right foot load center of gravity position, which is the position of the right foot load center of gravity, from the pressing state of the right foot sole, and detects the left foot load center of gravity position, which is the position of the left foot load center of gravity, from the pressing state of the left foot sole. Then, the processing unit may determine the front / rear load distribution, which is the load distribution in the front / rear direction of the entire body, using the fitting results of the right foot and the left foot, the right foot load gravity center position, and the left foot load gravity center position.

In the above input device, the detection unit detects from the pressure distribution data a right foot pressing state that is a pressing state of the right foot sole and a left foot pressing state that is a pressing state of the left foot sole, and the processing unit is a detection unit. The right foot pressing situation and the left foot pressing situation detected by the above may be compared to determine the left / right load distribution, which is the load distribution to the right foot and the left foot.
In the above-described input device, the detection unit detects the right foot pressing time-dependent change, which is a time-dependent change of the right foot sole pressing state, and the left foot pressing time-dependent change, which is a time-dependent change of the right foot sole pressing state. The change may be detected, and the processing unit may determine the vertical movement of the right foot from the change over time of the right foot press detected by the detection unit and determine the vertical movement of the left foot from the change over time of the left foot press.
A virtual space display device according to another aspect of the present invention includes the above-described input device and a display device that displays an image of the virtual space based on determination by a processing unit of the input device.

  The input device or the virtual space display device according to the present invention has an improved input function using an image display foot in which the operator is moving in the virtual space.

The conceptual diagram which shows the outline | summary of a structure of the virtual space display apparatus which concerns on one Embodiment of this invention. The pictorial diagram which shows an example of the image displayed with a virtual space display apparatus. The conceptual diagram for demonstrating the relationship between a display and an operator's direction. The state transition diagram which shows the transition of the input state of an input device. (A) The figure which shows the detection result of the pressure distribution in the state where the heel of the right foot is put on the pressure distribution sensor, (b) The figure which shows the detection result of the pressure distribution in the state where a lot of weight is put on the right foot, c) The figure which shows the detection result of the pressure distribution in the state where the left foot is put on the pressure distribution sensor, (d) The figure which shows the detection result of the pressure distribution in a state where the both feet stand still. The figure for demonstrating an example of a left foot press condition. The figure which shows an example of the shape of the right foot captured by the pressure distribution sensor, and the shape of the left foot. The flowchart for demonstrating the display operation relevant to an operator's front direction. The flowchart for demonstrating an example of the display operation | movement using front-and-back load distribution. The flowchart for demonstrating an example of the display operation | movement using right-and-left load distribution. The flowchart for demonstrating an example of the display operation | movement using the up-and-down movement of the right and left leg | foot. (A) Conceptual diagram showing an example of left and right feet in a state of facing the display, (b) Conceptual diagram for explaining a state where only the load is moved from the left foot to the right foot, and (c) the left foot FIG. 4 is a conceptual diagram for explaining a state in which the right foot is left in the position as it is stepped rightward, and (d) a conceptual diagram for explaining a state in which the foot is returned to face the display. The conceptual diagram for demonstrating an example of fitting.

1. First Embodiment (1) Configuration of Virtual Space Display Device A virtual space display device 10 according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a conceptual diagram showing an outline of the configuration of the virtual space display device.
The virtual space display device 10 includes an input device 20 for inputting operation information for the virtual space, and a display device 30 for displaying an image of the virtual space based on the operation information input to the input device 20.
The input device 20 includes a pressure distribution sensor 40, a detection unit 50, and a processing unit 60.

(1-1) Pressure distribution sensor The pressure distribution sensor 40 has, for example, a structure in which a plastic sheet is laid on a hard plate-like table. For example, conductive wiring is laminated on the sheet, and a plurality of pressure-sensitive sensors 41 arranged in a matrix are formed so as to correspond to the positions of each point in the XY coordinates. Each pressure-sensitive sensor 41 in the pressure distribution sensor 40 indicates a change in a physical quantity, for example, an electrical physical quantity, according to the pressure applied from above the sheet. For example, when the pressure applied to each pressure sensor 41 increases, the resistance value of each pressure sensor 41 decreases.
Accordingly, the pressure distribution sensor 40 can detect loads at a plurality of points distributed in a planar shape.
In order to extract electrical physical quantities from all the pressure sensitive sensors 41 of the pressure distribution sensor 40, for example, a wire harness 42 extends from the pressure distribution sensor 40. The wire harness 42 is connected to the A / D converter 72 of the detection unit 50. With such a configuration, the load detection result of each pressure-sensitive sensor 41 of the pressure distribution sensor 40 is sent to the detection unit 50.
The pressure distribution sensor 40 is provided with a measurement start / end button region 43. When pressure equal to or greater than a predetermined threshold is applied to the pressure-sensitive sensor 41 arranged in this region, the input device 20 starts measurement if measurement has not yet started, and performs measurement if measurement has already started. finish.

(1-2) Detection Unit The detection unit 50 includes, for example, a personal computer 71 and an A / D converter 72. In other words, the detection unit 50 includes an input interface (A / D converter 72) that converts measured values into data, and a calculation unit (for example, the CPU and memory of the personal computer 71) that processes the input data. Has been. The A / D converter 72 as an input interface converts a physical quantity input as an analog value of each pressure sensor 41 into data indicated by a digital value and transmits the data to a personal computer 71 as a calculation unit. In this way, the detection unit 50 acquires the pressure distribution data by inputting the detection result of the pressure distribution detected by the pressure distribution sensor 40.
The detection unit 50 detects the right foot position and the left foot position from the pressure distribution data for the right foot and the left foot. Further, a right foot pressing situation that is a pressing situation of the right foot sole and a left foot pressing situation that is a pressing situation of the left foot sole are detected from the pressure distribution. Further, the detection unit 50 detects the right foot load gravity center position, which is the position of the load gravity center of the right foot, from the pressing state of the sole of the right foot. Similarly, the detection unit 50 detects the left foot load gravity center position, which is the position of the load gravity center of the left foot, from the pressing state of the sole of the left foot. Further, the detection unit 50 detects a change in the right foot pressing with time, which is a change over time in the pressing state of the sole of the right foot, and a change in the left foot pressing with time, which is a change over time in the pressing state of the sole of the left foot. Detailed examples of the detection method of the right foot position and the left foot position will be described later.
The detection unit 50 transmits the detected detection information to the processing unit 60.

(1-3) Processing Unit The processing unit 60 is configured by a personal computer 71, for example. In other words, the processing unit 60 includes a calculation unit (for example, a CPU and a memory of the personal computer 71) that processes detection information obtained from the detection unit 50, and an output interface (for example, a personal computer) that outputs image information according to the processing result. And an output terminal of the computer 71).
The processing unit 60 determines the front direction of the operator 100 using the right foot position and the left position detected by the detection unit 50. In addition, the processing unit 60 determines a front / rear load distribution that is a load distribution in the front / rear direction of the entire body of the operator 100. In addition, the processing unit 60 determines left and right load distribution that is distribution of loads to the right foot and left foot of the operator 100. Further, the processing unit 60 determines the vertical movement of the left and right feet.
Then, the processing unit 60 generates virtual space image data displayed on the display device 30. The processing unit 60 generates image data so that the virtual space faces the operator 100 in the front direction. For example, with respect to the town image 200 shown in FIG. 2, it is assumed that the road 201 exists in the center of the front of the operator 100 when the operator 100 faces the town image 200. In this case, when the operator 100 stands facing diagonally right in FIG. 2, the house 202 exists at the center of the front of the operator 100. In such a state, when the operator 100 virtually walks toward the display device 30, that is, virtually walks in the diagonally right direction, the processing unit 60 seems to have moved the road 201 to the right front. Image data is generated. Thereby, it is possible to give the operator 100 a feeling of walking on a road 201 that is not a straight line in the virtual space.
In addition, when walking virtually here, the state which the operator 100 is stepping on is also included. The generation of the image data of the virtual space using the front direction determined by the processing unit 60 shown here is an example, and is not limited to the generation of such image data.
In addition, the processing unit 60 generates image data of a virtual space in consideration of the front / rear load distribution. For example, when there is a load distribution in front, the processing unit 60 determines that the head is ahead of the foot and the person is looking down. If it is determined that the user is looking down, for example, the image data in the virtual space is switched so as to look down slightly, that is, as if looking at the ground. For example, conversely, when there is load distribution behind, the processing unit 60 determines that the head recedes behind the foot and is looking up. When it is determined that the user is looking up, for example, the image data in the virtual space is switched so as to look up slightly, that is, look up toward the sky. Note that the switching of the image data in the virtual space in consideration of the front-rear load distribution shown here is an example, and is not limited to the generation of such image data.

Further, the processing unit 60 generates image data of a virtual space in consideration of the left / right load distribution. For example, when there is a load distribution on the right, the input device 20 recognizes that the operator 100 has instructed to scroll the image from right to left. For example, when a road (not shown) different from the road 201 exists further to the right of the house 202 on the right side of the road 201 in FIG. 2, the operator 100 leans his body to the right and puts his weight on the right foot. The different roads can be displayed in front of the operator 100. Note that the switching of the image data in the virtual space in consideration of the left / right load distribution shown here is an example, and is not limited to the generation of such image data.
In addition, the processing unit 60 generates image data in a virtual space considering the vertical movement of the right foot and the left foot. For example, when the right foot and the left foot are separated from each other at the same time, the processing unit 60 recognizes that the operator 100 has performed an input by jumping, and generates image data of the virtual space corresponding to the jump. For example, when jumping, image data such as jumping over an obstacle in the virtual space according to the flight time during jumping (the time from when both feet are released to landing), such as the virtual space of FIG. It is possible to generate image data of a virtual space that jumps up from the road 201 onto the roof of the house 202. Alternatively, when a plurality of types of images are prepared as virtual space images, such as shopping street images other than the back alleys, jumps are recognized as image type switching, and the back alley images are converted to shopping street images. It can be configured to switch to As to what kind of information is input by such a jump, it is possible to use it appropriately if it is decided in advance between the operator 100 and the provider of the input device 20.

  Further, for example, when the right foot and the left foot are alternately separated, the processing unit 60 recognizes that the operator 100 has walked and generates image data of a virtual space corresponding to the walk. For example, when it is recognized that the operator 100 is walking, the image data of the virtual space can be generated so that the operator 100 feels as if walking on the road 201 in the virtual space. The direction in which the operator 100 moves is, for example, the front direction of the operator 100. FIG. 2 shows the XY coordinate axes and the moving direction vw of the virtual space. By making this moving direction vw coincide with the front direction of the operator 100, the operator 100 moves to the virtual space in the direction in which he is facing. You can have a virtual experience as if you are moving in

(1-4) Display Device The display device 30 includes a display 31 that displays an image of the virtual space. The display device 30 generates an image from the image data transmitted from the processing unit 60 of the input device 20 and displays it on the display 31. As with the pressure distribution sensor 40, the display device 30 is stationary. In other words, even if the operator 100 moves, the display device 30 and the pressure distribution sensor 40 do not move. The relationship between the XY coordinates of the display device 30 (see FIG. 2) and the XY coordinates of the pressure distribution sensor 40 is fixed.
In the virtual space display device 10 according to this embodiment, the relationship between the display 31 and the operator 100 is, for example, the front direction of the operator 100 (the direction indicated by the arrow D1) as shown in FIG. Is set to fall within the range of the angle θ.

(2) State Transition FIG. 4 shows an example of state transition of the virtual space display device 10. Immediately after the power is turned on, the state is the IDLE state ST1. The IDLE state is a state of waiting for input of a start instruction having the measurement start / end button area 43 pressed. When the measurement start button is pressed, the state transitions from the IDLE state ST1 to the INIT state ST2. In the INIT state ST2, the operator 100 is asked to stand still, and information for subsequent processing includes, for example, the shape of the right foot, the shape of the left foot, the initial right foot position of the right foot, and the initial left foot position of the left foot. Detected. In the INIT state ST2, the INIT state ST2 is changed to the LOOK / AROUND state ST3 by, for example, the operation of the measurement start / end button area 43, the operation of an input device (not shown) provided separately, or the elapse of a preset initial setting time. Transition.
Note that the transition from ST1 to ST3 can be made without pressing a button or asking to stand still. For example, if the detection unit 50 detects boarding of the left and right feet in the IDLE state ST1, the transition to the INIT state ST2 is automatically performed, and the shape of the left and right feet in the INIT state ST2 is automatically detected, and the LOOK / AROUND state You may comprise so that it may change to ST3.
In the LOOK / AROUND state ST3, the operator 100 displays an image of the virtual space on the assumption that the operator 100 is stationary on the XY coordinates. That is, the operator 100 hardly moves from the current position on the XY coordinates. In the LOOK / AROUND state ST3, for example, a left turn, a right turn, switching to a display to look down, or switching to a state of looking up is performed. In the LOOK / AROUND state ST3, when the processing unit 60 recognizes that the operator 100 is walking, the transition is made from the LOOK / AROUND state ST3 to the WALK state ST4. When the processing unit 60 recognizes that the operator 100 has jumped in the LOOK / AROUND state ST3, the state transits from the LOOK / AROUND state ST3 to the JUMP state ST5. Furthermore, when the end button of the measurement start / end button area 43 is pressed in the LOOK / AROUND state ST3, the state transits from the LOOK / AROUND state ST3 to the ENDING state ST6.
Further, in the LOOK / AROUND state ST3, even when the left and right feet are not boarded for a certain period of time, the transition may be made from the LOOK / AROUND state ST3 to the ENDING state ST6.

In the WALK state ST4, the image changes as if the operator 100 is walking in the virtual space, for example. In the WALK state ST4, when the processing unit 60 recognizes that the operator 100 has returned to the standing posture, the state transits from the WALK state ST4 to the LOOK / AROUND state ST3. In addition, when the processing unit 60 recognizes that the operator 100 has jumped in the WALK state ST4, the state transits from the WALK state ST4 to the JUMP state ST5. Furthermore, when the end button in the measurement start / end button area 43 is pressed in the WALK state ST4, the state transits from the WALK state ST4 to the ENDING state ST6.
In the JUMP state ST5, the image changes as if the operator 100 jumped in the virtual space, for example. In the JUMP state ST5, when the processing unit 60 recognizes that the operator 100 has returned to a standing posture, the state transits from the JUMP state ST5 to the LOOK / AROUND state ST3. Since the jump of the operator 100 is completed in a very short time, the transition from the JUMP state ST5 to the WALK state ST4 and the transition to the ENDING state ST6 are not set in the processing unit 60.
In the ENDING state ST6, the data is initialized, the operator is returned to the IDLE state ST1, and then returned to the IDLE state ST1.

(3) Operation example of the input device 20 (3-1) Pressure distribution when standing still For example, when the operator 100 stands still in the INIT state ST2 of FIG. 4, the pressure distribution sensor 40 Then, the load applied from the soles of both the right foot and the left foot is detected. FIG. 5A, FIG. 5B, FIG. 5C, and FIG. 5D show the state of pressure distribution when the operator enters the pressure distribution sensor. FIG. 5A shows the detection result P2 of the pressure distribution in a state where the right foot heel is placed, and FIG. 5B shows the pressure distribution in the state where the entire sole of the right foot is placed. FIG. 5C shows the pressure distribution detection result P4 in the state where the entire sole of the right foot is placed and the pressure distribution in the state where most of the sole of the left foot is placed. FIG. 5D shows pressure distribution detection results P6 and P7 in a state where the entire soles of the left and right feet are placed. As shown in FIGS. 5A to 5D, the pressure distribution changes with time according to the weight shift of the operator 100 and the movement of the foot.
Therefore, when the input device 20 starts input, it is important to determine the pressing state of the right foot and the left foot, which is a reference when the operator 100 stands still. However, even if the operator 100 stands still, it is difficult for the operator 100 to completely stop the movement, and the body of the operator 100 is slightly shaken. For this reason, in this embodiment, the detection unit 50 detects the peak value of the pressure-sensitive sensor 41 during a predetermined time (herein, referred to as initial state measurement time). The detection unit 50 detects the shape of the right foot and the shape of the left foot with the peak value of the initial state measurement time.
The input device 20 is a device for inputting information necessary for displaying a virtual space. In other words, the input device 20 is required to perform accurate measurement for taking foot-related data such as foot-shaped measurement for making shoes, foot / leg measurement for medical treatment, and walking measurement. It is not a thing. Therefore, by detecting the peak value over the initial state measurement time as described above, the measured right foot shape and left foot shape may be slightly distorted from the actual right foot shape and left foot shape.

(3-2) Detection of Right Foot Position and Left Foot Position FIG. 6 shows an example of the left foot pressing situation P1, which is the pressing situation of the sole of the left foot. In FIG. 6, no pressure is applied to the pressure sensor 41 of the white block, and the pressure equal to or greater than the fourth threshold value is applied to the pressure sensor 41 of the cross-hatched block having a high density. A first threshold value, a second threshold value, and a third threshold value are set between the white block and the dense cross-hatched block. The block corresponding to the pressure sensor 41 to which the pressure of the first threshold and the second threshold is applied is hatched with low density, and the block corresponding to the pressure sensor 41 to which the pressure of the second threshold and the third threshold is applied. The block corresponding to the pressure-sensitive sensor 41 to which the pressure of the third threshold value and the fourth threshold value is applied is subjected to cross hatching with a low density.
For detection of the right foot position and the left foot position, as shown in FIG. 7, a right foot shape F1 and a left foot shape F2 are obtained. For this purpose, as shown in FIG. 6, the pressures are not classified in multiple stages, but for example, a set of locations where the pressure-sensitive sensor 41 that has detected a pressure equal to or higher than the first threshold is located as the first region Ar1. It is determined, and a set of locations where the pressure-sensitive sensor 41 in which a pressure equal to or higher than the first threshold cannot be detected (a pressure smaller than the first threshold is applied) is determined as the second region Ar2. Then, the detection unit 50 separates the first region Ar1 into a right block and a left block. The right mass of the first region Ar1 is regarded as the right foot shape F1, and the left mass is regarded as the left foot shape F2.
Further, assuming that the shape F1 of the right foot is made of a plate having a uniform density and a uniform thickness, the position of the center of gravity of the shape F1 of the right foot is calculated. Similarly, assuming that the shape F2 of the left foot is made of a plate having a uniform density and a uniform thickness, the position of the center of gravity of the shape F2 of the left foot is calculated. The position of the center of gravity of the right foot shape F1 calculated in this way is the right foot position coordinate acR (right foot area center position), and the position of the center of gravity of the left foot shape F2 is the left foot position coordinate acL (left foot area center position). It is. By calculating the average value ((xr + xl) / 2, (yr + yl) / 2) of each component of the position coordinates acR (xr, yr) of the right foot and the position coordinates acL (xl, yl) of the left foot, the position of these positions is calculated. The midpoint (position coordinate ac) is obtained.

(3-3) Display Using Front Direction of Operator 100 An example of the operation for determining the front direction of the operator 100 and displaying using the front direction will be described with reference to the flowchart shown in FIG.
In step S1, in the INIT state ST2 or the LOOK / AROUND state ST3, the input device 20 determines the front direction of the operator 100. As described in (3-2) above, the detection unit 50 determines the first area AR1 that is greater than or equal to the first threshold value from the pressure distribution data.
In step S2, the detection unit 50 classifies the first area AR1 into two left and right blocks.
In step S3, the detection unit 50 calculates the position of the center of gravity from the shapes of the left and right blocks, and detects the right foot area center (position coordinates acR in FIG. 7) and the left foot area center (position coordinates acL in FIG. 7).
In step S4, the processing unit 60 obtains a first straight line L1 that passes through the right foot area center (position coordinates acR) and the left foot area center (position coordinates acL).
In step S5, the processing unit 60 obtains a second straight line L2 orthogonal to the first straight line L1 at the midpoint (position coordinates ac) of the right foot area center and the left foot area center, and the second straight line L2 is directed toward the display 31. The extending direction is determined as the front direction v0 of the operator 100.
The front direction v0 is made to coincide with the moving direction in the virtual space in the WALK state ST4 described later.

As described above, in the input device 20, the processing unit 60 obtains the first straight line L1 passing through the right foot area center position and the left foot area center from the right foot area center position and the left foot area center position detected by the detection unit 50. Furthermore, the front direction v0 of the operator 100 is determined by obtaining a second straight line L2 orthogonal to the first straight line L1.
As a result, the input device 20 can realize the determination of the front direction v0 of the operator 100 with a simple operation. In the above embodiment, the direction in which the second straight line L2 extends is defined as the direction orthogonal to the first straight line L1, but the method of determining the direction in which the second straight line L2 extends is not limited to being orthogonal. For example, the angle may be corrected to the left or right with the load applied to the left and right feet.

Furthermore, as described above, the processing unit 60 determines the front direction of the operator 100 using the pressure distribution data used for detection by the detection unit 50. Here, it is expressed as the front direction of the operator 100. However, for example, the same applies to other expressions such as the direction of the foot or the direction of the trunk, and the direction in which the line of sight of the operator 100 is directed. It is the direction that is considered. This does not measure that the line of sight of the operator 100 is actually facing the direction, but is defined in a pseudo manner to display an image in the virtual space. And it can input, using the detection result of the right foot position and left foot position corresponding to the image display of such a front direction.
By determining the approximate front direction of the operator 100 and displaying the virtual space image accordingly, the operator 100 can input while viewing the image display as if moving in the virtual space. As a result, an input function using an image display foot such that the operator 100 is moving in the virtual space is improved.

ステップＳ１２では、検出部５０は、左足面積中心を検出した第１領域ＡＲ１の左の塊についての圧力分布のデータを左足の足裏の押圧状況として左足荷重重心位置（例えば、図７の位置座標ｇｃＬ）を検出する。左足の圧力分布のデータについて感圧エレメント（ｘｉ，ｙｉ）に掛かる荷重をＦｌｉと表し、左足に掛かる荷重の合計値をＦｌと表し、感圧エレメントの個数がｎ個であるとするとき、左足荷重重心位置の座標（Ｃｌｘ，Ｃｌｙ）は、例えば次の（６）式から（１０）式までの５つの式を用いて求められる。

(3-4) Display Using Front / Rear Load Distribution An example of the operation for determining the front / rear load distribution of the operator 100 and displaying using the same will be described with reference to the flowchart shown in FIG.
In step S11, in the LOOK / AROUND state ST3, the input device 20 determines the front / rear load distribution of the operator 100. The detection unit 50 detects the right foot load barycentric position (for example, the position coordinate gcR in FIG. 7) using the pressure distribution data of the right mass of the first area AR1 in which the center of the right foot area is detected as the pressing state of the sole of the right foot. To do. For the pressure distribution data of the right foot, when the load applied to the pressure sensitive element (xi, yi) is expressed as Fri, the total value of the load applied to the right foot is expressed as Fr, and when the number of pressure sensitive elements is n, The coordinates (Crx, Cry) of the load gravity center position can be obtained by using, for example, the following five expressions (1) to (5). In the following formulas (1) to (10), the subscript r of the mathematical expression represents a mathematical expression related to the right foot, the subscript l represents a mathematical expression related to the left foot, and the subscript i represents the item number. Represents.

In step S12, the detection unit 50 uses the pressure distribution data for the left mass of the first area AR1 that has detected the center of the left foot area as the pressing state of the left foot sole, for example, the left foot load gravity center position (for example, the position coordinates in FIG. 7). gcL) is detected. For the pressure distribution data of the left foot, the load applied to the pressure sensitive element (xi, yi) is represented as Fli, the total load applied to the left foot is represented as Fl, and the number of pressure sensitive elements is n. The coordinates (Clx, Cly) of the center of gravity position of the load are obtained using, for example, the following five expressions (6) to (10).

In step S13, the processing unit 60 determines the insensitive area Ns based on the first straight line L1. The dead area Ns is defined between two straight lines L3 and L4 by, for example, drawing two parallel straight lines L3 and L4 separated by a predetermined distance before and after the first straight line L1. When both the right foot load gravity center position coordinates (Crx, Cry) and the left foot load gravity center position coordinates (Clx, Cly) are in front, the processing unit 60 determines that the load is the front load. When both the right foot load gravity center position coordinates (Crx, Cry) and the left foot load gravity center position coordinates (Clx, Cly) are located later, the processing unit 60 determines the rear load. And it determines with "No" noting that it is not in any state except the above-mentioned pre-load and post-load.
If “No” is determined in step S <b> 13, the process proceeds to step S <b> 14, and the processing unit 60 sets the virtual space image displayed on the display device 30 to face the operator 100. If the preload is determined by the processing unit 60, the process proceeds to step S15, and the display device 30 is switched to image display looking down. Further, when it is determined that the post-load is in the processing unit 60, the process proceeds to step S16, and the display device 30 is switched to the image display to be looked up.
For example, even if it is switched to the image display in the state of looking down once, if the operator 100 returns to the posture that does not lean forward or backward, the image is returned to the directly-facing state and displayed again through steps S11 to S14. Can do.

As described above, the processing unit 60 uses the right foot area center position and the left foot area center position detected by the detection unit 50 and the right foot load gravity center position and the left foot load gravity center position to perform a preload, a rear load, a front load, or a rear load. It is determined whether there is no state. The determination method of the processing unit 60 described with reference to FIG. 9 is an example of a determination method of front and rear load distribution.
In the above embodiment, the front and rear load distribution is used to display the front load, the rear load, and the state that is neither the front load nor the rear load, in accordance with the case of looking down, the case of looking up, and the case of facing each other. ing. By displaying not only the directly facing image but also the image looking down or the image looking up, the realism of the three-dimensional virtual space can be improved.

(3-5) Display Using Left / Right Load Distribution An example of determination of the left / right load distribution of the operator 100 and a display using the same will be described with reference to the flowchart shown in FIG.
In the LOOK / AROUND state ST3, the input device 20 determines the front / rear load distribution of the operator 100.

In step S21, the detection unit 50 detects the right foot pressing state from the pressure distribution data for the right mass of the first region AR1 in which the center of the right foot area is detected.
In step S22, the detection unit 50 detects the left foot pressing state from the pressure distribution data for the left mass of the first region AR1 in which the center of the left foot area is detected.
In step S23, the processing unit 60 calculates the sum of the loads in the right foot pressing situation and the loads in the left foot pressing situation from the right foot pressing situation and the left foot pressing situation detected by the detection unit 50.

In step S24, the processing unit 60 calculates a left-right load difference that is a difference dSL (= SLr−SLl) between the total load SLr of the right foot pressing situation and the total load SLl of the left foot pressing situation, and the left-right load difference is discriminated from left to right. It is determined whether or not the threshold value THrl is exceeded. If dSL> 0 and | dSL |> THrl, the processing unit 60 determines that it is tilted to the right. If dSL <0 and | dSL |> THrl, it is determined that the processing unit 60 is tilted to the left. If | dSL | ≦ THrl, it is determined that the robot is not inclined to the left or right.
In the display device 30, if the processing unit 60 determines that the image is not tilted to the left or right, the process proceeds to step S25, and the display is continued without turning the image to the left or right. If it is determined that the processing unit 60 is tilted to the right, the process proceeds to step S26, and the display device 30 is switched to an image display that turns to the right. If it is determined that the processing unit 60 is tilted to the left, the process proceeds to step S27, and the display device 30 is switched to an image display that turns to the left.
For example, even if it is switched to the image display that turns to the right once, if the operator 100 returns to the posture that does not tilt to the left or right, it returns to the image display that does not turn to the left or right again through steps S21 to S25. .

As described above, the processing unit 60 determines the right / left load distribution by comparing the right foot pressing state and the left foot pressing state detected by the detection unit 50. The determination method of the processing unit 60 described with reference to FIG. 10 is an example of a determination method of left and right load distribution.
For example, when the operator makes a right turn when the center of gravity of the operator is on the right side of the center of the pressure distribution sensor 40 and turns when the center of gravity is on the left side of the pressure distribution sensor 40, the input is always made while paying attention to the center of the pressure distribution sensor 40. Since it is necessary, it is difficult to input if it is determined whether the center of gravity is located on the left or right side of the center of the sensor. In the above embodiment, the input device 20 can input a right turn and a left turn using the left / right load distribution. For example, even when standing in the vicinity of the right end of the pressure distribution sensor 40, the right foot pressing situation and the left foot pressing situation. Since the left and right load distribution is determined by comparing with the right and left, it is possible to perform both the right turn and the left turn as well as the right turn. Thus, the input using the left / right load distribution is easy for the operator 100 to handle.

(3-6) Display Using Vertical Movement of Left and Right Feet An example of determination of vertical movement of the left and right feet of the operator 100 and an example of display using the same will be described with reference to the flowchart shown in FIG.
In the LOOK / AROUND state ST3, the input device 20 determines the vertical movement of the operator 100. If the input device 20 is determined to be in a walking state, a transition is made from the LOOK / AROUND state ST3 to the WALK state ST4, and if the input device 20 is determined to be in a jumping state, the LOOK / AROUND state is displayed. A transition is made from the state ST3 to the JUMP state ST5 and display is performed.
The detection unit 50 stores pressure distribution data obtained repeatedly at a predetermined cycle. In step S31, the detection unit 50 detects a change in the right foot pressing with time, which is a change in the pressing state of the sole of the right foot, with the passage of time in the stored pressure distribution data. Similarly, in step S31, the detection unit 50 detects a left foot pressing temporal change, which is a temporal change in the pressing state of the sole of the left foot, from the temporal change of the stored pressure distribution data.

In step S32, for example, the processing unit 60 obtains the temporal change in the total load of the right foot pressing situation from the temporal change in the right foot press, obtains the temporal change in the total load of the left foot pressing situation from the temporal change in the left foot, and For each of the above, it is detected that the foot has left when the peak value of the sum total of the most recent loads becomes 5% or less.
In step S <b> 33, when it is determined that both the left and right feet are separated from the pressure distribution sensor 40 during the jump determination time, the processing unit 60 determines a jump pattern. In step S33, the processing unit 60 determines that either the left or right leg is separated from the pressure distribution sensor 40 during the first walking determination time and the one leg left during the second walking determination time following the first time. When the pressure distribution sensor 40 is pressed and the opposite foot is separated from the pressure distribution sensor 40, it is determined as a walking pattern. That is, the processing unit 60 determines whether the left and right feet are separated from the pressure distribution sensor 40 substantially at the same time or alternately from the pressure distribution sensor 40 one foot at a time. When walking, for example, it is common that only the right foot presses the pressure distribution sensor 40, then presses the pressure distribution sensor 40 with both feet, and then only the left foot presses the pressure distribution sensor 40. However, the case where both feet do not press the pressure distribution sensor 40 between the landing of the right foot and the left foot, such as when running, may be included. The cases where the jump pattern and the walking pattern are not determined are all in the upright pattern (step S33), but the standing pattern for a while with one foot can be included in the upright pattern, and in this case, it turns right or left. .

If the processing unit 60 determines that the pattern is an upright pattern, the process proceeds to step S34, and the display device 30 displays an image of the virtual space on the assumption that the operator 100 has not moved from the location in the virtual space. to continue. If the walking pattern is determined in the processing unit 60, the process proceeds to step S35, and in the display device 30, the speed at which the operator 100 is walking is the speed of the up and down movement that alternately moves the right foot and the left foot. Recognizing and displaying an image that allows the operator 100 to feel that the operator 100 is moving in the front direction v0 in accordance with the walking speed in the virtual space. If the processing unit 60 determines that the pattern is a jump pattern, the process proceeds to step S36, and the display device 30 switches the image display according to the jump of the operator 100.
For example, even if it is determined that the pattern is a walking pattern and the image display is switched, if the operator 100 returns to the upright posture, the process returns to the image display of the upright pattern again through steps S31 to S34. Such a change is accompanied by a transition from the WALK state ST4 to the LOOK / AROUND state ST3. For example, even if the jump pattern is once determined and the image display is switched, if the operator 100 returns to the upright posture, the display returns to the image display of the upright pattern again through steps S31 to S34. Such a change is accompanied by a transition from the JUMP state ST5 to the LOOK / AROUND state ST3.
The right foot shape F1 shown in FIG. 7 is updated when the right foot leaves and then lands, when a certain time has elapsed since landing or when the right foot leaves the next time, and accordingly, Other values are also updated.
The shape F2 of the left foot shown in FIG. 7 is updated when the left foot leaves, then lands, or after a certain period of time has elapsed since landing, or the next time the left foot leaves, The value of is also updated.

As described above, the processing unit 60 determines the vertical movement of the right foot from the change over time of the right foot press detected by the detection unit 50 and determines the vertical movement of the left foot from the change over time of the left foot press.
The determination method of the processing unit 60 described with reference to FIG. 11 is an example of a determination method of the vertical movement of the left and right feet. In the above embodiment, the movement of the operator 100 in the three-dimensional virtual space is determined by determining whether the walking pattern, the upright pattern, or the jump pattern is applied using the vertical movement of the left and right feet. Is entered. Since the operator 100 can actively move the position in the virtual space, the sense of presence in the three-dimensional virtual space can be improved.

(4) Modification (4-1) Modification A
In the above embodiment, the display device 30 and the pressure distribution sensor 40 are stationary, and the relationship between the XY coordinates of the display device 30 and the XY coordinates of the pressure distribution sensor 40 is fixed as an example. explained. However, the relationship between the XY coordinates of the display device 30 and the XY coordinates of the pressure distribution sensor 40 is not limited to being fixed.
For example, the display device 30 may have a display that moves with the movement of the body, such as a head-mounted display, and the display device 30 corresponds to detection information from a gyro sensor included in the moving display. The relationship between the XY coordinates of 30 and the XY coordinates of the pressure distribution sensor 40 may be adjusted as appropriate.

フィッティングによる正面方向ｖ０の単位ベクトルの求め方の一例について図１３を用いて説明する。パーソナルコンピュータ７１のメモリには、ＩＮＩＴ状態ＳＴ２で得られた操作者１００の右足の形状及び左足の形状が特定右足形状ＳＦ１及び特定左足形状ＳＦ２として記憶される。フィッティングは、例えば、ＬＯＯＫ・ＡＲＯＵＮＤ状態ＳＴ３での右足の形状Ｆ１と特定右足形状ＳＦ１とを重ねたときに、特定右足形状ＳＦ１を回転させたり移動させたりして最も重なり部分の面積が大きくなる状態を処理部６０において決定することで行われる。なお、左右の長軸方向の単位ベクトルｖｒとｖｌは、特定右足形状ＳＦ１と特定左足形状ＳＦ２に対して予め設定されている。
この場合も右足面積中心と左足面積中心を求めて上述のように前後荷重配分を決定してもよいが、右足の形状Ｆ１に重なった特定右足形状ＳＦ１と左足の形状Ｆ２に重なった特定左足形状ＳＦ２の右足面積中心と左足面積中心は、右足の形状Ｆ１と左足の形状Ｆ２から求まる右足面積中心と左足面積中心の近傍にそれぞれ位置する。従って、右足及び左足のフィッティング結果と右足荷重重心位置及び左足荷重重心位置を用いて、前後荷重配分を決定してもよい。
（４−３）変形例Ｃ
上記実施形態では、右足荷重重心位置と左足荷重重心位置（図７の位置座標ｇｃＲ，ｇｃＬ）を使って前後荷重配分を決定したが、本発明はそのような実施例に限定されない。
例えば、右足荷重重心位置と左足荷重重心位置から全体の重心位置（例えば、図７の位置座標ｇｃ）を決定して、全体の重心位置と右足面積中心位置及び左足面積中心位置との関係から前後荷重配分を決定してもよい。例えば、全体の重心位置が不感領域Ｎｓよりも前にあれば前荷重、不感領域Ｎｓよりも後にあれば後荷重と決定するなどである。 (4-2) Modification B
In the above embodiment, the case where the first straight line and the second straight line are determined from the right foot area center position and the left foot area center position to determine the front direction of the operator 100 has been described. However, the present invention is not limited to such an example.
By fitting with the shape of the foot of the operator 100 obtained in the INIT state ST2, the unit vectors vr and vl in the left and right major axis directions are obtained, and the front direction v0 of the operator 100 is obtained from the following equation (11). May be obtained.

An example of how to obtain the unit vector in the front direction v0 by fitting will be described with reference to FIG. In the memory of the personal computer 71, the shape of the right foot and the shape of the left foot of the operator 100 obtained in the INIT state ST2 are stored as the specific right foot shape SF1 and the specific left foot shape SF2. In the fitting, for example, when the right foot shape F1 and the specific right foot shape SF1 are overlapped in the LOOK / AROUND state ST3, the specific right foot shape SF1 is rotated or moved to maximize the area of the overlapping portion. Is determined by the processing unit 60. Note that the unit vectors vr and vl in the left and right major axis directions are set in advance for the specific right foot shape SF1 and the specific left foot shape SF2.
In this case, the right and left load area centers may be obtained by determining the right foot area center and the left foot area center as described above. However, the specific right foot shape SF1 that overlaps the right foot shape F1 and the specific left foot shape that overlaps the left foot shape F2. The right foot area center and the left foot area center of SF2 are respectively located in the vicinity of the right foot area center and the left foot area center obtained from the right foot shape F1 and the left foot shape F2. Therefore, the front / rear load distribution may be determined using the fitting results of the right foot and the left foot, the right foot load gravity center position, and the left foot load gravity center position.
(4-3) Modification C
In the above embodiment, the front / rear load distribution is determined using the right foot load gravity center position and the left foot load gravity center position (position coordinates gcR, gcL in FIG. 7), but the present invention is not limited to such an embodiment.
For example, the overall gravity center position (for example, the position coordinate gc in FIG. 7) is determined from the right foot load gravity center position and the left foot load gravity center position, and the relationship between the overall gravity center position, the right foot area center position, and the left foot area center position is The load distribution may be determined. For example, if the entire center of gravity position is before the insensitive area Ns, it is determined as a preload, and if it is after the insensitive area Ns, it is determined as a postload.

(4-4) Modification D
For example, the dead interval will be briefly described with reference to FIGS. 12 (a), 12 (b), 12 (c), and 12 (d). In FIG. 12 (a), the operator 100 stands still with the body weight almost evenly applied to the right foot FR1 and the left foot FL1 facing the display 31. Next, in the state shown in FIG. 12B, only the load is moved from the left foot FL1 to the right foot FR2, and the vehicle stands still. Therefore, in the state of FIG. 12B, the image of the virtual space turns to the right. In FIG. 12B, the right turn is indicated by an arrow AM1, and the movement of the load is indicated by an arrow AM2. FIG. 12C shows a case where, for example, the operator 100 who has lost the sense of equilibrium due to the right turn of the image in the virtual space steps on the left foot FL3 in the right direction and the right foot FR3 is in the same position. . In this state, since the image in the virtual space changes unexpectedly, the operator 100 hurries back to face the display 31 as shown in FIG. If the input device 20 recognizes the state of FIG. 12C as a change in input, the image of the virtual space is frequently switched, and it becomes difficult to see. Therefore, the input device 20 may set a dead interval in order to ignore short-time changes other than jumps. That is, in a state where one foot is not separated, the input device 20 is set so that it is not recognized as a new input unless the input continues beyond the dead interval. If such a dead interval is set, even if the operator 100 in FIG. 12C performs an erroneous operation, the image in the virtual space can be kept unchanged.

2. Features of Embodiments An input device 20 (an example of an input device) includes a pressure distribution sensor 40 (an example of a pressure distribution sensor), a detection unit 50 (an example of a detection unit), a processing unit 60 (an example of a processing unit), Is provided.
The pressure distribution sensor 40 is disposed on the back side of the foot and detects loads at a plurality of points distributed in a planar shape.
The detection unit 50 receives the pressure distribution detected by the pressure distribution sensor 40, creates pressure distribution data for the right foot and the left foot, and detects the right foot position and the left foot position.
The processing unit 60 determines the front direction of the operator 100 (an example of the operator) using the pressure distribution data used for detection by the detection unit 50.
In the input device 20, since the processing unit 60 determines the front direction of the operator 100 using the pressure distribution data used for detection by the detection unit 50, the front of the virtual space and the front direction of the operator 100 are determined. Since they are related to each other, the function of inputting an image display as if the operator 100 is moving in the virtual space is improved.

3. Other Embodiments Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the scope of the invention. In particular, a plurality of embodiments and modifications described in this specification can be arbitrarily combined as necessary.

  The present invention can be widely applied to an input device that performs input by an operator's foot and a virtual space display device that displays a virtual space using such an input device.

10: Virtual space display device 20: Input device 30: Display device 31: Display 40: Pressure distribution sensor 41: Pressure sensor 42: Wire harness 43: Measurement start / end button region 50: Detection unit 60: Processing unit 71: Personal Computer 72: A / D converter 100: Operator 200: Image 201: Road 202: House

Claims (8)

A pressure distribution sensor that is disposed on the back side of the foot and detects a load of a plurality of points distributed in a planar shape;
A detection unit that inputs the pressure distribution detected by the pressure distribution sensor, creates data of the pressure distribution for the right foot and the left foot, and detects the right foot position and the left foot position;
A processing unit that determines the front direction of the operator using the data of the pressure distribution used for the detection of the detection unit;
An input device comprising: The detection unit discriminates a first region where a pressure equal to or higher than a specific threshold is applied to the right foot and the left foot, and a second region where a pressure smaller than the threshold is applied, and the shape of the first region The right foot area center position, which is the center of the area of the right foot shape obtained from the above, is detected as the right foot position, and the left foot area center position, which is the center of the area of the left foot shape obtained from the shape of the first region, is detected as the left foot position. Detect as
The processing unit determines the direction of the extending leg of the second straight line that intersects at a specific angle with respect to the first straight line passing through the right foot area center position and the left foot area center position as the front direction.
The input device according to claim 1. The detection unit detects a right foot load gravity center position, which is a position of the right foot load center of gravity, from a pressing situation of the right foot sole and a left foot, which is a position of the left foot load gravity center, from the pressing situation of the left foot sole. Detect the center of gravity of the load,
The processing unit uses the right foot area center position and the left foot area center position detected by the detection unit, and the right foot load gravity center position and the left foot load gravity center position to distribute the load in the front-rear direction of the entire body. Determine a certain longitudinal load distribution,
The input device according to claim 2. The detection unit discriminates a first region where a pressure equal to or higher than a specific threshold is applied to the right foot and the left foot, and a second region where a pressure smaller than the threshold is applied, and the shape of the first region Detecting the shape of the right foot obtained from the above, detecting the shape of the left foot obtained from the shape of the first region,
The processing unit fits the shape of the right foot and the specific right foot shape stored in advance, and fits the shape of the left foot and the specific left foot shape stored in advance to obtain the longitudinal direction of the right foot Determining the front direction from the longitudinal direction of the left foot;
The input device according to claim 1. The detection unit detects a right foot load gravity center position, which is a position of the right foot load center of gravity, from a pressing situation of the right foot sole and a left foot, which is a position of the left foot load gravity center, from the pressing situation of the left foot sole. Detect the center of gravity of the load,
The processing unit determines a front / rear load distribution, which is a load distribution in the front / rear direction of the entire body, using the fitting result of the right foot and the left foot, and the right foot load gravity center position and the left foot load gravity center position,
The input device according to claim 4. The detection unit detects a right foot pressing situation that is a pressing situation of the sole of the right foot and a left foot pressing situation that is a pressing situation of the sole of the left foot from the data of the pressure distribution,
The processing unit compares the right foot pressing state and the left foot pressing state detected by the detection unit, and determines left and right load distribution that is distribution of load to the right foot and the left foot.
The input device according to claim 1. The detection unit detects a change in the right foot pressing over time from the time distribution of the pressure distribution data and a time change in the left foot pressing over time from a change in the right foot pressing over time. Detect
The processing unit determines the vertical movement of the right foot from the right foot press temporal change detected by the detection unit and determines the left foot vertical movement from the left foot press temporal change,
The input device according to claim 1. An input device according to any one of claims 1 to 7,
A virtual space display device comprising: a display device that displays an image of a virtual space based on the determination by the processing unit of the input device.

Images