JP5262682B2 - Head mounted display - Google Patents

Description

  The present invention relates to a head mounted display that allows a user to grasp an environment such as a building with a sense of reality.

  As shown in Patent Document 1, a thermal environment display device capable of displaying environmental information such as the temperature of a building on a display together with a floor plan of the building is known. Such an apparatus can visually grasp the environment such as the temperature of the building.

  However, in reality, since the user needs to imagine the user's vision from the floor plan of the building, it is difficult for the user to grasp the environmental information of the building with a sense of reality.

JP 11-153492 A

  An object of the present invention is to provide a head mounted display that solves the above-described problems and allows a user to grasp the environment of a building or the like with a sense of reality.

The invention according to claim 1, which has been made to solve the above problems,
A head mounting part to be mounted on the user's head;
In a head-mounted display that is attached to the front part of the head-mounted unit, displays an image, and has an image display unit that allows the user to visually recognize the image.
Sensing information measured by sensors installed at a plurality of positions, individual identification information of each sensor, information acquisition means for acquiring coordinate information of each sensor,
Head-mounted display coordinate acquisition means for calculating or acquiring the coordinates of the head-mounted display;
Gaze direction measuring means for measuring the gaze direction of the user;
Based on the coordinate information of each sensor acquired by the information acquisition unit and the coordinates of the head mounted display acquired by the head mounted display coordinate acquisition unit, the relative coordinates of the sensors based on the coordinates of the head mounted display are obtained. Sensor relative coordinate calculating means for calculating;
A visual field area calculating means for calculating a visual field area of the user from the visual line direction information of the user measured by the visual line direction measuring means;
Based on the field of view calculated by the field of view calculation unit and the sensor relative coordinates of each sensor calculated by the sensor relative coordinate calculation unit, communication is possible within the field of view and within a region extending the field of view in the field of view direction. Sensor selection means for selecting a sensor,
Sensing information image generating means for generating a sensing information image to be displayed on the image display unit based on sensor relative coordinates of each sensor selected by the sensor selecting means and sensing information of each sensor acquired by the information acquiring means;
It is characterized by having.

The invention according to claim 2 is the invention according to claim 1,
Image coordinate calculation means for calculating image coordinates, which are coordinates on an image displayed on the image display unit of each sensor selected by the sensor selection means, based on the sensor relative coordinates calculated by the sensor relative coordinate calculation means. Have
The sensing information image generation unit generates a sensing information image to be displayed on the image display unit based on the image coordinates of each sensor calculated by the image coordinate calculation unit and the sensing information of each sensor acquired by the information acquisition unit. It is characterized by that.

The invention according to claim 3 is the invention according to claims 1 to 2 ,
The information acquisition means acquires sensing information measured by a plurality of sensors arranged in a room and coordinate information of each sensor,
A boundary information storage unit storing boundary information of the room;
Boundary line synthesizing means for synthesizing the boundary line of the room with the sensing information image based on the user's gaze direction measured by the gaze direction measuring means, the boundary information, and the coordinates of the head mounted display acquired by the head mounted display coordinate acquiring means. When,
It is characterized by having.

The invention according to claim 4 is the invention according to claim 3 ,
The boundary line synthesis means is
The present invention is characterized in that the display of the boundary lines is changed for each different room, and these boundary lines are synthesized with the sensing information image.

The invention according to claim 5 is the invention according to claims 1 to 4 ,
Sensor distance calculation means for calculating distance information for each sensor head-mounted display,
The head mounted display coordinate acquisition means calculates the coordinates of the head mounted display based on the distance information calculated by the sensor distance calculation means and the individual identification information and coordinate information of each sensor acquired by the information acquisition means. And

According to the first aspect of the present invention, the sensing information measured by the sensors installed at a plurality of positions, the individual identification information of each sensor, the information acquisition means for acquiring the coordinate information of each sensor, and the coordinates of the head mounted display are obtained. Head-mounted display coordinate acquisition means for calculating or acquiring, eye-gaze direction measuring means for measuring the user's line-of-sight direction, coordinate information of each sensor acquired by the information acquisition means, and the head-mounted display coordinate acquisition means Based on the coordinates of the head mounted display, the sensor relative coordinate calculating means for calculating the relative coordinates of each sensor with the coordinates of the head mounted display as a base point, and the user's gaze direction information measured by the gaze direction measuring means, A view area calculating means for calculating a view area of the view area and the view area calculating means A sensor that selects a communicable sensor in the visual field region and a region obtained by extending the visual field region in the visual field direction based on the projected visual field region and the sensor relative coordinate of each sensor calculated by the sensor relative coordinate calculation unit Sensing information image generation for generating a sensing information image to be displayed on the image display unit based on selection means, sensor relative coordinates of each sensor selected by the sensor selection means, and sensing information of each sensor acquired by the information acquisition means Means.
For this reason, since the user can view the sensing information image superimposed on the actual user's field of view, the user can grasp the sensing information detected by the sensor with a sense of reality.

The invention according to claim 2 is a coordinate on an image displayed on the image display unit of each sensor selected by the sensor selection unit based on the sensor relative coordinate of each sensor calculated by the sensor relative coordinate calculation unit. An image coordinate calculating unit that calculates image coordinates, and the sensing information image generating unit is based on the image coordinates of each sensor calculated by the image coordinate calculating unit and the sensing information of each sensor acquired by the information acquiring unit; A sensing information image to be displayed on the image display unit is generated.
For this reason, it is possible to accurately display the sensing information image displayed by the image generation unit.

The invention according to claim 3 is based on the boundary information of the room, the line-of-sight direction of the user measured by the line-of-sight direction measuring means, the boundary information, and the coordinates of the head-mounted display acquired by the head-mounted display coordinate acquisition means. The room boundary is combined with the sensing information image. For this reason, since the boundary line of the area | region which a user cannot visually recognize can also be visually recognized with the sensing information of the area | region which a user cannot see, there exists a very realistic feeling.

In the invention according to claim 4 , since the boundary line synthesizing unit changes the display of the boundary line for each different room and synthesizes these boundary lines into the sensing information image, the user can select the room in the back and the room. It becomes possible to distinguish and recognize the sensing information image of the room in the back of the room.

The invention according to claim 5 is a sensor distance calculation unit that calculates distance information of each sensor with respect to the head-mounted display, distance information calculated by the sensor distance calculation unit, and individual identification of each sensor acquired by the information acquisition unit It has a head mounted display coordinate calculation means for calculating the coordinates of the head mounted display based on the information and the coordinate information.
For this reason, since the coordinates of the head mounted display can be accurately calculated, it is possible to accurately display the sensing information image displayed by the image generation unit.

(Outline of the present invention)
The outline of the present invention will be described below with reference to the drawings. FIG. 1 is an overall view of a head mounted display 100 of the present invention. The head-mounted display 100 includes a head-mounted unit 1 that is mounted on the user's head and an image display unit 2 that displays an image to be visually recognized by the user.

  The image display unit 2 of the head mounted display 100 directly irradiates the user's eyeball with scanned image light that is two-dimensionally scanned with laser light whose intensity is modulated according to the image, thereby allowing the user to visually recognize the image ( A scanning image light irradiation unit 20 (shown in FIG. 3) (hereinafter referred to as “display”) is included. The head-mounted display 100 of the present invention is a so-called see-through type head-mounted display that allows the user to see the outside world while visually recognizing the image displayed by the image display unit 2.

  FIG. 2 is a diagram showing the field of view of a user who wears the head-mounted display 100 on the head. The head-mounted display 100 of the present invention reads sensing information detected by the sensor 500, which is mounted in a building or the like, and can visually display the sensing information on the image display unit 2. It has become. When the sensor 500 is a sensor that detects a gas concentration, the region where the high-concentration gas 600 is retained is displayed on the image display unit 2 of the head mounted display 100 as shown in FIG. As a result, the user can visually recognize the region where the high-concentration gas 600 is accumulated in the field of view of the actual user, and the user can grasp the sensing information with a sense of reality. . The sensing information detected by the sensor 500 includes gas concentrations such as city gas and carbon monoxide, temperature, humidity, atmospheric pressure, volume, dust concentration, smoke, radiation dose, presence / absence of water, and the like. Below, the head mounted display 100 of this invention is demonstrated in detail.

(Block diagram of head-mounted display)
FIG. 3 is a block diagram of the head mounted display 100 of the present invention. The image display unit 2 includes a control unit 40. The control unit 40 includes a CPU 41, a RAM 42, a ROM 43, a nonvolatile memory 44, an input interface 45, and a wireless interface 55, which are connected to each other via a bus 46.

  A CPU (abbreviation of central processing unit) 41 performs various calculations and processes in cooperation with a RAM (abbreviation of random access memory) 42 and a ROM (abbreviation of read only memory) 43.

  The RAM 42 temporarily stores a program processed by the CPU 41 and data processed by the CPU 41 in its address space.

  Various programs and parameters for controlling the head mounted display 100 are stored in the ROM 43. The various programs are processed by the CPU 41 to realize various functions. The ROM 43 includes a sensor distance calculation unit 43a, a head-mounted display coordinate calculation unit 43b, a line-of-sight direction calculation unit 43c, a sensor relative coordinate calculation unit 43d, a field-of-view area calculation unit 43e, a sensor selection unit 43f, and an image coordinate calculation unit 43g. Programs such as the sensing information image generating unit 43h and the boundary information synthesizing unit 43i are stored. Note that these programs may be stored in the nonvolatile memory 44.

  The sensor distance calculation unit 43 a is a unit that calculates the distance between the head mounted display 100 and each sensor 500 from the received radio wave intensity of each sensor 500 received by the wireless interface 55.

  The head-mounted display coordinate calculation unit 43b (head-mounted display coordinate acquisition unit) includes the “position coordinates” of the plurality of sensors 500 close to the head-mounted display 100 and the head-mounted display 100 and the sensor 500 calculated by the sensor distance calculation unit 43a. Is a means for calculating the position coordinates of the head-mounted display 100 from the distance.

  The line-of-sight direction calculation means 43c is a means for calculating the user's line-of-sight direction based on signals detected by the azimuth sensor 53 and the tilt sensor 54.

  The sensor relative coordinate calculation unit 43d subtracts the “position coordinates” of the head mounted display 100 from the “position coordinates” of each sensor 500, and uses the “position coordinates” of each sensor 500 as the center coordinates of the head mounted display 100. It is a means for converting into “sensor relative coordinates” converted into a coordinate system.

  The visual field area calculation unit 43e is a unit that calculates the user's visual field area from the user's visual line direction calculated by the visual line direction calculation unit 43c.

  Based on the user's field of view calculated by the field of view calculation unit 43e and the “sensor relative coordinates” of each sensor 500 calculated by the sensor relative coordinate calculation unit 43d, the sensor selection unit 43f Is a means for selecting a communicable sensor 500 in a region extending in the visual field direction.

  The image coordinate calculation unit 43g is a coordinate on the image displayed on the image display unit 2 of each sensor 500 selected by the sensor selection unit 43f based on the “sensor relative coordinate” of each sensor 500 calculated by the sensor relative coordinate calculation unit 43d. It is a means for calculating “image coordinates”.

  The sensing information image generation unit 43h displays on the image display unit 2 based on the “image coordinates” of each sensor 500 selected by the sensor selection unit 43f and the “sensing information” of each sensor 500 acquired by the wireless interface 55. It is means for generating a “sensing information image”.

  The boundary information synthesizing unit 43i is based on the user's gaze direction calculated by the gaze direction calculating unit 43c, the boundary information 44b stored in the nonvolatile memory 44, and the coordinates of the head mounted display acquired by the head mounted display coordinate calculating unit 43b. This is a means for synthesizing the boundary line of the room with the “sensing information image”.

  Sensor distance calculation means 43a, head-mounted display coordinate calculation means 43b, line-of-sight direction calculation means 43c, sensor relative coordinate calculation means 43d, field-of-view area calculation means 43e, sensor selection means 43f, image coordinate calculation means 43g, sensing information image generation The means 43h and the boundary information synthesis means 43i may be configured as an ASIC (Application Specific Integrated Circuit).

  The nonvolatile memory 44 stores various settings of the head mounted display 100, a display pattern table 44a, and boundary information 44b. As shown in FIG. 18, the display pattern table 44 a is a table that represents a threshold value of “sensing information” (measured value) and how to display it as a “sensing information image” according to the threshold value. The boundary information 44b is information on the boundary coordinates of each room as shown in FIG.

  The input interface 45 is connected to the power button 50, the setting change button 51, the mode switching button 52, the azimuth sensor 53, and the tilt sensor 54, and converts the physical and logical formats of signals output from these buttons and sensors. , Hand over to bus 46.

  The power button 50 is a button for turning on / off the power to the head mounted display 100. The setting change button 51 is a button for changing the setting of the head mounted display 100. The mode switching button 52 is means (mode switching means) for switching between a “transmission mode”, a “non-transmission mode”, and a “view field entire sensor browsing mode” (transmission mode in a broad sense), which will be described later. As shown in FIG. 1, the power button 50, the setting change button 51, and the mode switching button 52 are provided on the image display unit 2.

  The azimuth sensor 53 is a sensor that measures the azimuth in the front direction of the head mounted display 100, that is, the azimuth in the horizontal direction of the user's line of sight by detecting geomagnetism.

  The tilt sensor 54 is a sensor that measures the elevation angle and depression angle of the head mounted display 100 with respect to the horizontal plane in the front direction, that is, the angle of the user's line of sight with respect to the horizontal plane. The tilt sensor 54 includes a gyro sensor.

  The azimuth sensor 53 and the tilt sensor 54 constitute a “gaze direction measuring unit” that measures the gaze direction of the user.

  The wireless interface 55 is an “information acquisition unit” that receives “sensing information”, “individual identification information”, and “coordinate information” from a plurality of sensors 500.

  The scanned image light irradiating unit 20 generates scanned image light obtained by two-dimensionally scanning laser light whose intensity is modulated according to an image, and directly irradiates the user's eyeball with the scanned image light so that the user can visually recognize the image. It is a device to let you. The scanning image light irradiation unit 20 includes a scanning light generation unit 21, a collimating optical system 22, a vertical scanning unit 23, a horizontal scanning unit 24, a relay optical system 25, and a relay optical system 26.

  The scanning light generation unit 21 is a device that reads the “sensing information image” generated by the sensing information image generation unit 43h for each dot clock, and modulates the intensity according to the read “sensing information image” to generate scanning light. is there. The scanning light generation unit 21 includes a signal processing circuit 211, a light source unit 212, and a light combining unit 213.

  The signal processing circuit 211 is connected to the bus 46 of the control unit 40. The signal processing circuit 211 is based on the “sensing information image” input from the bus 46 of the control unit 40, and serves as an element for generating scanning image light B (blue), G (green), and R (red). The image signals 214 a to 214 c are generated and output to the light source unit 212. The signal processing circuit 211 is connected to a vertical scanning control circuit 23b of the vertical scanning unit 23 described later. The signal processing circuit 211 generates a vertical drive signal 215 and outputs the vertical drive signal 215 to the vertical scanning control circuit 23b. Further, the signal processing circuit 211 is connected to a horizontal scanning control circuit 24b described later. The signal processing circuit 211 generates a horizontal drive signal 216 and outputs the horizontal drive signal 216 to the horizontal scanning control circuit 24b.

  The light source unit 212 includes a B laser driver 212a, a G laser driver 212b, an R laser driver 212c, a B laser 212d, a G laser 212e, and an R laser 212f. The B laser driver 212a drives the B laser 212d based on the B (blue) image signal 214a output from the signal processing circuit 211 for each dot clock. The B laser 212d emits intensity-modulated blue laser light based on the B (blue) image signal 214a. Similarly, the G laser 212e and the R laser 212f emit green laser light and red laser light, which are respectively intensity-modulated.

  Each of the lasers 212d to 212f includes a semiconductor laser and a fixed laser with a harmonic generation function. When a semiconductor laser is used, the drive current is directly modulated to modulate the intensity of the laser beam. When using a fixed laser with a harmonic generation function, each of the lasers 212d to 212f is provided with an external modulator to modulate the intensity of the laser light.

  The light combining unit 213 includes collimating optical systems 213a to 213c, dichroic mirrors 213d to 213f, and a coupling optical system 213g. The collimating optical systems 213a to 213c are disposed in front of the lasers 212d to 212f, respectively, and collimate the laser beams emitted from the lasers 212d to 212f. The dichroic mirrors 213d to 213f are arranged in front of the collimating optical systems 213a to 213c, respectively, and convert the laser beams collimated by the collimating optical systems 213a to 213c to only laser beams having a predetermined range of wavelengths. Optionally, reflect or transmit.

  The coupling optical system 213g is disposed in front of the dichroic mirror 213d. Blue laser light transmitted through the dichroic mirror 213d and green laser light and red laser light reflected from the dichroic mirrors 213e and 213f are incident on the coupling optical system 213g. The coupling optical system 213g condenses (mixes) the laser beams of the three primary colors and enters the optical fiber 27.

  In order to irradiate the laser beam incident on the optical fiber 27 as an image, the vertical scanning unit 23 and the horizontal scanning unit 24 scan the laser beam in the vertical direction and the horizontal direction to generate scanning image light.

  The vertical scanning unit 23 includes a resonant deflection element 23a and a vertical scanning control circuit 23b. The laser light incident on the optical fiber 27 is collimated by the collimating optical system 22 and is incident on the resonant deflection element 23a. The resonant deflection element 23a has a reflecting surface 23d that is oscillated by the vertical scanning control circuit 23b, and scans the incident laser beam in the vertical direction by reflecting the incident laser beam on the oscillating reflecting surface 23d. Based on the vertical drive signal 215 output from the signal processing circuit 211, the vertical scanning control circuit 23b generates a drive signal that swings the reflecting surface 23d of the resonant deflection element 23a.

  The horizontal scanning unit 24 includes a deflection element 24a and a horizontal scanning control circuit 24b. The deflecting element 24a has a reflecting surface 24d that is oscillated by the horizontal scanning control circuit 24b. The incident laser light is reflected by the oscillating reflecting surface 24d and scanned in the horizontal direction to scan two-dimensionally. The emitted image light is emitted to the relay optical system 26. The horizontal scanning control circuit 24b generates a driving signal for swinging the reflecting surface 24d of the deflection element 24a based on the horizontal driving signal 216 output from the signal processing circuit 211.

  The relay optical system 25 is disposed between the resonant deflection element 23a and the deflection element 24a. The relay optical system 25 converges the laser beam scanned in the vertical direction on the reflection surface 23d of the resonance type deflection element 23a and makes it incident on the reflection surface 24d of the deflection element 24a.

  When the power is turned on, the signal processing circuit 211 outputs the vertical drive signal 215 and the horizontal drive signal 216 to the vertical scanning control circuit 23b and the horizontal scanning control circuit 24b, respectively, and vibrates the reflecting surfaces 23d and 24d to stabilize the signal. Then, control is performed so that two-dimensional scanning is performed.

  The relay optical system 26 includes lens systems 26a and 26b having a positive refractive power. The image light emitted from the deflection element 24a is converted into convergent image light by the lens system 26a so that the respective image lights have their center lines substantially parallel to each other. The converged image light is converted into substantially parallel scanned image light by the lens system 26b, and is condensed so that the center line of these scanned image light converges on the pupil Ea of the user.

  In this embodiment, the laser light incident from the optical fiber 27 is scanned in the vertical direction by the vertical scanning unit 23 and then scanned in the horizontal direction by the horizontal scanning unit 24. The arrangement of the horizontal scanning unit 24 may be changed, the horizontal scanning unit 24 may be scanned in the horizontal direction, and then the vertical scanning unit 23 may be scanned in the vertical direction.

(Main process)
FIG. 4 shows a flowchart of the main process, and the main process will be described. When the power of the head mounted display 100 is turned on by the user operating the power button 50, the process proceeds to S11 “initialization”. In the process of S <b> 11, the user can set “display update interval” and “mode” by operating the setting change button 51. The “display update interval” is an interval for updating the “sensing information image” displayed on the image display unit 2. The “mode” includes a “non-transmission mode”, a “transmission mode”, and a “view region entire sensor browsing mode” (transmission mode in a broad sense). As shown in FIG. 20, the “non-transmission mode” is measured by the sensor 500 attached to an area where the head mounted display 100 is located, that is, an area where the user can see (for example, a room where the user is present). In this mode, a “sensing information image” is generated only from “sensing information”. The “transmission mode” is a mode for generating a “sensing information image” from “sensing information” measured by the sensor 500 attached to an area where the head-mounted display 100 is not present, that is, an area where the user cannot see. . When the process of S11 ends, the process proceeds to S12 “Activation of sensing information reception process”.

  In the process of S12 “sensing information reception process activation”, the CPU 41 activates the “sensing information reception process” shown in FIG. This process will be described later in detail with reference to FIG. When the process of S12 ends, the process proceeds to the process of S13 “timer start”.

  In the process of S13 “timer start”, the CPU 41 performs a process of starting a timer for updating the “sensing information image”. When the process of S13 is completed, the process proceeds to the determination process of S15 “display update interval time elapsed?”.

  In the determination process of S15 “display update interval time elapsed?”, The CPU 41 determines whether the timer started in the process of S13 has passed the “display update interval”. When the CPU 41 determines that the “display update interval” has elapsed, the process proceeds to S16 “head mounted display position coordinate calculation processing”. If the CPU 11 determines that the “display update interval” has not elapsed, the process proceeds to the determination process of S21 “transparent mode ON?”.

  In the determination process of S21 “transparent mode ON?”, The CPU 41 determines whether the “transparent mode” is selected by the user pressing the mode switching button 52 or not. When the CPU 41 determines that the “transparent mode” is selected, the process proceeds to the process of S22 “switching to the transparent mode”. On the other hand, when the CPU 41 does not determine that the “transparent mode” is selected, the process proceeds to the determination process of S25 “non-transparent mode ON?”.

  In S22 “switching to transparent mode”, the CPU 41 executes a process of switching to “transparent mode”. When the process of S22 ends, the process returns to the determination process of S15.

  In the determination process of S25 “non-transparent mode ON?”, The CPU 41 determines whether or not the “non-transparent mode” is selected when the user presses the mode switching button 52. If the CPU 41 determines that the “non-transparent mode” has been selected, the process proceeds to S26 “switch to non-transparent mode”. On the other hand, if the CPU 41 does not determine that the “non-transparent mode” has been selected, the CPU 41 proceeds to a determination process of S28 “view area all-sensor browsing mode ON?”.

  In S <b> 26 “switch to non-transparent mode”, the CPU 41 executes a process to switch to “non-transparent mode”. When the process of S26 ends, the process returns to the determination process of S15.

  In the determination process of S <b> 28 “viewing area all-sensor browsing mode ON?”, The CPU 41 determines whether or not the “viewing area all-sensor viewing mode” is selected when the user presses the mode switching button 52. When the CPU 41 determines that the “viewing area all-sensor browsing mode” has been selected, the CPU 41 proceeds to the process of “switching to the viewing area all-sensor viewing mode”. On the other hand, when the CPU 41 does not determine that the “viewing area all-sensor browsing mode” is selected, the process returns to the determination process of S15.

  In S <b> 29 “switching to the viewing area all-sensor browsing mode”, the CPU 41 executes a process of switching to the “viewing area all-sensor viewing mode”. When the process of S29 ends, the process returns to the determination process of S15.

  In S16 “Head Mount Display Position Coordinate Calculation Processing”, which will be described in detail later with reference to FIG. 8, the head mount display coordinate calculation means 43b calculates the “position coordinates” of the head mount display 100. When the process of S16 ends, the process proceeds to S17 “sensor selection process”.

  In S17 “sensor selection process”. As will be described in detail later with reference to FIG. 10, the sensor selection unit 43 f selects the communicable sensor 500 that is in the “visual field region” of the user and in the region obtained by extending the “visual field region” in the visual field direction. When the process of S17 ends, the process proceeds to S18 “sensing information display process”.

  In S18 “sensing information image display processing”, which will be described in detail later with reference to FIG. 14, the sensing information image generating unit 43h generates a “sensing information image”. When the process of S18 ends, S19 “Power OFF?”

  In the determination process of S19 “Power OFF?”, The CPU 41 detects the pressing of the power button 50, and determines whether or not the user has performed an operation to turn off the power of the head mounted display 100. If the CPU 41 detects that the power button 50 has been pressed, the process proceeds to S20 “Cancel all active processes”. When the CPU 41 does not detect pressing of the power button 50, the process returns to S13.

  In the process of S20 “stop all active processes”, the CPU 41 ends all active processes and turns off the power of the head mounted display 100. When the process of S20 ends, the “head mounted display main process” ends.

(Sensing information reception processing)
FIG. 5 is a flowchart of sensing information acquisition processing, and the processing will be described. When the CPU 41 starts the “sensing information reception process” in the process of S12, the following process is executed in parallel with the main process. First, the process proceeds to S31 “data reception from sensor?”. In the determination process of S31, if the CPU 41 determines that data has been received from the sensor 500, the process proceeds to the process of S32 "Save in sensor information table". The data is “individual identification information”, “coordinate information”, “room information”, and “sensing information”. Here, normally, data can be received from the sensor 500 in the room where the user is located by direct wireless communication. Further, as an example, data from a sensor 500 in a room where there is no user is transferred to a sensor 500 near the direction in which the user extends the viewing area in the viewing direction, and this is sequentially repeated to the sensor 500 in the room where the user is present. Thus, data from the sensor 500 in the room where the user is not present can also be received from the sensor 500 in the room where the user is present. When a so-called sensor network is formed and the sensor 500 performs multi-hop communication (relay communication), it is possible to receive all data from the sensor 500 in a room away from the building or in a place where radio waves do not reach directly. Is also possible.

  In the process of S32 “store in sensor information table”, the CPU 41 receives “individual identification information” (sensor ID), “coordinate information” (positional coordinates), “room information” (room), “ The “sensing information” (measured value (%)) is stored in the storage area of the RAM 42 as a “sensor information table” as shown in FIG. In addition, the example of arrangement | positioning of a room is shown in FIG. The position coordinates of the sensor 500 are the position coordinates of the sensor 500 with the predetermined position of the building as the origin. When the process of S32 ends, the process proceeds to S33 “Calculate distance to sensor”.

  In the process of S33 “Calculate the distance to the sensor”, the sensor distance calculation means 43a calculates the head-mounted display from the received radio wave intensity and the radio wave attenuation curve of each sensor 500 that the radio interface 55 can receive directly. The distance between the sensor 100 and each sensor 500 is calculated and stored in the “sensor information table”. When the process of S33 ends, the process returns to the determination process of S31.

(Head-mounted display position coordinate calculation process)
FIG. 8 shows a flowchart of the head-mounted display position coordinate calculation process, which will be described below. When the process of S16 in FIG. 4 starts, in the process of S41 “select four sensors in the closest order” in FIG. 8, the head mounted display coordinate calculation unit 43b uses the “sensor information table” ( 4), the four closest sensors 500 are selected from the head mounted display 100. Since a plurality of sensors 500 are provided in one room, the sensor 500 that can calculate the distance becomes the sensor 500 in the room. When data can be directly received from the sensor 500 in the adjacent room, the sensor 500 is calculated as a sensor 500 at a far distance due to attenuation of a wall or the like, and therefore, it can be considered that the sensor 500 in the room is selected here. When the process of S41 ends, the process proceeds to S42 "Calculation of position coordinates of head mounted display".

  In the processing of S42 “calculation of position coordinates of the head mounted display”, the head mounted display coordinate calculation means 43b has the position coordinates of the sensor 500 as the center for each of the four sensors 500 selected in the processing of S41. The coordinates of the intersection of the four spheres having a radius of the distance (the “distance (m)” shown in FIG. 6) are calculated. The intersection of the four spheres is the “position coordinate” of the head mounted display 100 (coordinate with the predetermined position of the building shown in FIG. 7 as the origin). FIG. 9 shows the position coordinates of the head mounted display 100 calculated in the process of S42. The head mounted display coordinate calculation means 43b compares the “position coordinates” of the head mounted display 100 with the boundary information 44b stored in the nonvolatile memory 44, and in which room of the building the head mounted display 100 is located (user Which room is). When the process of S42 ends, the “head mounted display position coordinate calculation process” ends, and the process proceeds to S17 of the “main process” shown in FIG.

  Note that the process of calculating the “position coordinates” of the head mounted display 100 is not limited to the embodiment of the flow shown in FIG. 8, and uses GPS (head mounted display coordinate acquisition means), wireless LAN, or PHS. The “position coordinates” of the head mounted display 100 may be calculated from the radio wave intensity of the radio wave emitted from the base station by using a base station. Further, the “position coordinates” of the head mounted display 100 may be calculated by the active tag method.

(Sensor selection processing)
FIG. 10 shows a flowchart of the sensor selection process, and the flow will be described below. When the process of S17 in FIG. 4 is started, in the process of S51 “initialization” in FIG. 10, the CPU 41 sets “visual field horizontal angle” dh ° and “visual field vertical angle” dv °. As shown in FIG. 11B, the “field-of-view horizontal angle” dh ° is an angle in the horizontal direction of the user's field of view, and can be arbitrarily set by the user operating the operation button 51. Yes. As shown in FIG. 11C, the “visual field vertical angle” dv ° is an angle in the vertical direction of the user's visual field, and can be arbitrarily set by the user operating the operation button 51. Yes. Of course, it may be set in advance so as to obtain an average human field of view. 11A to 11C, the x, y, and z coordinates are such that the position of the user (the position of the head mounted display) in the building shown in FIG. 7 is the origin. Relative coordinates obtained by translating the coordinate axes that determine the coordinates of the sensor with reference to. When the process of S51 ends, the process proceeds to S52 “Calculate the gaze horizontal angle”.

  In the process of S52 “calculate gaze horizontal angle”, the gaze direction calculation means 43c, based on the signal output from the azimuth sensor 53, as shown in FIG. Calculate h °. The horizontal angle h ° is an angle formed by the line-of-sight direction with respect to the x axis of the relative coordinate system with the position of the user in the building shown in FIG. 7 as the origin. When the process of S52 is completed, the process proceeds to S53 "Calculate gaze vertical angle".

  In the process of S53 “Calculate the gaze vertical angle”, the gaze direction calculation means 43c, based on the signal output from the tilt sensor 54, as shown in FIG. v ° is calculated. The vertical angle v ° is an angle formed by the line-of-sight direction with respect to the xy plane of the relative coordinate system with the position of the user in the building shown in FIG. 7 as the origin. When the process of S53 is completed, the process proceeds to S54 “Is there an unchecked sensor?”

  In the determination process of S54 “Is there an unchecked sensor?”, The CPU 41 determines whether there is an unchecked sensor 500 that has not undergone the processes of S55 to S6 or the determination process. If the CPU 41 determines that there is an unchecked sensor 500, the process proceeds to S55 “sensor relative coordinates”. On the other hand, if the CPU 41 determines that there is no unchecked sensor 500, the “sensor selection process” ends, and the process proceeds to S18 of the “main process” shown in FIG.

  In the process of S55 “sensor relative coordinate calculation”, the sensor relative coordinate calculation unit 43d refers to the “sensor information table” (shown in FIG. 6) stored in the RAM 42 and uses the building of each sensor 500 as a reference. By subtracting the “position coordinates” of the head mounted display 100 from the “position coordinates”, the “position coordinates” of each sensor 500 are converted into “sensor relative coordinates” converted into a relative coordinate system with the head mounted display 100 as the center coordinates. Convert. When the process of S55 ends, the process proceeds to the process of S56 “Calculate sensor angle relative to head mounted display”.

In the process of S56 “calculate the angle of the sensor with respect to the head-mounted display”, the sensor selection unit 43f performs each sensor with respect to the head-mounted display 100 in the coordinate system having the head-mounted display 100 as the center coordinate as shown in FIG. An angle of 500 is calculated. Specifically, by substituting the “sensor relative coordinates” of each sensor calculated in the process of S55 into the following equations (Equation 1 and Equation 2), the horizontal direction angle θ_h ° of each sensor 500 with respect to the head mounted display 100. And the vertical direction angle θ_v ° is calculated.

  Two values are derived for the horizontal direction angle θ_h ° and the vertical direction angle θ_v ° derived from the above equations (Equation 1 and Equation 2), and calculated in the processing of S52 and S53, respectively. The value closer to the horizontal angle h ° and the vertical angle v ° is adopted. When the process of S56 is completed, the process proceeds to S57 “visual field calculation”.

In the processing of S57 “visual field calculation”, the visual field calculation unit 43e calculates the visual field of the user.
Specifically, as shown in FIG. 12A, regarding the horizontal angle, the “view horizontal angle” set in S51 from the horizontal angle h ° in the user's line-of-sight direction calculated in S52. The range of angles obtained by adding and subtracting dh ° is the horizontal field of view (hereinafter referred to as “horizontal field of view”).
Further, as shown in FIG. 12B, regarding the vertical angle, the “view vertical angle” dv ° set in the process of S51 is calculated from the vertical angle v ° of the user's line-of-sight direction calculated in the process of S53. The range of the angle obtained by adding and subtracting is the vertical field of view (hereinafter referred to as “vertical field of view”). When the process of S57 is completed, the process proceeds to the determination process of S58 “A sensor is in the horizontal field of view?”.

In the determination process of S58 “Sensor is in horizontal viewing area?”, Sensor selection unit 43f determines that communication capable sensor 500 is within “horizontal viewing area” and “horizontal viewing area” shown in FIG. Is in the region extending in the viewing direction. Specifically, the determination is made based on whether or not the horizontal angle θ_h ° of the sensor 500 calculated in the process of S56 matches the following expression 3.
(H ° −view horizontal angle dh °) ≦ θ_h ° ≦ (h ° + view horizontal angle dh °) (Equation 3)
If the sensor selection unit 43f determines that the communicable sensor 500 is in the “horizontal field of view” and the “horizontal field of view” extended in the direction of field of view, S59 “the sensor is in the vertical field of view”. The process proceeds to the determination process. On the other hand, if the sensor selection unit 43f determines that the communicable sensor 500 is not in the “horizontal field of view” and the region extending the “horizontal field of view” in the field of view direction, the process returns to the determination process in S54. .

In the determination process of S59 “A sensor is in the vertical viewing area?”, The sensor selecting unit 43f determines that the communicable sensor 500 is within the “vertical viewing area” and “vertical viewing area” shown in FIG. Is in the region extending in the viewing direction. Specifically, determination is made based on whether or not the vertical angle θ_v ° of the sensor 500 calculated in the process of S56 matches the following expression 4.
(V ° −view vertical angle dv °) ≦ θ_v ° ≦ (v ° + view vertical angle dv °) (Formula 4)
If the sensor selection unit 43f determines that the communicable sensor 500 is in the “vertical field of view” and the region extending the “vertical field of view” in the direction of field of view, S60 “registers the sensor in the display list”. Proceed to the process. On the other hand, when the sensor selection unit 43f determines that the communicable sensor 500 is not in the “vertical field of view” and the region extending the “vertical field of view” in the field of view direction, the process returns to the determination process in S54. .

  In the process of S60 “register sensor in display list”, the sensor 500 determined to be in the user's field of view in the processes of S58 and S59 is registered in the “display list” as shown in FIG. When the S60 process ends, the process returns to the determination process of S54.

(Sensing information image display processing)
FIG. 14 shows a flowchart of the sensing information image display process, and the process will be described below. When the process of S18 in FIG. 4 starts, the CPU 41 determines whether or not the state of the head mounted display 100 is “transparent mode” in the determination process of S71 “transparent mode?”. If the CPU 41 determines that the state of the head mounted display 100 is “transparent mode”, the process proceeds to S72 “Delete sensor in area that user can see from display list”. On the other hand, when the CPU 41 determines that the state of the head mounted display 100 is not “transparent mode”, the CPU 41 proceeds to a determination process of S75 “non-transparent mode?”.

  FIG. 13 shows the sensor 500 in the area (the room where the user is present) that the user can see in the process of S72 “Delete the sensor in the area that the user can see from the display list”. Delete the “display list”. In FIG. 13, it may be determined whether the room information of each sensor matches the room where the head mounted display 100 exists. When the process of S72 ends, the process proceeds to the process of S73 “Calculate image coordinates on the image to be displayed on the head mounted display of the sensor”.

In the process of S73 “calculate image coordinates on the image displayed on the head-mounted display of the sensor”, the image coordinate calculation means 43g is based on the “sensor relative coordinates” of each sensor 500 calculated by the sensor relative coordinate calculation means 43d. The “image coordinates” which are coordinates on the image displayed on the image display unit 2 of each sensor 500 selected by the sensor selection unit 43 f are calculated. That is, a process of calculating coordinates for displaying the sensor 500 arranged in three dimensions on a two-dimensional image generated by the image generation apparatus 2 is performed. Specifically, FIG. 15 and FIG. 16 will be described with reference to explanatory diagrams. FIG. 15 is a diagram showing the horizontal coordinate of the coordinate system with the head mounted display 100 as the center coordinate.
(1) A straight line in the user's line-of-sight direction is taken as the w-axis.
The axis orthogonal to the w axis is the p axis.
(2) Select an arbitrary sensor 500 (sensor A in the example of FIG. 15) within the visual field range, and draw a straight line v that passes through the sensor A and is orthogonal to the w axis. The straight line v can be expressed as wa = d_a × cos (h ° −θ_h °) = √ (xa ² + ya ² ) × cos (h ° −θ_h °) (Formula 5).
(3) The straight line in the maximum right visual field direction and the straight line in the maximum left visual field direction are both horizontal to the w-axis (shown in FIG. 12), so the right visual field horizontal angle is dh1 ° and the left visual field horizontal When the angle is dh2 °, it can be expressed as P = a ₁ w (formula 6) P = a ₂ w (formula 7). Here, a1 = tan (−dh1 °), a2 = tan (dh2 °)
(4) From Equations 5 and 6, the coordinates (pm, wm) of the intersection m between the straight line V and the straight line in the right-side maximum visual field direction are calculated.
Further, the coordinates (pn, wn) of the intersection point n between the straight line V and the straight line in the maximum left visual field direction are calculated from the formulas 5 and 7.
From the coordinates (pm, wm) of the intersection point m and the coordinates (pn, wn) of the intersection point n, the horizontal width of the temporary screen is calculated as shown in Equation 8 below.
| Pm−pn | (Formula 8)
(5) Further, the distance p_a of the sensor A from the w-axis (the center of the virtual screen) can be obtained by the following formula 9.
p_a = d_a × sin (h ° −θ_h °) = √ (xa ² + ya ² ) × sin (h ° −θ_h °) (Equation 9)
(6) For sensor B, the coordinates of the intersection of both straight lines can be calculated from the formula of the straight line passing through the head mounted display origin and sensor B and the formula of straight line v (xb ′, yb ′) The distance p_b of the sensor B from the w axis (the center of the virtual screen) is calculated by the same method as the above (1) to (5). For the other sensors 500, the position in the horizontal width direction of each sensor 500 on the temporary screen is calculated in the same manner.
(7) As shown in FIG. 16, the position in the vertical width direction of each sensor 500 on the temporary screen is calculated by the same method.
(8) As shown in FIG. 17, by reducing the size of the screen generated by the image display unit 2 while maintaining the positional relationship of the sensors on the temporary screen, the image on the image displayed by the image generation device 2 is displayed. The “image coordinates” of each sensor 500 are calculated.
When the process of S73 ends, the process proceeds to the process of S74 “display sensing information image and boundary line on head mounted display”.

  In the process of S74 “display the sensing information image and the boundary line on the head mounted display”, the sensing information image generating unit 43h calculates the “image coordinates” of each sensor 500 calculated in the process of S73 and as shown in FIG. Based on the “display pattern table”, a “sensing information image” is generated, and the “sensing information image” is output to the image display unit 2 and displayed. Here, the sensor value is given to each image coordinate of the sensor, but an image showing a surface as shown in FIG. 19 is generated by well-known contour line processing and interpolation processing. From such a viewpoint, the visual field horizontal angle dh ° and the visual field vertical angle dv ° may be actually set slightly wider than the user's field of view, and the diopter of the contour line processing and the interpolation processing may be increased. Further, the boundary information synthesizing unit 43i includes the user's line-of-sight direction measured by the line-of-sight direction calculating unit 43c, the boundary information 44b stored in the nonvolatile memory 44, and the coordinates of the head-mounted display acquired by the head-mounted display coordinate calculating unit 43b. Based on the above, the room boundary line 710 is combined with the “sensing information image”. In addition, the said room means the area | region where the building was divided, and a corridor, a lobby, etc. are also regarded as a room in this invention. FIG. 19 shows the user's field of view in the “transmission mode” state. In this embodiment, since it is “transmission mode”, the sensing information image 700 is displayed on the lower side of the floor surface. Further, a boundary line 710 of an area that cannot be visually recognized by the user is displayed. In the embodiment shown in FIGS. 18 and 19, the sensing information image 700 represents a region where a high concentration of carbon monoxide is retained. Thus, if the head-mounted display 100 of the present invention is used, the boundary line 710 of the region that the user cannot visually recognize can be viewed together with the “sensing information” of the region that cannot be viewed by the user. There is a sense of realism. It is preferable to change the display of the boundary line 710 for each different room and synthesize these boundary lines 710 with the sensing information image. In this way, the user can distinguish and recognize the sensing information image 700 of the room in the back and the room in the back of the room. When the process of S74 ends, the “sensing information image display process” ends, and the process proceeds to the determination process of S19 of the “main process” shown in FIG.

  In the determination process of S75 “non-transparent mode?”, The CPU 41 determines whether or not the state of the head mounted display 100 is “non-transparent mode”. If the CPU 41 determines that the state of the head mounted display 100 is the “non-transparent mode”, the process proceeds to S76 “Delete sensor other than the area that the user can see from the display list”. On the other hand, if the CPU 41 determines that the state of the head-mounted display 100 is not the “non-transparent mode” but the “viewing area all-sensor browsing mode”, the image coordinates on the image displayed on the head-mounted display of the sensor are S79. The process proceeds to “Calculate”.

  In the process of S76 “deleting a sensor other than the area that the user can see from the display list”, the sensor selection unit 43f selects the sensor 500 other than the area that the user can see (other than the room where the user is currently present) A process of deleting from the “display list” shown in FIG. 13 is performed. When the process of S76 ends, the process proceeds to the process of S77 “calculate image coordinates on the image displayed on the head mounted display of the sensor”.

  In the process of S77 “calculate the image coordinates on the image displayed on the head-mounted display of the sensor”, the image coordinate calculation unit 43g is similar to the process of S73, and “the sensor relative coordinate calculation unit 43d calculates“ Based on “sensor relative coordinates”, “image coordinates”, which are coordinates on the image displayed on the image display unit 2 of each sensor 500 selected by the sensor selection unit 43f, are calculated. When the process of S77 is completed, the process proceeds to S78 “Display Sensing Information Image on Head Mount Display”.

  In the process of S78 “display the sensing information image on the head mounted display”, the sensing information image generating unit 43h calculates the “image coordinates” of each sensor 500 calculated in the process of S77 and the “display” as shown in FIG. Based on the “pattern table”, a “sensing information image” is generated, and the “sensing information image” is output to the image display unit 2 and displayed. In the process of S78, since the head mounted display 100 is in the “non-transparent mode”, “sensing information” in an area that can be viewed by the user is displayed on the image display unit 2 as shown in FIG. When the process of S78 ends, the “sensing information image display process” ends, and the process proceeds to the determination process of S19 of the “main process” shown in FIG.

  In the process of S79 “Calculate the image coordinates on the image displayed on the head-mounted display of the sensor”, the image coordinate calculation means 43g is the same as the process of S73 and the process of S77. Based on the “sensor relative coordinates” of the sensor 500, “image coordinates” that are coordinates on the images displayed on the image display units 2 of all the sensors 500 in the “display list” selected by the sensor selection unit 43 f are calculated. When the process of S79 ends, the process proceeds to the process of S80 “display sensing information image on head mounted display”.

  In the process of S80 “display the sensing information image on the head mounted display”, the sensing information image generating unit 43h calculates the “image coordinates” of each sensor 500 calculated in the process of S79 and the “display” as shown in FIG. Based on the “pattern table”, a “sensing information image” is generated, and the “sensing information image” is output to the image display unit 2 and displayed. In the processing of S80, since the head mounted display 100 is in the “viewing area all-sensor browsing mode”, “sensing information” of the area that the user can see and the area that the user cannot see is displayed on the image display unit 2. Is done. When the process of S80 ends, the “sensing information image display process” ends, and the process proceeds to the determination process of S19 of the “main process” shown in FIG.

  In the embodiment described above, the control unit 40 is attached to the image display unit 2, but the control unit 40 may be separated from the image display unit 2.

  The head mounting portion 1 shown in FIG. 1 has a frame shape of eyeglasses. The head mounting portion 1 is not limited to this shape, and may be a helmet shape or the like, and includes any structure that is mounted on the user's head. An image display unit 2 is attached to the front side of the head mounting unit 1. In the head mounted display 100 shown in FIG. 1, one image display unit 2 is attached to the front side of the head mounting unit 1, but the two image display units 2 are the head mounting unit 1. A head-mounted display that is attached to the front part on both sides and allows the user to visually recognize an image with both eyes may be used. Further, when sensing information is received, it may be determined whether the user is in the field of view and data from a sensor located in the field of view may be received. Thereby, the processing load can be reduced.

  Although the present invention has been described above in connection with the most practical and preferred embodiments at the present time, the present invention is not limited to the embodiments disclosed herein. However, the present invention can be appropriately changed without departing from the gist or concept of the invention that can be read from the claims and the entire specification, and a head-mounted display accompanying such a change should also be understood as being included in the technical scope. I must.

It is an external view of the head mounted display of this invention. It is a figure showing the field of view of the user wearing a head mounted display. It is a block diagram of a head mounted display. It is a flowchart of a main process. It is a flowchart of a sensing information acquisition process. It is explanatory drawing of a sensor information table. It is explanatory drawing of room arrangement | positioning. It is a flowchart of a head mounted display position coordinate calculation process. It is a figure showing the position coordinate of a head mounted display. It is a flowchart of a sensor selection process. It is explanatory drawing of a gaze horizontal angle and a gaze vertical angle. It is explanatory drawing which calculates | requires a visual field area | region and a sensor angle. It is explanatory drawing of a display list. It is a flowchart of a sensing information image display process. It is the figure which showed the horizontal coordinate of the coordinate system which makes a head mounted display a center coordinate. It is the figure which showed the vertical coordinate of the coordinate system centering on a head mounted display. It is explanatory drawing which reduces the size of the screen which an image display part produces | generates, maintaining the positional relationship of the sensor on a temporary screen, and calculates the coordinate of each sensor on the image which an image generation apparatus produces | generates. It is explanatory drawing of a display pattern table. It is a figure showing the field of view of the user wearing a head mounted display. (Transparent mode and boundary information display) It is explanatory drawing of a non-transmissive mode and a transmissive mode.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Head mounting part 2 Image display part 20 Scanning image light irradiation part 21 Scanning light production | generation part 22 Collimating optical system 23 Vertical scanning part 23a Resonance-type deflection | deviation element 23b Vertical scanning control circuit 24 Horizontal scanning part 24a Deflection element 24b Horizontal scanning control circuit 25 relay optical system 26 relay optical system 26a lens system 26b lens system 27 optical fiber 40 control unit 41 CPU
42 RAM
43 ROM
43a sensor distance calculation means 43b head mounted display coordinate calculation means 43c gaze direction calculation means 43d sensor relative coordinate calculation means 43e visual field area calculation means 43f sensor selection means 43g image coordinate calculation means 43h sensing information image generation means 43i boundary information synthesis means 44 nonvolatile Memory 44a Display pattern table 44b Boundary information 45 Input interface 46 Bus 50 Power button 51 Setting change button 52 Mode switch button 53 Surrounding sensor 54 Tilt sensor 55 Wireless interface 100 Head mounted display 211 Signal processing circuit 212 Light source 212a B laser driver 212b G laser driver 212c R laser driver 212d B laser 212e G laser 212f R laser 213 Photosynthesis unit 13a to 213c Collimating optical system 213d to 213f Dichroic mirror 213g Coupling optical system 214 Image signal 215 Vertical driving signal 216 Horizontal driving signal 500 Sensor 600 Region where high concentration gas stays 700 Sensing information image (high concentration carbon monoxide Area where the
710 Boundary line E Eyeball Ea Pupil

Claims (5)

A head mounting part to be mounted on the user's head;
In a head-mounted display that is attached to the front part of the head-mounted unit, displays an image, and has an image display unit that allows the user to visually recognize the image.
Sensing information measured by sensors installed at a plurality of positions, individual identification information of each sensor, information acquisition means for acquiring coordinate information of each sensor,
Head-mounted display coordinate acquisition means for calculating or acquiring the coordinates of the head-mounted display;
Gaze direction measuring means for measuring the gaze direction of the user;
Based on the coordinate information of each sensor acquired by the information acquisition unit and the coordinates of the head mounted display acquired by the head mounted display coordinate acquisition unit, the relative coordinates of the sensors based on the coordinates of the head mounted display are obtained. Sensor relative coordinate calculating means for calculating;
A visual field area calculating means for calculating a visual field area of the user from the visual line direction information of the user measured by the visual line direction measuring means;
Based on the field of view calculated by the field of view calculation unit and the sensor relative coordinates of each sensor calculated by the sensor relative coordinate calculation unit, communication is possible within the field of view and within a region extending the field of view in the field of view direction. Sensor selection means for selecting a sensor,
Sensing information image generating means for generating a sensing information image to be displayed on the image display unit based on sensor relative coordinates of each sensor selected by the sensor selecting means and sensing information of each sensor acquired by the information acquiring means;
A head-mounted display comprising: Image coordinate calculation means for calculating image coordinates, which are coordinates on an image displayed on the image display unit of each sensor selected by the sensor selection means, based on the sensor relative coordinates calculated by the sensor relative coordinate calculation means. Have
The sensing information image generation unit generates a sensing information image to be displayed on the image display unit based on the image coordinates of each sensor calculated by the image coordinate calculation unit and the sensing information of each sensor acquired by the information acquisition unit. The head mounted display according to claim 1. The information acquisition means acquires sensing information measured by a plurality of sensors arranged in a room and coordinate information of each sensor,
A boundary information storage unit storing boundary information of the room;
Boundary line synthesizing means for synthesizing the room boundary line with the sensing information image based on the user's gaze direction measured by the gaze direction measuring means, the boundary information, and the coordinates of the head mounted display acquired by the head mounted display coordinate acquiring means. When,
The head mounted display according to claim 1, wherein the head mounted display is provided. The boundary line synthesis means includes:
4. The head mounted display according to claim 3, wherein the display of the boundary lines is changed for each different room, and the boundary lines are combined with the sensing information image. Sensor distance calculation means for calculating distance information for each sensor head-mounted display,
The head mounted display coordinate acquisition means calculates the coordinates of the head mounted display based on the distance information calculated by the sensor distance calculation means and the individual identification information and coordinate information of each sensor acquired by the information acquisition means. The head mounted display according to any one of claims 1 to 4.

Images