Open AccessArticle

Perceived Brightness and Resolution of Holographic Augmented Reality Retinal Scan Glasses

Institute for Ophthalmic Research, University of Tübingen, 72076 Tübingen, Germany

Bosch Sensortec GmbH, 72770 Reutlingen, Germany

Institute for Optics and Mechatronics, University of Applied Sciences Aalen, 73430 Aalen, Germany

⁴

University Eye Hospital Tübingen, 72076 Tübingen, Germany

Author to whom correspondence should be addressed.

Appl. Sci. 2025, 15(3), 1540; https://doi.org/10.3390/app15031540

Submission received: 12 January 2025 / Revised: 28 January 2025 / Accepted: 30 January 2025 / Published: 3 February 2025

(This article belongs to the Special Issue Virtual and Augmented Reality: Theory, Methods, and Applications)

Download

Browse Figures

Versions Notes

Abstract

Featured Application

This work can be applied to the display calibration and characterization of augmented reality glasses with a retinal scan architecture.

Abstract

Augmented reality display performance depends strongly on features of the human visual system. This is especially true for retinal scan glasses, which use laser beam scanning and transparent holographic optical combiners. Human-centered approaches allow us to go beyond conventional optical metrology and evaluate display performance as it is perceived in actual augmented reality use cases. Here, we first present a theoretical formula for the retinal scan luminance and ambient contrast ratio calculated from optical powers, wavelengths, field of view, and human pupil diameter. As a promising insight, we found that the pupil diameter dependence is beneficial in assimilating the virtual image luminance to the ambient luminance. Second, we designed and performed a psychophysical experiment to assess perceived resolution in augmented reality settings using a fully functional retinal scan glasses prototype. We present the results of the trials and illustrate how this approach can be used in the further development of augmented reality smart glasses.

Keywords:

augmented reality; smart glasses; photometry; display metrology; perception; psychophysics; human–machine interaction

1. Introduction

Retinal scan (“RS”) glasses are near-eye displays that use laser beam scanning for retinal image projection. A collimated light beam is projected via an optical combiner and through the human pupil onto the retina [1]. Recent advances in holographic manufacturing [2] have enabled the use of holographic optical elements (“HOEs”) as transparent, diffractive optical combiners [3]. This allows for lightweight augmented reality displays to be integrated into all-day wearable smart glasses with a small formfactor.

Research and development in the field is progressing quickly [4]. First efforts have been made to define new parameters like the ambient contrast ratio (“ACR”) [5] that are necessary to characterize the performance of holographic augmented reality displays [6] in real visual environments. We elaborate on these efforts and show that for the special case of RS displays, interactions with the human eye must be considered.

Most conventional display technologies such as LCD, LED, or waveguides emit light from each pixel into a large solid angle. Their emission profiles roughly obey Lambert’s cosine law to keep luminance constant across different viewing angles. We use the term “Lambertian” to describe such displays. From Lambertian displays, only a small portion of the emitted light enters the human eye at any given time. In contrast, all light emitted by RS displays is confined to a narrow scanning beam which passes through the pupil. This makes RS displays not only energy efficient but also leads to important consequences for fundamental display parameters like the brightness and resolution.

One challenge is that some conventional measurement devices such as luminance cameras cannot be used directly with RS glasses due to the interaction of the projection system with the human pupil. Instead, the luminance of an RS display must be computed in a certain way to achieve accurate results. We introduce a theoretically derived formula to calculate the retinal scan luminance and ambient contrast ratio. This formula incorporates various parameters including optical powers, wavelengths, field of view, and human pupil diameter. Building on this formula, we designed and conducted a psychophysical study to measure the perceived resolution of RS glasses in augmented reality settings.

While contrast and resolution perception have been investigated in many previous studies as fundamental parameters of human visual perception [7], augmented reality glasses provide new opportunities to take the measurements of these parameters outside of controlled laboratory environments. This work presents novel theory and methods for such out-of-laboratory measurements.

2. Materials and Methods

2.1. Retinal Scan Glasses

The BML500P RS glasses prototype available for this study (Figure 1) was developed by Bosch Sensortec and presented at the Consumer Electronics Show 2020 [8].

A light engine in the right temple of the glasses projects images via a transparent holographic optical element through the pupil onto the retina. Two oscillating micromirrors steer light from three low-power RGB laser diodes in a raster scanning pattern to form an image at a refresh rate of 60 Hz.

2.1.1. Brightness in Retinal Raster Scanning

The human eye detects retinal illuminance as brightness. Retinal illuminance

E_{R}

is defined as luminous flux

Φ_{V}

per retinal area

A_{R}

E_{R} = \frac{Φ_{V}}{A_{R}}

(1)

Display brightness is usually denoted in terms of luminance L, which is defined as the luminous flux

Φ_{V}

emitted into a solid angle

Ω

by a light source with area

A_{S}

L = \frac{Φ_{V}}{{Ω A}_{S}}

(2)

Luminance is a measure for the light source, while the perceived brightness depends on the illuminance on the retina. When comparing Lambertian displays with each other, luminance can be used as a proxy for perceived brightness because luminance changes lead to proportional retinal illuminance changes.

This is, however, not valid for a comparison between Lambertian and RS displays. The size of the human pupil influences the luminous flux

Φ_{V}

from a Lambertian display but not from an RS display, as shown in Figure 2.

For this reason, measuring the luminance of an RS image with a luminance camera leads to inconsistent results: the larger the entrance pupil of the luminance camera (to which it is calibrated using a Lambertian light source), the smaller the measured RS luminance.

In the first part of the Results Section, we show that it is nevertheless possible to calculate an equivalent luminance for RS displays. This luminance depends on the RS luminous flux, RS field of view, and human pupil size and is directly comparable with the luminance of Lambertian displays.

2.1.2. Focus and Resolution in Retinal Raster Scanning

Figure 3 shows the impact that ocular imperfections have on Lambertian and RS images. The circle of confusion for individual pixels is smaller for RS images due to the strong collimation and long depth of field of the scanning light beam.

While this effect can be illustrated clearly, it is difficult to assess the differential impact on display performance with methods from conventional display metrology since the human eye needs to be brought “into the loop”.

2.2. Perceived Resolution Study Design

To measure the perceived resolution of the BML500P RS glasses prototype, we conducted a psychophysical discrimination test with 20 participants (17 males, 3 females; aged 18–60 years, mean ± sd: 36.0 ± 7.7 years). We implemented a prescreening routine for visual acuity and contrast sensitivity with Landolt rings and a test for color vision with color plates to ensure a minimum standard level of vision ability among all participants. Participants who could not pass these screening tests were not included in the study. Inclusion criteria were right-eye dominancy, a right monocular visual acuity of at least 0.8 decimal (screening results: logMAR mean ± sd: −0.073 ± 0.060), a right monocular contrast vision threshold below 15% Weber contrast (mean ± sd: 4.2 ± 0.8%), no color vision deficiency, and no known eye diseases. The experiment was approved by the ethics committee of the Medical Faculty of the University of Tübingen (Code 410/2022B02) and conducted according to the provisions of the Declaration of Helsinki. Informed consent was obtained from all participants.

The study setup is shown in Figure 4. Participants were equipped with the RS glasses on a chinrest 2.3 m in front of a 60-inch LED monitor (PN-R603; Sharp Corporation; Sakai, Osaka, Japan). The light-isolated examination room was illuminated at an average of 2500 lux, equal to outdoor illumination on a cloudy day.

Participants were shown a right-eye monocular stimulus in the glasses with a size of 11.15 ± 0.22 deg × 14.99 ± 0.17 deg [mean ± sd from 10 measurements]. The transparent RS glasses superimposed this virtual image onto the real image from the LED monitor. Psychopy [9] was used to control the monitor and RS glasses stimuli simultaneously. To minimize distractions, participants wore a black eye patch over their left eye.

The glasses stimulus was picked randomly from a selection of monochromatic line patterns with different spatial frequencies from 1.8 cyc/deg to 16.0 cyc/deg and a full monochromatic rectangle. The line pattern consisted of lines with equal digital pixel width for both projection and non-illuminated dark background lines. The spatial frequency range of the lines was determined by the 480 × 320 pixels that could be set within the software configuration of the prototype. The pattern with the highest spatial frequency had a width of one digital pixel.

For high digital spatial frequencies, the resulting optical lines started to blur into a full rectangle. The task of the participants was to report whether they saw lines or a rectangle using a wireless keyboard. The spatial frequency of the lines was adjusted based on the participants’ responses using a 3-down 1-up staircase algorithm to determine the highest resolvable spatial frequency. The algorithm terminated after three reversals. There was no time limit for responses. To ensure that a staircase level could not be passed by only identifying rectangles, a line pattern was always presented after two correct responses. A 0.5 s pause between stimuli was added to avoid afterimages and to reduce cues from stimulus transitions.

We performed 15 trials each for horizontal and vertical lines for different stimulus colors (laser wavelengths red [640 nm], green [521 nm], blue [452 nm]), stimulus brightness (maximum radiant flux red [272 nW]), green [736 nW], blue [1263 nW], equiluminant radiant flux green [66 nW], blue [1145 nW]), and LED monitor backgrounds [black, red, green, blue, orange] to measure the influence of different stimulus and background conditions on the perceived resolution.

The LED monitor was set to a luminance of 27 cd/m² for each color and 1 cd/m² for the black background. The right spectacle lens with the holographic optical combiner had an optical transmittance of

τ_{C}

= 0.90.

The optical power of the retinal scan glasses was measured by setting an optical power meter (PM160 power meter; Thorlabs, Inc.; Newton, NJ, USA) in the beam path such that all light contributing to the image was captured by the optical sensor. The wavelengths of the retinal scan glasses as well as the luminance of the LED monitor were measured with a spectroradiometer (JETI spectraval 1511; JETI Technische Instrumente GmbH; Jena, Germany). The wavelength measurement only depends on the relative magnitudes of optical power. Therefore, measuring the wavelengths of retinal scan glasses with a spectroradiometer will provide accurate results, while measuring the luminance directly will not.

Generally, the perceived resolution threshold not only depends on the spatial frequency but also on the contrast [10]. If the contrast between stimuli and background is too small, the perceived resolution threshold decreases across spatial frequencies [11]. Previous research showed that this influence is persistent up to contrast levels of about 0.5 [12]. Since we wanted to avoid the influence of contrast on the perceived resolution in this study, we needed to ensure that the contrast between the retinal scan stimuli and the Lambertian background stayed above 0.5. As shown in Section 2.1.1, however, this contrast can only be calculated correctly if the brightness for both display types is denoted in the same physical unit. We therefore first derive a formula for this calculation in the first part of the Results Section before presenting the results of the study in the second part of the section.

3. Results

3.1. Retinal Scan Luminance and Ambient Contrast Ratio

To express the brightness of an RS display in terms of luminance, we determine the luminance of a Lambertian display that would cause the same retinal illuminance

E_{R}

as the RS glasses (see Table A1 in Appendix A for an overview of mathematical symbols).

E_{R_{Lam}} = E_{R_{Scan}}

(3)

To find the RS retinal illuminance

E_{R_{Scan}}

, we first calculate the luminous flux

Φ_{V_{Scan}}

from the radiant flux

Φ_{E_{Scan}}

, the luminous efficacy constant

K_{m} = 683 \frac{l m}{W}

, and the luminous efficiency function

V (λ)

. The luminous efficacy constant

K_{m}

tells us how many lumens result from 1 watt of optical power. It acts as the conversion factor from radiometric to photometric quantities. The luminous efficiency function

V (λ)

denotes how bright light with different wavelengths appears to the human eye. In practice, the resulting integral is approximated by a finite sum of measurements.

Φ_{V_{Scan}} = K_{m} \int V (λ) Φ_{E_{Scan}} (λ) d λ \approx K_{m} \sum_{λ} V (λ) Φ_{E_{Scan}} (λ)

(4)

Next, we calculate the illuminated retinal area

A_{R_{Scan}}

geometrically from the RS field of view angles

α

and

β

and the eye’s focal distance

f_{e}

A_{R_{Scan}} = f_{e} 2 \tan (\frac{α}{2}) f_{e} 2 \tan (\frac{β}{2}) = 4 {f_{e}}^{2} \tan (\frac{α}{2}) \tan (\frac{β}{2})

(5)

Plugging Equations (4) and (5) into Equation (1) gives us the retinal illuminance for an RS display.

E_{R_{Scan}} = \frac{Φ_{V_{Scan}}}{A_{R_{Scan}}} = \frac{K_{m} \sum_{λ} V (λ) Φ_{E_{Scan}} (λ)}{4 {f_{e}}^{2} \tan (\frac{α}{2}) \tan (\frac{β}{2})}

(6)

Next, we express the Lambertian display retinal illuminance

E_{R_{Lam}}

in terms of the Lambertian display luminance

L

[13]. For an observer with pupil area

A_{P}

at distance

R

to the display, the solid angle

Ω

will be given by the ratio of

A_{P}

and

R^{2}

Ω = \frac{A_{P}}{R^{2}}

(7)

Plugging Equation (7) into Equation (2) and rearranging terms yields the luminous flux

Φ_{V_{Lam}}

entering the pupil as a function of the Lambertian display luminance

L_{Lam}

, pupil area

A_{P}

, display area

A_{S}

, and display distance

R

L_{Lam} = \frac{Φ_{V_{Lam}}}{\frac{A_{P}}{R^{2}} A_{S}} \Leftrightarrow Φ_{V_{Lam}} = \frac{L_{Lam} A_{P} A_{S}}{R^{2}}

(8)

To find the retinal image area

A_{R_{Lam}}

, we use the lens formula from geometrical optics. A display with area A_S at a distance R to the eye creates an image with area A_R at focal distance

f_{e}

on the retina. The magnification between

A_{S}

and

A_{R}

is given by the square of the ratio between image distance

f_{e}

and object distance

R

A_{R_{Lam}} = A_{S} {(\frac{f_{e}}{R})}^{2}

(9)

Plugging Equations (8) and (9) into Equation (1) gives us the retinal illuminance for a Lambertian display:

E_{R_{Lam}} = \frac{Φ_{V_{Lam}}}{A_{R_{Lam}}} = \frac{L_{Lam} A_{P} A_{S}}{R^{2} A_{S} {(\frac{f_{e}}{R})}^{2}} = \frac{L_{Lam} A_{P}}{{f_{e}}^{2}}

(10)

Finally, we can set both retinal illuminances

E_{R_{Lam}}

and

E_{R_{Scan}}

as equal.

\frac{L_{Lam} A_{P}}{{f_{e}}^{2}} = \frac{K_{m} \sum_{λ} V (λ) Φ_{E_{Scan}} (λ)}{4 {f_{e}}^{2} \tan (\frac{α}{2}) \tan (\frac{β}{2})}

(11)

After rearranging terms and renaming

L_{Lam}

L_{RS}

, we arrive at Equation (12) for the luminance

L_{RS}

of an RS display.

L_{RS} = \frac{K_{m} \sum_{λ} V (λ) Φ_{E_{Scan}} (λ)}{4 A_{P} \tan (\frac{α}{2}) \tan (\frac{β}{2})}

(12)

To calculate the pupil area

A_{P}

, we measure the pupil diameter

d_{p}

in mm.

A_{P} = \frac{1}{4} π {(0.001 d_{P_{mm}})}^{2}

(13)

For an RS display with narrow RGB laser spectral linewidths, we approximate

\sum_{λ} V (λ) Φ_{E_{Scan}} (λ)

in Equation (12) with radiant powers

Φ_{i}

and average wavelengths

{\bar{λ}}_{i}

L_{RS} = \frac{K_{m} (V ({\bar{λ}}_{red}) Φ_{red} + V ({\bar{λ}}_{green}) Φ_{green} + V ({\bar{λ}}_{blue}) Φ_{blue})}{π {(0.001 d_{P_{mm}})}^{2} \tan (\frac{α}{2}) \tan (\frac{β}{2})}

(14)

K_{m}

and

V (λ)

are constant and tabularized [14].

Φ_{i}

α

, and

β

must be measured for the RS display. The only remaining variable is the pupil diameter

d_{P_{mm}} .

Changes in pupil diameter can be caused by different factors [15]. In most circumstances, however, the main contributor is the ambient luminance

L_{A}

. We use this fact to express the RS luminance as a function of ambient luminance instead of pupil size.

Several mathematical models have been developed for this relationship [16]. We picked the model from Stanley and Davies [17] for binocular background adaptation with an adaptive field size a of (60 deg)². The model allows us to express pupil diameter

d_{P}

in mm as a function of ambient luminance

L_{A}

in cd/m² for an average observer.

d_{P_{mm}} = 7.75 - 5.75 (\frac{{(\frac{L_{A} 3600}{846})}^{0.41}}{{(\frac{L_{A} 3600}{846})}^{0.41} + 2})

(15)

Plugging Equation (15) into Equation (14) allows us to calculate the RS luminance

L_{RS}

as a function of the ambient luminance

L_{A}

L_{RS} = \frac{K_{m} (V ({\bar{λ}}_{red}) Φ_{red} + V ({\bar{λ}}_{green}) Φ_{green} + V ({\bar{λ}}_{blue}) Φ_{blue})}{π {(0.001 (7.75 - 5.75 (\frac{{(\frac{L_{A} 3600}{846})}^{0.41}}{{(\frac{L_{A} 3600}{846})}^{0.41} + 2})))}^{2} \tan (\frac{α}{2}) \tan (\frac{β}{2})}

(16)

Expressing the RS luminance in terms of the ambient luminance not only eliminates the need for direct pupil size measurements but also simplifies another useful quantity, the “retinal scan ambient contrast ratio”

{ACR}_{RS}

, which is defined by the RS luminance

L_{RS}

, the combiner transparency

τ_{C}

, and the ambient luminance

L_{A}

{ACR}_{RS} = \frac{L_{RS}}{τ_{C} L_{A}}

(17)

In non-transparent displays, the contrast ratio is determined from the brightest and darkest pixel’s luminance. RS displays do not project any light into fully dark pixels. The luminance of the darkest RS display pixel is therefore given by the luminance of the darkest background element within the field of view. This could be measured for a fixed head orientation and homogeneous background in a controlled laboratory setting. For most natural visual scenes, however, such a pixel-precise background luminance would be difficult to determine. For those cases, we use the average ambient luminance

L_{A}

as an estimate.

Since the ambient light passes through the spectacle lens, its luminance is reduced by the transmittance

τ_{C}

of the HOE combiner. Optical power measurements of the RS display are taken at the eye position, so

τ_{C}

does not influence the numerator in Equation (17). The transmittance could however influence the pupil diameter in Equation (15). This will depend on the size of the combiner in comparison to the adaptive field size. Since the HOE region in our glasses prototype was small, and, more importantly, monocular, we neglected this influence. If this does not hold true for other prototypes,

τ_{C}

can be included as an additional coefficient to

L_{A}

in Equation (15).

Figure 5 shows the RS luminance

L_{RS}

and the RS ambient contrast ratio

{ACR}_{RS}

as a function of the ambient luminance

L_{A}

for a fixed set of RS glasses parameters.

3.2. Perceived Resolution Study Results

This study was conducted in a controlled environment which allowed us to determine the pupil size directly. We measured the pupil diameter of all participants right before the experiment while they faced the monitor on the chinrest and found a mean ± sd pupil diameter of 3.3 ± 0.4 mm within a range from 2.5 mm to 4.0 mm. With the optical measurement values from the glasses, we determined

L_{RS}

according to Equation (14). We then used the luminance of the LED monitor backgrounds for

L_{A}

and the transmittance of the combiner

τ_{C}

to calculate

{ACR}_{RS}

according to Equation (17).

Table 1 provides the trial parameters and results from the perceived resolution study. The left side of the table shows the trial parameters including the ambient contrast ratio ACR_RS for the retinal scan stimuli and the LED monitor background. Since ACR_RS stayed well above 0.5 for all trials, we can now assume that the perceived resolution threshold is only a function of the spatial frequency and not the contrast. The right side of the table shows the number of participants that successfully passed each staircase level. “Passing” a staircase level means that the participant provided three correct answers in a row at that level during the staircase algorithm. The level number corresponds to the digital pixel width of the line patterns. All 20 participants were able to complete horizontal level 4H (3.59 cyc/deg) and vertical level 5V (3.20 cyc/deg) in all trials. No participant was able to complete horizontal level 1H (14.35 cyc/deg) or vertical level 1V (16.01 cyc/deg) in any trial.

The endpoints of the staircase algorithm provide an overview over the results but do not represent all individual datapoints. We therefore additionally fitted sigmoidal psychometric functions to the trial data. We determined the perceived resolution threshold from the inflection point of the functions and calculated the 95% confidence intervals. Figure 6 shows two separate fits for horizontal and vertical lines. This gives us a perceived resolution of 7.09 cyc/deg (95% CI: 6.91 to 7.27) for horizontal and 7.43 cyc/deg (95% CI: 7.02 to 7.89) for vertical lines.

We fitted separate psychometric functions for trials with different stimulus and background conditions but found similar thresholds for all of them. The biggest differences can be seen for trials with different stimulus colors. Participants reached a threshold of 6.68 cyc/deg (95% CI: 6.09 to 7.28) in trials with red stimuli, a threshold of 7.39 cyc/deg (95% CI: 7.29 to 7.48) in trials with green stimuli and a threshold of 7.28 cyc/deg (95% CI: 7.10 to 7.47) in trials with blue stimuli (Figure 7). These results agree with observations from Table 1, since participants performed slightly worse in the staircase algorithm for red stimuli as well.

Participants achieved a threshold of 7.11 cyc/deg (95% CI: 6.89 to 7.33) in trials with maximum stimulus luminance and a threshold of 7.39 cyc/deg (95% CI: 7.27 to 7.50) in trials with equal stimulus luminance (Figure 8).

They reached a threshold of 7.25 cyc/deg (95% CI: 7.01 to 7.49) in trials with a black background, a threshold of 7.20 cyc/deg (95% CI: 7.02 to 7.38) in trials with complementary color backgrounds (red/green, green/red, blue/orange) and a threshold of 7.36 cyc/deg (95% CI: 7.18 to 7.53) in trials with the same color backgrounds (Figure 9).

4. Discussion

RS glasses use an optical architecture that is different from conventional display technologies. We explained how the human pupil influences the perceived brightness of an RS display and presented a physical derivation for the RS luminance

L_{RS}

. Since the pupil adapts to ambient luminance, RS glasses appear darker in dark environments and brighter in bright environments. For an observer with standard pupil dynamics, the RS display luminance multiplies fivefold from mesopic to bright photopic environments.

For conventional displays, Troland is commonly used to describe retinal illuminance as a function of pupil size [18]. Although Troland accounts for the pupil area

A_{P}

in Equation (10), one still needs to make an assumption about the eye’s focal distance

f_{e}

(i.e., use a value for a standard eye) to calculate an actual Troland value [13]. In our derivation, however,

f_{e}

cancels out in Equation (11) after setting

E_{R_{Lam}}

equal to

E_{R_{Scan}}

. This means that

L_{RS}

is independent from any assumptions about the eye apart from the pupil size.

We picked the model from Stanley and Davies [17] to express pupil size as a function of ambient luminance. Using another model with additional parameters such as age [16] could further increase the accuracy but would also increase the complexity. The fundamental relationship between pupil size and RS luminance holds independently from this model choice.

Using the ambient luminance

L_{A}

and combiner transmittance

τ_{C}

, we defined the ambient contrast ratio for RS glasses as

{ACR}_{RS} = \frac{L_{RS}}{τ_{C} L_{A}}

. In some previous research [4,5], the ambient contrast ratio has been defined as

ACR = \frac{L_{on} + {τ L}_{A}}{L_{off} + τ L_{A}}

, where

L_{on}

and

L_{off}

are the luminance of on and off pixels,

τ

is the see-through transmittance, and

L_{A}

is the ambient luminance. RS displays do not project light into dark pixel areas.

L_{off}

is always zero, so the denominator of the two formulas does not differ. More thought must be put into the ambient light term

{τ L}_{A}

in the numerator. If the term is included, the ambient contrast ratio would equal 1 for an off display. This seems in conflict with the conventional notion of a contrast ratio. We therefore propose to make a distinction between a standard “ambient contrast ratio” that does not include the background term

{τ L}_{A}

in the numerator, and a second quantity, potentially called the “additive ambient contrast ratio”, that does include it. In this paper, we use the standard ambient contrast ratio.

With the derivation for the brightness and contrast of visual stimuli, we were finally able to design and evaluate our psychophysical study for the perceived resolution. Overall, we found consistent results across 15 trials with different stimulus parameters. This study had a gamified design, but we provided no reward for correct answers. Instead, we instructed the participants to give accurate feedback according to their perception. Several participants reported after the study that they did not find any visual cues to manipulate the results even if they had wanted to, hinting toward a robust experimental design.

From the field-of-view size and the number of digital pixels, we can infer that the pixels were not perfectly quadratic. This led to different spatial frequencies for horizontal and vertical staircase levels (Table 1). The fact that the correct response ratios in Figure 6 still follow the trend of a common overall sigmoidal function further speaks to the validity of our measurement approach.

The 0.8 decimal (0.097 logMAR) minimum visual acuity that participants needed to achieve to be included in the study corresponds to a visual resolution of 24.0 cyc/deg [19]. Since we found an average perceived resolution threshold of 7.2 cyc/deg across all trials, we conclude that the perceived resolution was limited by the optical resolution of the RS glasses and not the visual perception of any of the participants.

Our results indicate that the background conditions did not have a notable influence on the perceived resolution within the range of conditions we tested. This could, however, still be the case for other background parameter ranges. As expected, the RS luminance and the ambient contrast ratio, which was above 0.5 in all trials, did not influence the perceived resolution thresholds, despite varying over several orders of magnitude. These findings agree with the previous literature in which it is argued that the neural response in the visual pathway saturates, and further contrast increases do not lead to improved perceived resolution thresholds [7]. We see our study as a first step toward more human-centered characterizations of augmented reality glasses and recognize the potential for follow-up studies in additional parameter ranges. In particular, the RS luminance and ambient contrast ratio could be decreased in a future study to values around and below 0.5 to also investigate the effect of contrast levels on the perceived resolution. Since calculating and setting the ambient contrast ratio of retinal scan glasses stimuli are only possible with the correct equation for the RS luminance, a follow-up experiment that varies stimulus contrast in addition to stimulus resolution would further illustrate the benefits of our theoretical derivation.

We neither measured nor controlled for the polarization of light from the retinal scan glasses. The influence of the linear polarization from the laser source on the visual perception of retinal scan displays could be an interesting aspect for further investigation, especially when compared with conventional display architectures.

We also measured the RS glasses’ optical display resolution according to the IEC 61947-2 standard [20] and found a maximum value of 3.4 cyc/deg for monochromatic green images. In our psychophysical study, we found a perceived resolution threshold of 7.4 cyc/deg for monochromatic green images.

The perceived resolution threshold adds additional interpretability to the IEC 61947-2 optical resolution threshold, which only uses a modulation depth of 30% to determine the spatial frequency cutoff, independent of visual perceptibility. Knowing the number of pixels per degree that humans can distinguish in the glasses helps in taking better-guided design decisions for future prototypes; for example, for finding the necessary field-of-view and holographic combiner area size to show image content with a certain desired size and resolution.

Furthermore, in accordance with the ideas presented in Section 2.1.2, we expect the ratio between the perceived and optical resolution to be higher for retinal scan displays than for conventional Lambertian displays. A promising next step to test this claim would be to conduct a second, comparative study with a Lambertian augmented reality display that only differs in its optical architecture and is otherwise as identical to our retinal scan glasses prototype as possible.

The BML500P prototype used in this study still had some known limitations. In Figure 4, one can see brightness inhomogeneities, particularly in the top left corner of the green and blue stimuli. Such inhomogeneities typically arise from local variations in the diffraction efficiency of the HOE. This is a manufacturing issue that will be resolved in the future.

The second limitation was the small exit pupil of the single eyebox prototype. The RS display is only visible if the narrow light beam fully passes through the pupil. If this alignment is lost, the image is not visible anymore. Therefore, we added a 5 min fitting procedure for each participant at the beginning of our study. The nose bridge of the glasses could be quickly adjusted to fit to the pupil position of each participant (see Figure 1, left). In the future, several approaches such as eyebox replication or eyebox steering can be implemented to remediate this issue.

5. Conclusions

We found a physical derivation for the perceived brightness and ambient contrast ratio of holographic augmented reality retinal scan glasses. Building on those results, we designed and conducted a psychophysical study to determine the perceived resolution of a fully functional retinal scan glasses prototype.

The perceived brightness as quantified by the retinal scan luminance

L_{RS}

changes with pupil size: it increases for small pupils and decreases for large pupils. As a result, retinal scan displays appear brighter in bright environments and darker in dark environments in comparison to conventional displays. This is a unique and advantageous property of retinal scan glasses for augmented reality use cases.

The psychophysical study produced consistent perceived resolution thresholds of about seven cycles per degree across all trials despite varying background and stimulus conditions. This result shows that within the range of parameters we tested, the optical resolution of the retinal scan glasses and not the human visual perception was the limiting factor for the perceived resolution. Overall, this study demonstrated how both the display and the background can be included in a psychophysical experiment to better understand and optimize augmented reality display performance for realistic use cases.

Author Contributions

Conceptualization, M.R.; methodology, M.R. and P.N.; software, P.N. and M.R.; validation, M.R. and P.N.; formal analysis, M.R. and S.P.; investigation, P.N. and M.R.; resources, S.P. and T.S.; data curation, P.N. and M.R.; writing—original draft preparation, M.R.; writing—review and editing, M.R., P.N., S.P., and T.S.; visualization, M.R.; supervision, S.P. and T.S.; project administration, M.R.; funding acquisition, M.R. All authors have read and agreed to the published version of the manuscript.

Funding

Bosch Sensortec supported this study and itself acknowledges financial support for the development of the augmented reality glasses by the Federal Ministry for Economic Affairs and Climate Action on the basis of a decision by the German Bundestag and by the Ministry for Economic Affairs, Labor and Tourism of Baden-Württemberg based on a decision of the State Parliament of Baden-Württemberg as well as co-financed with tax funds on the basis of the budget financed by the European Union—NextGenerationEU.

Institutional Review Board Statement

This study was conducted in accordance with the Declaration of Helsinki and approved by the ethics committee of the Medical Faculty of the University of Tübingen (Code 410/2022B02, Approval 30 August 2023).

Informed Consent Statement

Informed consent was obtained from all subjects involved in this study. Written informed consent has been obtained from the participant identifiable in Figure 4 to publish this paper.

Data Availability Statement

The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding author.

Acknowledgments

We thank Andreas Petersen, Hendrik Specht, Matthias Heberle, Julian Heinzelmann, Henning Kästner, and the entire Bosch Sensortec optical engineering team for many fruitful discussions and technical support during this study. We acknowledge support from the Open Access Publishing Fund of the University of Tübingen.

Conflicts of Interest

MR, PN, and SP were employed by Bosch Sensortec GmbH, Reutlingen, Germany, during the creation of this work. MR and SP filed patents for Bosch Sensortec GmbH on retinal scan glasses technology. TS declares no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:

RS	Retinal Scan
HOE	Holographic Optical Element
ACR	Ambient Contrast Ratio

Appendix A

Table A1. Mathematical symbols and their meaning for the derivation of the retinal scan luminance and ambient contrast ratio.

Symbol	Meaning
$A_{P}$	Area of human pupil
$A_{R}$	Area of image on retina
$A_{R_{Lam}}$	Area of image on retina for a Lambertian system
$A_{R_{Scan}}$	Area of image on retina for a retinal scan system
$A_{S}$	Area of a light source (e.g., area of a display panel)
${ACR}_{RS}$	Ambient contrast ratio of a retinal scan system
$d_{P_{mm}}$	Diameter of human pupil in millimeters
$E_{R}$	Retinal illuminance
$E_{R_{Lam}}$	Retinal illuminance from a Lambertian system
$E_{R_{Scan}}$	Retinal illuminance from a retinal scan system
$f_{e}$	Focal distance of the human eye
$K_{m}$	Luminous efficacy constant
$L$	Luminance
$L_{A}$	Ambient luminance
$L_{Lam}$	Luminance of a Lambertian system
$L_{RS}$	Luminance of a retinal scan system
$R$	Distance between human pupil and a Lambertian display
$V (λ)$	Luminous efficiency function
$α$	First projection angle of a retinal scan system
$β$	Second projection angle of a retinal scan system
$λ$	Wavelength
${\bar{λ}}_{blue}$	Average wavelength of blue retinal scan system primary
${\bar{λ}}_{green}$	Average wavelength of green retinal scan system primary
${\bar{λ}}_{red}$	Average wavelength of red retinal scan system primary
$τ_{C}$	Transparency of retinal scan system optical combiner
$Φ_{E_{Scan}}$	Radiant flux from retinal scan system
$Φ_{blue}$	Radiant flux from blue retinal scan system primary
$Φ_{green}$	Radiant flux from green retinal scan system primary
$Φ_{red}$	Radiant flux from red retinal scan system primary
$Φ_{V}$	Luminous flux
$Φ_{V_{Lam}}$	Luminous flux from a Lambertian system
$Φ_{V_{Scan}}$	Luminous flux from a retinal scan system
$Ω$	Solid angle

References

Suzuki, M.; Ishimoto, M.; Sugawara, M.; Hasegawa, K.; Miyauchi, H.; Teshima, N.; Yasui, K. Every Aspect of Advanced Retinal Imaging Laser Eyewear: Principle, Free Focus, Resolution, Laser Safety, and Medical Welfare Applications. In Proceedings of the MOEMS and Miniaturized Systems XVII, San Francisco, CA, USA, 22 February 2018; Piyawattanametha, W., Park, Y.-H., Zappe, H., Eds.; SPIE: San Francisco, CA, USA, 2018; p. 22. [Google Scholar]
Wilm, T.; Hofmann, J.; Fiess, R.; Höckh, S.; Stork, W. Immersion-Based Holographic Wave Front Printer Setup for Volume Holographic Retinal Projection Elements. In Proceedings of the Holography: Advances and Modern Trends VII, Online, 7 May 2021; Fimia, A., Hrabovský, M., Sheridan, J.T., Eds.; SPIE: Bellingham, WA, USA, 2021; p. 12. [Google Scholar]
Maimone, A.; Georgiou, A.; Kollin, J.S. Holographic Near-Eye Displays for Virtual and Augmented Reality. ACM Trans. Graph. 2017, 36, 85. [Google Scholar] [CrossRef]
Xiong, J.; Hsiang, E.-L.; He, Z.; Zhan, T.; Wu, S.-T. Augmented Reality and Virtual Reality Displays: Emerging Technologies and Future Perspectives. Light Sci. Appl. 2021, 10, 216. [Google Scholar] [CrossRef] [PubMed]
Fidler, F.; Balbekova, A.; Noui, L.; Anjou, S.; Werner, T.; Reitterer, J. Laser Beam Scanning in XR: Benefits and Challenges. In Proceedings of the Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) II, Online, 28 March 2021; Kress, B.C., Peroz, C., Eds.; SPIE: Bellingham, WA, USA, 2021; p. 1. [Google Scholar]
Park, J.-H.; Lee, B. Holographic Techniques for Augmented Reality and Virtual Reality Near-Eye Displays. Light Adv. Manuf. 2022, 3, 9. [Google Scholar] [CrossRef]
Campbell, F.W.; Green, D.G. Optical and Retinal Factors Affecting Visual Resolution. J. Physiol. 1965, 181, 576–593. [Google Scholar] [CrossRef] [PubMed]
Ackerman, E. Bosch Gets Smartglasses Right with Tiny Eyeball Lasers. IEEE Spectrum, 4 February 2020. [Google Scholar]
Peirce, J.; Gray, J.R.; Simpson, S.; MacAskill, M.; Höchenberger, R.; Sogo, H.; Kastman, E.; Lindeløv, J.K. PsychoPy2: Experiments in Behavior Made Easy. Behav. Res. 2019, 51, 195–203. [Google Scholar] [CrossRef] [PubMed]
Pelli, D.G.; Bex, P. Measuring Contrast Sensitivity. Vis. Res. 2013, 90, 10–14. [Google Scholar] [CrossRef] [PubMed]
Chung, S.T.L.; Tjan, B.S. Spatial-Frequency and Contrast Properties of Reading in Central and Peripheral Vision. J. Vis. 2009, 9, 16. [Google Scholar] [CrossRef] [PubMed]
Dressier, M.; Rassotv, B. Neural Contrast Sensitivity Measurements with a Laser Interference System for Clinical and Screening Application. Investig. Ophthalmol. Vis. Sci. 1981, 21, 737–744. [Google Scholar]
Bass, M.; Enoch, J.M.; Lakshminarayanan, V. Handbook of Optics. Volume III, Vision and Vision Optics, 3rd ed.; McGraw-Hill: New York, NY, USA, 2010; ISBN 978-1-61583-165-4. [Google Scholar]
CIE JTC 2 CIE 018:2019; The Basis of Physical Photometry. 3rd ed. International Commission on Illumination (CIE): Vienna, Austria, 2019.
Kelbsch, C.; Strasser, T.; Chen, Y.; Feigl, B.; Gamlin, P.D.; Kardon, R.; Peters, T.; Roecklein, K.A.; Steinhauer, S.R.; Szabadi, E.; et al. Standards in Pupillography. Front. Neurol. 2019, 10, 129. [Google Scholar] [CrossRef] [PubMed]
Watson, A.B.; Yellott, J.I. A Unified Formula for Light-Adapted Pupil Size. J. Vis. 2012, 12, 12. [Google Scholar] [CrossRef] [PubMed]
Stanley, P.A.; Davies, A.K. The Effect of Field of View Size on Steady-state Pupil Diameter. Ophthalmic Physiol. Opt. 1995, 15, 601–603. [Google Scholar] [CrossRef] [PubMed]
Thibos, L.N.; Lopez-Gil, N.; Bradley, A. What Is a Troland? J. Opt. Soc. Am. A 2018, 35, 813. [Google Scholar] [CrossRef] [PubMed]
Caltrider, D.; Gupta, A.; Tripathy, K. Evaluation of Visual Acuity. In StatPearls; StatPearls Publishing: Treasure Island, FL, USA, 2024. [Google Scholar]
IEC 61947-2:2001; International Electrotechnical Commission (IEC). Electronic Projection—Measurement and Documentation of Key Performance Criteria—Part 2: Variable Resolution Projectors. International Electrotechnical Commission: Geneva, Switzerland, 2001.

Figure 1. (a) BML500P RS glasses prototype with light engine in the right temple. (b) Transparent holographic optical element embedded in the right spectacle lens.

Figure 2. Beam paths of a single pixel for a Lambertian display (top) and an RS display (bottom). Without an aperture (left), all light rays hitting the lens contribute to the luminous flux on the retina. When the human pupil acts as an aperture (right), the total luminous flux is reduced for a Lambertian display but not for an RS display as long as the narrow RS beam (diameter < 0.5 mm) fully passes through the pupil.

Figure 3. Beam paths of a single pixel entering the eye for a Lambertian display (top) and an RS display (bottom). An ideal cornea and lens (left) focus light from a single pixel onto a narrow point on the retina. Aberrations of the cornea and lens (middle) or an out-of-focus retinal plane in myopia or hyperopia (right) have a much smaller impact on the focus and perceived resolution of an RS image.

Figure 4. (a) Chinrest in front of LED monitor displaying an even green background. (b) LED monitor seen through right spectacle lens with embedded HOE. (c) Red rectangle stimulus from glasses on green background seen through right spectacle lens. (d) Trial stimulus with horizontal red lines, green background. (e) Trial stimulus with vertical red lines, green background. (f) Trial stimulus with horizontal green lines, green background. (g) Trial stimulus with green horizontal lines, red background. (h) Trial stimulus with blue horizontal lines, orange background.

Figure 5. RS luminance

L_{RS}

and RS ambient contrast ratio

{ACR}_{RS}

for ambient luminance

L_{A}

from 1 to 10,000 cd/m² and corresponding pupil diameters

d_{p}

. Radiant fluxes

Φ_{i}

, average wavelengths

{\bar{λ}}_{i}

, field of view angles

α

β

, and combiner transparency

τ_{C}

are constant.

Figure 5. RS luminance

L_{RS}

and RS ambient contrast ratio

{ACR}_{RS}

for ambient luminance

L_{A}

from 1 to 10,000 cd/m² and corresponding pupil diameters

d_{p}

. Radiant fluxes

Φ_{i}

, average wavelengths

{\bar{λ}}_{i}

, field of view angles

α

β

, and combiner transparency

τ_{C}

are constant.

Figure 6. Sigmoidal psychometric curves for perceived resolution of horizontal and vertical lines.

Figure 7. Sigmoidal psychometric curves for perceived resolution with different stimulus colors.

Figure 8. Sigmoidal psychometric curves for perceived resolution with different stimulus brightness.

Figure 9. Sigmoidal psychometric curves for perceived resolution with different stimulus and background color combinations.

Table 1. Trial parameters and results from staircase algorithm in perceived resolution study.

Trial	Trial Parameters				Number of Participants That Passed Staircase Resolution Level Horizontal [H] and Vertical [V]
Trial	Glasses	Monitor	L_RS [cd/m²]	ACR_RS	4H 3.59 c/d	3H 4.78 c/d	2H 7.17 c/d	1H 14.35 c/d	4V 4.00 c/d	3V 5.34 c/d	2V 8.01 c/d	1V 16.01 c/d
1	red	black	83	92	20	17	0	0	20	17	0	0
2	red	green	83	3	20	19	0	0	20	18	0	0
3	red	red	83	3	20	19	0	0	19	19	0	0
4	green	black	929	1032	20	20	4	0	20	20	0	0
5	green	black	83	92	20	20	5	0	20	20	0	0
6	green	red	929	38	20	20	5	0	20	20	0	0
7	green	red	83	3	20	20	6	0	20	20	0	0
8	green	green	929	38	20	20	6	0	20	20	0	0
9	green	green	83	3	20	20	5	0	20	20	0	0
10	blue	black	91	101	20	20	4	0	20	20	0	0
11	blue	black	83	92	20	20	5	0	20	20	2	0
12	blue	orange	91	4	20	20	3	0	20	20	1	0
13	blue	orange	83	3	20	20	4	0	20	20	1	0
14	blue	blue	91	4	20	20	5	0	20	20	1	0
15	blue	blue	83	3	20	20	5	0	20	20	1	0

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Rutz, M.; Neuberger, P.; Pick, S.; Straßer, T. Perceived Brightness and Resolution of Holographic Augmented Reality Retinal Scan Glasses. Appl. Sci. 2025, 15, 1540. https://doi.org/10.3390/app15031540

AMA Style

Rutz M, Neuberger P, Pick S, Straßer T. Perceived Brightness and Resolution of Holographic Augmented Reality Retinal Scan Glasses. Applied Sciences. 2025; 15(3):1540. https://doi.org/10.3390/app15031540

Chicago/Turabian Style

Rutz, Maximilian, Pia Neuberger, Simon Pick, and Torsten Straßer. 2025. "Perceived Brightness and Resolution of Holographic Augmented Reality Retinal Scan Glasses" Applied Sciences 15, no. 3: 1540. https://doi.org/10.3390/app15031540

APA Style

Rutz, M., Neuberger, P., Pick, S., & Straßer, T. (2025). Perceived Brightness and Resolution of Holographic Augmented Reality Retinal Scan Glasses. Applied Sciences, 15(3), 1540. https://doi.org/10.3390/app15031540

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Perceived Brightness and Resolution of Holographic Augmented Reality Retinal Scan Glasses

Abstract

Featured Application

Abstract

1. Introduction

2. Materials and Methods

2.1. Retinal Scan Glasses

2.1.1. Brightness in Retinal Raster Scanning

2.1.2. Focus and Resolution in Retinal Raster Scanning

2.2. Perceived Resolution Study Design

3. Results

3.1. Retinal Scan Luminance and Ambient Contrast Ratio

3.2. Perceived Resolution Study Results

4. Discussion

5. Conclusions

Author Contributions

Funding

Institutional Review Board Statement

Informed Consent Statement

Data Availability Statement

Acknowledgments

Conflicts of Interest

Abbreviations

Appendix A

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI