Roadmap on spatiotemporal light fields

¹Centre for Disruptive Photonic Technologies, School of Physical and Mathematical Sciences & The Photonics Institute, Nanyang Technological University, Singapore 637378, Singapore

²Optoelectronics Research Centre & Centre for Photonic Metamaterials, University of Southampton, Southampton SO17 1BJ, United Kingdom

³School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, People's Republic of China

It is a perpetual human aspiration to manipulate extreme spatiotemporal structures of light, as which not only promise the exploration of fundamental physics, but also meet the ultimate applications for ultrafast and ultra-intense energy or information extraction and storage. This objective can be aligned with the human aspiration of pursuing more beautiful and grandiose music. Indeed, the importance of electromagnetic wave to optics is same as the acoustic wave to music. Therefore, a piece of music played by an instrument is analogue to a temporal optical pulse (figures 1(a) and (b)). For convenience of composition, the composer use frequency notation, the stave, to abstractly describe the temporal music (figure 1(c)), which can be seem as the process of Fourier transformation (the music player translation of the score is just the inverse Fourier transformer). Similarly, in optics, to characterize a chirped pulse, we usually measure it in the spectral domain to record the pulse information (figure 1(d)), rather than directly capture the very fast temporal signal. In retrospect, music evolved from ancient simple solo by a single instrument to the contemporary complex symphony by spatially distributed instruments recorded by complex full score (figures 1(e) and (f)). In parallel, recent advances of optics enable researchers to spatiotemporally sculpture light pulses and correspondingly characterize them with novel spectroscopic technology in the spatio-frequency domain (figures 1(g) and (h)), opening a whole new chapter of modern optics.

Zoom In Zoom Out Reset image size

Structuring light in all its degrees of freedom (DoF) is steadily gaining traction [1], extending our familiar 2D transverse mode pattern to 3D spatial control, and to include spatiotemporal control for 4D even higher-dimensional forms [2]. Temporal shaping of light, especially in the ultrafast domain, has already benefited diverse applications, from material manufacturing to ultraprecise metrology. For example, the locking of carrier envelope phase of optical pulse can result in frequency comb as a powerful tool for ultraprecise metrology [3]. The shaping of ultrafast laser pulse into burst profiles enables the improved effect of ablation-cooled material removal for advanced laser processing [4]. On the other hand, spatial shaping of light also attracted growing attention for arbitrary tailoring of light patterns for diverse purposes, with recent advances of the exploiting of geometric transformation under general symmetry to reveal the 'hidden' DoF of light [5]. When temporal shaping meets spatial shaping with complex space-time (ST) nonseparability to control, spatiotemporally structured light comes into play, injecting new vitality to photonic science and applications [6]. For instance, the generation of toroidal ST nonseparable pulse has promised to excite new forms of multipole moments in matter and free space [7]. The ultrafast vortex or vector pulses can be used to manufacture helical micro- or nano-structure for producing chiral functional materials [8], emulate topological textures of quasiparticles in condensed matters [9], and to excite new nonlinear effect in light-matter interactions [10]. The typical works above are related to the same goal of manipulating novel topological structures in ultrasmall and ultrafast ST domain. Therefore, sculpturing light in the spatiotemporal domain with increasingly complex topological structures represent a promising research direction for both fundamental physics and applied optics. Notably, 'spatiotemporal' can refer to much broader sense beyond the above examples. For such a dynamic and diverse field, it is inevitable some of the important topics and applications, such as intense laser physics, self-phase modulation, and time refraction, may have been left out. We expect these topics and application will benefit from the developments covered by this roadmap and may be covered in its future version.

The intention of this roadmap is to bring the recently rapidly growing developments in this field on the generation and characterization of spatiotemporally structured light pulses as well as the associated novel physics. We will make an attempt to summarize a unified framework for their classification, generation and characterization, and review the widespread and potential applications of spatiotemporal light pulse along with a perspective on new opportunities for both fundamental and applied science. The chapters are logically organized as follow. We start by some important physical effects of spatiotemporal light fields with ST separable structures (chapters 2–3). Then, we introduce novel forms of exotic ST coupling structures of light, especially, where multiple chapters focus on optical vortices carrying orbital angular momentum (OAM) for highlighting its paradigmatic bases and celebrating the 30th anniversary of OAM since its connection to structured light (chapters 4–7), and a number of emerging forms and generations of spatiotemporal light fields from localized wave packets to paraxial beams (chapters 8–13). Following that are advanced technologies of measurement, detection, and characterization of complex spatiotemporal light structures (chapters 14–15). Finally, we include advanced applications of spatiotemporal light fields, from light–matter interaction to imaging, and from classical to quantum regimes (chapters 16–19).

¹Physics & Informatics Laboratories, NTT Research, Inc., 940 Stewart Drive, Sunnyvale, CA 94085, United States of America

²College of Optics & Photonics-CREOL, University of Central Florida, Orlando, FL 32816, United States of America

³School of Applied and Engineering Physics, Cornell University, Ithaca, NY 14853, United States of America

In a mode-locked light source, fields in many modes self-synchronize to oscillate collectively, leading to a beam of pulses of light that is, by any reasonable human standards, utterly astounding. The pulses are short. In a sense, they can be as 'short' as the proton is small (1 femtometer). The regular train of pulses equivalently defines a 'comb' of discrete frequency 'teeth' in the spectral domain. This frequency comb is equally remarkable: if a 10 THz bandwidth optical comb is compared to a 10 cm long hair comb, the optical comb's teeth correspond to about 1 nm thickness. These unique qualities have uniquely enabled an ever-expanding list of methods and applications, for which there have been many excellent reviews and books, and at least three Nobel prizes so far.

Thus far, virtually all optical mode-locking has been one-dimensional: the modes that lock together all share a roughly identical transverse (x,y) shape, and as a result the physics can be accurately described by factoring out those spatial dimensions, considering nonlinear optical dynamics in just one space dimension (z) plus time (t). One of the grand successes of mode-locking physics has been to understand and classify stable mode-locked pulses within certain families of one-dimensional dissipative Kerr solitons, stationary solutions to dissipative nonlinear wave equations [11]. In the last several years, we and many other researchers have started to explore a more general scenario, so-called spatiotemporal mode-locking (STML) [12–15]. In STML, the modes that lock together can be of a much richer form, and the mode-locked pulses formed are multimode, or spatiotemporal, dissipative solitons.

There are at least three reasons to be excited about STML, which will be elaborated on in this article.

• 1.  
  STML represents arguably the least understood regime of mode-locking physics. Better understanding of STML is thus especially likely to (eventually) lead to breakthrough discoveries or inventions in mode-locked light sources that translate to even more astounding, application-enabling photonic capabilities.
• 2.  
  The ultrafast, high-dimensional nonlinear physics involved in multidimensional mode-locking is both an ideal laboratory testing ground, and an ideal candidate, for emerging data-driven control and nonlinear inverse design techniques.
• 3.  
  Light sources based on spatiotemporal nonlinear optical wave physics are uniquely qualified to produce spatiotemporally structured light with extremely high intensity.

A first challenge (rather specific and technical) for the science of STML is that the multimode graded-index gain fiber utilized in the first experimental demonstration [12] (figure 2(a)) is not commercially available, making direct replication of that specific laser design challenging. Some aspects of multidimensional mode-locking physics can be explored in a more accessible cavity design, based on few- or single-mode step-index gain fiber, and a large section of passive graded-index multimode fiber [12] (figure 2(b)). This cavity, made entirely of off-the-shelf components, is suitable for exploring a rather constrained subspace of multidimensional dissipative soliton physics, and for developing techniques to observe and control it. It is, however, unlikely to be the basis for major breakthroughs in mode-locked laser performance, especially if the performance targets are maximum peak intensity or pulse energy.

Zoom In Zoom Out Reset image size

This challenge is related to the more exciting issue of achieving better control over STML and/or observing it in more controllable settings. In multimode fibers, manufacturing disorder and mechanical perturbations cause a complicated coupling between transverse modes. Historically, observing multidimensional solitons in nature has been extremely challenging, owing to their fragility to even minor perturbations [16]. Yet, we have found that in real disordered multimode lasers, a wide range of qualitatively diverse multimode dissipative solitons form spontaneously from noise, and then stably persist [13]. While the fact these stable spatiotemporal pulses form at all challenges many historical notions of multidimensional nonlinear waves [16], we expect that researchers will need to learn how to better control these pulses' spatiotemporal structure before they will have widespread utility. To both better understand and control multidimensional mode-locking, it will be essential to design and introduce new optical systems and mode-locking strategies in which disorder is either strongly suppressed, and/or in which the cavity's configuration can be adjusted rapidly and along many different DoF.

Finally, our theoretical understanding of the physics of mode-locking in multimode optical cavities is still at an early stage, especially in comparison to the accuracy with which we can describe one-dimensional mode-locking. While advances in control and simplification of experimental prototypes will help, the greatest opportunities presented by multidimensional mode-locking physics are implicit in their demand for new theoretical techniques. While complexity has never stopped physicists from developing predictive models, and progress on predictive models of STML is ongoing, today we have fewer excuses than ever. We are amidst an explosion of progress in data-driven and computationally intensive modelling and control techniques, such as deep learning [17], inverse design [18], and reinforcement learning. These provide researchers with an unprecedented toolbox to understand—and control—high-dimensional nonlinear processes like STML.

To address the first and second challenge, we recommend that researchers explore new cavities and photonic systems for multimode mode-locking. Researchers may design and fabricate new multimode gain fibers suitable for STML. Fibers currently being developed for mode-division multiplexed telecommunications are suitable, for example. Multidimensional nonlinear wave physics may also advance lasers based on step-index multimode gain media, such as the multimode transient dynamics that have been predicted to dramatically increase achievable mode-locked pulse energy [13, 15]. Another especially promising route is STML in cavities or regenerative amplifiers that do not consist of fiber at all, but rather rod-like or thin-disk gain media (figure 2(d)). In fully multimode lasers with sufficiently high gain per pass (whether fiber or otherwise), including elements like spatial light modulators (SLMs) within the cavity should afford researchers with rich, high-dimensional control (figure 2(c)). Mode-locking mechanisms beyond nonlinear polarization rotation, such as Kerr-lens-inspired [19] or Mamyshev saturable absorbers, are likely to prove more useful and flexible in the long run. Finally, still greater control and performance may be achieved with STML not in lasers, but in coherently driven resonators [20] or bulk resonators (figure 2(e)).

At its heart, the theoretical challenge of STML is an inverse problem: How can we cause (by fixed cavity designs or adjustable control parameters) coherent mode-locked pulses to self-organize with the qualities (shape, energy, stability) we desire? For example, if we want to maximize the peak intensity of a pulse formed by STML, how should we configure the laser cavity? If we want to produce a pulse whose electric field envelope, E(x,y,t), has a particular distribution, what elements (e.g. a SLM with a specific phase distribution ϕ(x,y)) could we place inside the cavity to ensure that pulse is produced? Answering these questions is challenging because the physics that underlies pulse formation is high-dimensional, nonlinear, stochastic, and dynamic. In other words, the steady state pulse E(x,y,t) is not at all a simple deterministic function of the cavity parameters. On the other hand, with modern tools this challenge seems surmountable. In fact, we believe STML's inverse problem is a uniquely accessible, and even uniquely enticing, hard inverse problem.

Why? For one, while the STML inverse problem is significantly more challenging than any problem attempted in inverse photonic design so far [18], it is not obviously harder than recent feats of data-driven models, such as the mastery of ImageNet, text generation, and the games of Go or Starcraft II (These feats are all effectively solutions to high-dimensional, stochastic nonlinear inverse problems). Just as games of Go may be simulated quickly, experiments with reconfigurable multimode lasers can be executed rapidly, facilitating the training of data-intensive schemes like deep reinforcement learning. Second, in contrast to Starcraft and Go, multimode lasers are within or just beyond the reach of traditional physics models. For this reason, solving STML's inverse problem represents a uniquely apt opportunity to develop symbiotic methods that combine traditional physics with computation and data-intensive machine learning. Success in this pursuit will have far-reaching implications not just in new light sources with tailored spatiotemporal features, but more broadly in the control of complex nonlinear systems throughout science and engineering, such as fluid dynamics, chemical reactions, and magnetohydrodynamics.

As other articles in this and other Roadmaps lay bare, perhaps the single most promising frontier for future developments in light and light-based applications is the control of light in many dimensions and DoF. This is particularly true for high-intensity light, which is used to drive and control extreme processes with multifaceted spatiotemporal scales, like the dynamics of plasmas, free electrons, and x-rays. To shape high-intensity light in space and time, it seems unlikely that it will be possible to linearly filter it from an unstructured pulse, since such multidimensional pulse carving will inevitably be lossy, nonlinear, and prone to damage. Instead, it seems more promising (or at least highly complementary) to learn how to shape light directly from the source. Addressing this challenge of tailored light from the source is the inverse problem of STML: how can we design or control a complex, multimode nonlinear photonic system like a laser so that it self-organizes coherent fields with the spatiotemporal qualities we desire? This STML inverse problem is enticing as a route to extremely high intensity, spatiotemporally tailored light (and all its applications), and as an accessible challenge problem for physics-informed, data-driven control of complex nonlinear physics.

Department of Electrical Engineering, University of Southern California, Los Angeles, CA 90089, United States of America

Structured light has gained increasing interest recently due to its unique amplitude and phase spatial distributions that can be used in various applications [1]. Such optical beams could carry a pure single spatial mode or a coherent superposition of many spatial modes, and these modes can be from one of various types of modal basis sets (e.g. Laguerre Gaussian (LG), Hermite Gaussian (HG)). Moreover, each mode: (a) is orthogonal to all other modes within that set, and (b) has a complex weighted coefficient.

For a single-frequency beam at a given propagation distance: (a) a single-mode beam has a static spatial distribution of that mode [1], whereas (b) a beam composed of many weighted modes can be tailored to have nearly any static amplitude and phase spatial distribution [21]. This tailoring is accomplished by the constructive and destructive interference that occurs among the different orthogonal modes [21].

The above beams are generated from a single-frequency source. However, spatially structured wave packets that are generated from multiple frequencies can exhibit interesting dynamic (i.e. time-variant) behavior at a given propagation distance [22]. Such ST wave packets can be achieved by various methods [22], with one example being through the synthesis of multiple coherent frequency lines from an optical comb and each line carrying a different combination of weighted orthogonal spatial modes [23]. Depending on the elapsed time, the comb frequency spacing (Δf) will induce a time-dependent phase shift Δϕ(t) = 2πΔft between neighboring lines. A phase shift at a given propagation distance will be frequency dependent, such that the instantaneous constructive or destructive interference between a spatial mode on one frequency with a spatial mode on a different frequency would change with time (see figure 3) [23]. Therefore, when combining multiple modes on each of multiple frequencies, time-variant spatial profiles can be generated that result in an ST wave packet with a dynamic spatial distribution [23].

Zoom In Zoom Out Reset image size

The transmission of multiple frequencies each with multiple modes can produce dynamic ST wave packets (see figure 4) [22]. Examples of such structured-light ST wave packets include those that carry OAM. Light carrying OAM can be represented as a subset of LG modes, and can be characterized as having: (i) a phasefront that 'twists' in the azimuthal direction as it propagates, (ii) the OAM value, ℓ, is the number of 2π phase shifts in the azimuthal direction, and (iii) the intensity profile has a ring structure with a central null [21]. Moreover, the index p + 1 of LG modes indicates the number of intensity rings in the radial direction.

Zoom In Zoom Out Reset image size

Examples of dynamic spatiotemporal light include the following:

• (i)  
  ST wave packets carrying OAM with time-dependent beam radii (figure 4(a)) [24]: an ST wave packet carries a single OAM value, and its beam radius expands and contracts periodically at a given propagation distance. Such an ST wave packet is generated by synthesizing comb lines carrying complex-weighted LG modes with the same single azimuthal index ℓ but with multiple radial indices p. The time-varying frequency- and mode-dependent constructive and destructive interference between the p modes results in the dynamic oscillating behavior.
• (ii)  
  ST wave packets carrying two forms of orbital angular momenta (figure 4(b)) [23]: an ST wave packet can carry two forms of orbital angular momenta when viewed at a given propagation distance. The wave packet's momenta can be tailored to include: (1) rotation : the wave packet rotates around its own center, and (2) revolution : the rotating wave packet can also revolve around some external axis point from which it is spatially offset. Note that this is similar to the way the Earth revolves around the Sun yet simultaneously rotates around its own axis. Such an ST wave packet is generated by combining multiple frequency lines, with each line carrying complex-weighted LG modes composed of a single ℓ value and multiple p values. By choosing proper weights, the dynamic frequency- and mode-dependent constructive and destructive interference produces an LG mode with a spatial offset that has rotation around the mode center and revolution around the central axis.
• (iii)  
  Near-diffraction-free ST OAM wave packets with tunable group velocities (figure 4(c)) [25]: An ST wave packet that carries OAM can be created that has reduced diffraction and a tunable group velocity. Such ST wave packets are generated by synthesizing multiple comb lines, each line carrying a single Bessel mode with an identical ℓ value but a different Bessel radial spatial frequency k_r . Reduced diffraction is achieved by the use of Bessel modes that naturally have such a characteristic. The intensity peak of such an ST wave packet occurs when the different Bessel modes on different frequencies constructively interfere. Due to the dynamic phase delay between the neighboring Bessel modes, the constructive interference occurs at different times as the wave packet propagates. This movement of the wave packet intensity peak results in a group velocity that can be controlled by selecting the specific modes and frequencies.

There are several challenges in the generation and propagation of dynamic spatiotemporal structured light with high modal purity, including the following:

• (1)  
  Diffraction : diffraction might affect the dynamic spatial properties when viewed at different propagation distances. This is because: (i) higher-order modes diffract faster than lower-order modes, thus producing a different coherent spatial interference pattern depending on propagation distance; and (ii) the same mode on different frequencies will diffract differently, thus producing a different dynamic spatial interference pattern depending on propagation distance. This frequency-dependent modal diffraction can be significant when the frequency spacing increases and the propagation distance grows beyond the Rayleigh range [23]. In such cases, the resulting modal purity of the spatiotemporal light can decrease. To alleviate this challenge, it might be possible to use Bessel modes with limited diffraction to reduce this effect [25].
• (2)  
  Modal Basis Set : different modal basis sets can be carried on multiple frequency comb lines to achieve different dynamic properties. For example, circularly-symmetric LG and Bessel modes may be well-suited for dynamic angular and radial movement, whereas rectangularly-symmetric HG modes may be well-suited for dynamic lateral movement. Other spatial modes might also be explored for various dynamic behavior. Moreover, apart from modes with structured phase, beams with spatially tailored polarization states (e.g. vector modes) could also be further explored for achieving interesting dynamic spatiotemporal behavior.
• (3)  
  Comb Source : the properties of the optical frequency comb source can affect the generated ST wave packets. Specific examples include: (i) the frequency spacing can determine the speed of the dynamic change as the spatial distribution depends on the relative phase shift Δϕ(t) = 2πΔft; (ii) the number of comb lines can affect the quality of the spatiotemporal light, with more comb lines providing higher purity and dynamic range; (iii) a highly coherent comb is desirable, since the phase noise on the comb lines could induce phase distortion on the modes, thus degrading the generated spatiotemporal light.

There are various potential advances in generating, propagating, and using dynamic spatiotemporal light composed of multiple modes on multiple frequencies. These include:

• (1)  
  Spatial Modulation : in general, spatial modulation should be performed on multiple comb-line frequencies to generate spatiotemporal light. Such spatial modulation can be achieved several ways, including the use of SLMs [25], metasurfaces [26] and multi-plane light converters [27]. Some desirable features of future spatial modulation include rapidly tunable and reconfigurable operation for many modes, broad bandwidth covering a large number of comb lines, compact size, low cost, and high purity of complex weighted modes.
• (2)  
  Photonic Integrated Circuits (PIC) : PIC could reduce the size, weight, complexity, power consumption, and cost of spatiotemporal light generation. It might be desirable to have a single device for spatiotemporal light generation that includes multiple elements (e.g. a comb source, wavelength (de)muxes, and spatial amplitude and phase modulators of the individual frequencies. This vision could encompass technologies such as: (i) integrated Kerr frequency combs based on optical microresonators [28], (ii) integrated spatial mode converters [29], and (iii) tunable optical gratings. Integration of these various types of devices is quite challenging, but future deployment would likely be significantly accelerated with PIC implementation.
• (3)  
  Applications : single-frequency-based structured light has shown various potential applications, such as imaging, sensing, and communications [1]. At present, it is interesting but still unclear as to the potential applications of such dynamic ST wave packets. We imagine that there are various applications that have yet to be identified, taking advantage of the dynamic spatial distributions. Moreover, it remains to be explored how to utilize dynamic ST wave packets in a highly reconfigurable communication system. Many more unanswered questions exist, and yet it is exciting to ponder the uses of this novel light.

Unlike single-frequency-based structured light, ST wave packets can exhibit unique dynamic spatial properties due to the time-dependent interference of various spatial modes that are carried on multiple comb-line frequencies. This can be considered as a 'toolkit' for generating dynamic light distributions with tailorable properties. It is exciting to envision the future innovations of more sophisticated spatiotemporal light and their potential unique applications.

Grupo de Sistemas Complejos, ETSIME, Universidad Politécnica de Madrid, Rios Rosas 21, 28003 Madrid, Spain

Space-time couplings (STCs) may be intrinsic to the nature of waves, and as such unavoidable. They become relevant in few-cycle, ultrafast pulses (UFPs) and ultrafast vortices (UFVs). Even if one "prepares" a ST separable field $E\left( {t,x,y} \right) = P\left( t \right)B\left( {x,y} \right)$ on a certain aperture, of frequency spectrum $\hat E\left( {\omega ,x,y} \right) = { }\hat P\left( \omega \right){ }B\left( {x,y} \right),$ each color propagates differently, transforming into a nonseparable spectrum $\hat E^{^{\prime}}\left( {\omega ,x,y} \right) = \,\hat P\left( \omega \right)\,B^{^{\prime}}\left( {\omega ,x,y} \right),$ and thus a nonseparable field $E^{^{\prime}}\left( {t,x,y} \right)$ upon propagation. The associated STC effects are known for decades, but are still often ignored. It took two decades to definitively confirm that the carrier-envelope-phase (CEP) in a focal volume differs from Gouy's phase [30] because of these STCs, to mention a crucial parameter in field-sensitive light-matter interactions. In UFPs, STCs such as sub- and superluminality, spatial chirps, pulse front curvature and broadening, all in a vacuum, become only large when the UFP duration decreases down to single- and sub-cycle durations, as observed in single- and sub-cycle THz pulses [31].

For UFVs, an azimuthal-temporal coupling has turned out to impose a minimum duration that may be well-above single-cycle durations, e.g. ${{\Delta }}t \geqslant \surd \left| l \right|/{\omega _0},$ where ${\omega _0}$ is the central frequency and l the topological charge (TC), for pulsed Laguerre–Gauss (LG) beams [32]. STCs also become large when the UFV duration approaches the above minimum duration, as from figures 5(a) and (b), with the greatly enhanced transverse chirp in (c), occurring at minimum durations of one or more cycles for large TCs. The limit ${{\Delta }}t \approx \surd \left| l \right|/{\omega _0}$ has not yet been reached, but experiments approximate it from afar. Small transverse chirps have been reported in [33].

Zoom In Zoom Out Reset image size

It turned out that the most important of these STCs are controlled by a single parameter pertaining the femtosecond laser source, at least for high quality UFPs and UFVs [30, 34]. The ${g_0}$-factor characterizes the $\omega$-dependence of the Rayleigh range of the $\omega$-dependent beam profile $\hat E\left( {\omega ,x,y} \right)$. Depending on the ${g_0}$-factor of the broadband source creating the UFV, different colors are focused differently (figure 6(a)), which translates into different STCs, as in the focused UFVs in figure 6(b) differing in duration and transverse chirp. Measurements of ${g_0}$ would predict these STCs, but they are scarce. The only systematic measurement yielded values between $- 1$ and ${ } - 2$, and predicted indeed the observed STCs [30].

Zoom In Zoom Out Reset image size

The knowledge and control of these intrinsic STCs is imperative in experiments with UFPs and UFVs. They ultimately and inevitably affect all broadband laser-matter interactions Jolly [35], e.g. highlights the strong dependence of electron acceleration on ${g_0}$. Its determination will open the route to control and optimization of these interactions.

In the quest towards ever shorter, higher TC, and higher quality pulses, these intrinsic STCs will eventually dominate. The current techniques of ST resolved characterization of ultrashort pulses such as spatially-resolved Fourier-transform spectrometry, the SEA-F-SPIDER technique, Hartmann wavefront sensors and others, are more than enough for the observation of these STCs, as done for the 5.5 fs UFV in [36], and even for EUV vortices in [37].

For UFPs without OAM the observation of intrinsic STCs requires the minimization of other STCs arising from their generation or shaping devices that may mask the former, or to enhance the intrinsic STCs by lowering their duration down to single-cycle or sub-cycle pulses. Indeed, the few unambiguous observations involved almost perfectly focused Gaussian beams [30] and single and sub-cycle THz pulses [31].

With UFVs, there is more flexibility, but also added difficulty. Durations need to be decreased and/or TCs need to be increased to get somewhat closer to the limit ${{\Delta }}t \approx \surd \left| l \right|/{\omega _0}$ while maintaining good vortex quality. The shortest UFVs generated are close to two cycles, but with low TC [36, 38]. UFVs with TCs reaching the hundred are no longer uncommon in high-harmonic generation experiments [37], or directly from laser sources, but they are quasi-monochromatic. The challenge is to do both together. As a reference, a single-cycle UFV must carry TC $l$ = 27 for the intrinsic STC effects to be huge. This is far from current technical achievements, but STCs would also be large with, say, double duration or half TC, as in figure 6. The small but observable STCs in [33] belong to an UFVs with even smaller TC $l$ = 5 and larger duration ${{\Delta }}t$ = 6 fs.

Techniques of characterization of the ST-resolved optical field are well-established, but they are still complex and time-consuming. A measurement of the ${g_0}$-factor of the femtosecond source with which the UFVs is generated is simpler and provides a first approximation to the expected STCs, as corroborated in [30]. Systematic measurements of the ${g_0}$-factor of different sources would fill the gap in the knowledge of typical values. The THz source in [31] appears to be characterized by ${g_0} \approx$+1, high-power femtosecond laser sources are conjectured to have also ${g_0} \approx$+1 [35], but negative values were measured after post-compression at hollow core fibers [30]. A measurement of ${g_0}$ only requires a spectrally resolved measurement of the Rayleigh range, or equivalently, a spectrally resolved measurement of the beam width [30], and would allow us to discern, for example, which of the focused UFVs in figure 6(b) would interact with matter in an experiment.

Demands of high temporal resolution and control of OAM content in applications such as superdense optical communications, ultrafast spectroscopy, or ultrafast quantum interference experiments, will push research towards a new generation of UFV generators reaching the fundamental limits. The optimization of these applications will press towards a full characterization of the intrinsic STCs in UFVs.

The generation of sub-2-cycle UFVs with current UFV generation techniques presents difficulties associated with chromaticity. Recent advances report added tunability of TC and energy, but not in shortness. Spiral phase plates, astigmatic mode converters, computer generated holograms, diffractive elements displayed in SLMs, and others, present a variety of dispersive effects with broadband radiation including angular, group velocity, delay, and TC dispersion that enlarge or deteriorate UFVs. Many techniques have been implemented to eliminate them, often at the price of introducing expensive, specially designed optical components.

Advances would involve the development of achromatic vortex generators. An intrinsically achromatic vortex generator based on interference of two polychromatic Gaussian beams in a Sagnac interferometer and employing only mirrors and beam-splitters has been proposed [39]. It does not present limitations with regard to energy and wavelength, but still is limited to single-charged vortices.

Alternate routes will be improvement of nonlinear compression techniques, and, in strong-field, broadband light-matter interactions, improved control of OAM content of high-harmonics in attosecond pulse generation experiments. The new frequencies in supercontinuum spectra generated from self-phase modulation of femtosecond vortices in multiple thin plates have been shown to preserve the vortex structure, opening a road for the generation of near single-cycle UFVs of different TCs in the near infrared (IR) [40]. In turn, in strong-field light-matter interactions with few-cycle, near-IR UFVs, recent developments in high-harmonic and attosecond pulse generation allow the control of the OAM content of the harmonics using multiple, non-collinear driving UFVs with equal or unequal TCs, and via spin–orbit coupling. It appears to be possible the generate, in principle, a spectrum of harmonics with a single-valued OAM, which points to a new way to achieve extremely short vortices with a single-valued TC.

Other STCs considered in this roadmap may be unwanted, but are in principle removable. Others are intentionally introduced for certain purposes, as in the ST wave packets in chapters 8 and 12. The STCs considered here are fundamental, mere consequences of the ondulatory nature of light. Broadening, distortion, separation of colors in UFVs of very short duration and high OAM propagating in vacuum, are multifaceted manifestations of approximating the limit of existence of these wave objects, and can ultimately be understood as a 'reaction' of waves to too high structuring at too short time scales. For the time being, only small manifestations have been observed, but as technology in UFV generation advances, it will be necessary to face up to them in order to improve the myriad of applications of UFVs.

As a last remark, pulsed LG beams underlie the discussion in this chapter because of their broad use and since they can be free of STCs at long durations. Similar limits and associated intrinsic STCs do exist for other wave packets carrying OAM such as diffraction-free, ST wave packets (in addition to the STCs needed for diffraction-free propagation), which will be observable when synthesized in the few-cycle regime.

¹Physics Department, Pusan National Umiversity, Geumjeong-gu, Busan 46241, Republic of Korea

²School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, People's Republic of China

³School of Optical and Electronic Information and Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, Hubei 430074, People's Republic of China

⁴Zhangjiang Laboratory, 100 Haike Road, Shanghai 201204, People's Republic of China

Optical vortex beams possess zero intensity phase singularities with spiral phases. Recently, new optical vortices with phase singularities in ST have been demonstrated [41]. Such optical vortices are referred to as spatiotemporal optical vortices (STOV). In contrast to the longitudinal OAM of vortex beams, spatiotemporal vortices possess the transverse OAM.

The generation of the STOVs is based on the Fourier synthesis by a pulse shaper with a two-dimensional (2D) SLM (figure 7) [41]. In the setup, the optical frequencies are spread on the SLM and recombined which can be understood as a Fourier transform and an inverse Fourier transform, respectively. By applying a spiral phase profile on the SLM, which is the spatial frequency—frequency domain, a spatiotemporal phase singularity in ST occurs by the recombination inverse Fourier transform process.

Zoom In Zoom Out Reset image size

Even though the spatiotemporal vortex has been demonstrated very recently, it has attracted significant attention already. Lately, the research progress on the spatiotemporal vortex has advanced noticeably. For example, the nonlinear processes, such as the second harmonic generation (SHG), on the spatiotemporal vortex have been studied [42]. In the SHG process, the TC of the spatiotemporal is doubled indicating the existence and the conservation of the transverse OAM. Forming a spatiotemporal vortex at a tight focus also has been theoretically studied. It is believed that such tightly focused transverse OAM will induce unique light-matter interactions.

More sophisticated optical wave packets with spatiotemporal vortices have been studied in a variety of aspects. Multiple spatiotemporal vortices with designed TCs varying in time have been demonstrated [43]. It is believed that such wave packets can be useful for telecommunications. The vortex and therefore the direction of the OAM can be also adjusted [44]. It has been demonstrated that the spatiotemporal vortex may have a tilted OAM direction by a simple lens system [45]. A spatiotemporal vortex can have an exotic Bessel profile in ST. Such a Bessel vortex is a localized wave packet that resists the dispersive and diffractive effects. As the vortex line is bent to form a ring, an optical toroidal vortex is formed (figure 8) [46]. Toroidal vortices, such as smoke rings, are prolific in fluids, but it has been elusive in optics. However, the transverse phase singularity has led the way to the first demonstration of the optical toroidal vortex.

Zoom In Zoom Out Reset image size

The spatiotemporal vortex has a spatiotemporally coupled wave packet structure and such structures are quite challenging to attain in general. As the spatiotemporal vortex is achieved, it is believed that it has a variety of applications in nanoparticle manipulation, telecommunications, quantum optics, unique light–matter interaction, nonlinear interaction, etc due to its unique structure with a transverse OAM property.

However, the future research direction is rather clear. The current research desires even more complicated optical wave packets. For example, the localized waves with spherical harmonic symmetries have very complicated structures with optical vortices [47]. Interestingly, such wave packets mimic Hydrogen atom orbitals and therefore, they are referred to as hydrogen orbital-like wave packets. Optical wave packets with spherical harmonic symmetries are three-dimensional (3D) localized waves that maintain the 3D profiles in dispersive and diffractive propagation. Achieving such novel states of light mimicking atomic structures will be an interesting achievement. However, there has not been much experimental progress to achieve them even though they have been theoretically proposed quite a while ago (>10 years ago). The reason for lacking experimental progress is that such a complicated optical structure has been a significant experimental challenge. The Hydrogen orbital-like wave is merely one example. In fact, there are numerous theoretically studied 3D wave packets without experimental realizations. Some examples of such wave packets are quasi-nonspreading wave packets which are localized 3D wave packets with very complicated structures [48], abruptly autofocusing 3D wave packets which can focus to enhance the intensity by 4 orders of magnitude by the intricate wave packet profiles, ellipsoidal wave packets which can be the ideal source for the free electron lasers, etc. Again, there are no clear synthesis methods for such wave packets due to complex spatiotemporally coupled structures.

To achieve such wave packets, it is necessary to develop a method to control the optical wave packets in amplitude, phase, and even polarization spatiotemporally in a three-dimensional fashion. In parallel, it is crucial to develop techniques to precisely measure the 3D profiles of optical wave packets. While some 3D intensity measurements are available in various techniques, precise 3D phase and polarization measurements are strongly desired to diagnose such sophisticated wave packets. The 3D profile measurement is desired even on a very small scale such as a tight focus of an optical wave packet.

Such complicated spatiotemporally coupled optical wave packets demand the control of many parameters three-dimensionally. The Fourier synthesis is the viable candidate to achieve such controls (figure 7). By adjusting the amplitude, phase, and polarization of light in the pulse shaper setup [49], the Fourier synthesized wave packet can be controlled in 2D spatiotemporal amplitude, phase, and polarization. Even though the Fourier synthesis is a promising candidate to generate a variety of spatiotemporally coupled wave packets, it can control only up to two-dimensional ST. To accomplish the full 3D control, an additional dimension control is required. This extra dimension control can be provided by an extra pulse shaper setup or a beam shaping which will be a bit simpler than a series of pulse shapers.

An important beam shaping skill to add an additional dimension control is the optical conformal mapping. The optical conformal mapping is the technique to convert the beam shape in some coordinates into a more useful coordinate system. For example, forming a symmetric beam in the cylindrical coordinates is sensible since the diffraction effect is in the radial direction of the cylindrical coordinates. In fact, such conformal mapping has been successfully implemented already in various optics research. For example, for the toroidal optical vortex, an elongated spatiotemporal vortex line in the Cartesian coordinates was mapped into the cylindrical coordinates to form a ring of the phase singularity [46]. So far, most of the optical conformal mappings are from the Cartesian to the cylindrical or vice versa. However, other types of conformal mappings can be explored for some exotic optical wave packets.

Another candidate to achieve the spatiotemporally coupled 3D wave packet is the metasurfaces/metamaterials. Since the metasurface can control the amplitude, phase, and polarization of light by the design, it is strongly believed that the metasurfaces are suitable to generate some interesting wave packets. It has been already proposed that the spatiotemporal vortex can be generated by a photonic crystal slab [50].

Developing the 3D measurement capability is essential to diagnosing the wave packet profile. In fact, there are a variety of 3D profile measurement techniques available (see chapter 12). However, the 3D profile measurement for a very small scale in the order of the wavelength is not available up to date. It is strongly believed that such small-scale measurement capability will enhance the understanding and applications of 3D wave packets.

Due to its exotic spatiotemporal structures with a transverse OAM, the interest in STOVs has risen lately. While future applications of spatiotemporal vortices are anticipated, much more complicated wave packets are desired for future research. To attain such complicated optical wave packets, it is necessary to develop reliable methods to control amplitude, phase, and polarization in 3D. In this roadmap article, some feasible 3D wave packet control strategies such as Fourier synthesis, conformal mappings, and metamaterials have been briefly discussed.

The 3D optical wave packet measurement is indispensable in studying spatiotemporal optical wave packets. Besides available 3D measurement techniques, the 3D measurement in the wavelength scale will be a significant accomplishment to widen the understanding and the applications of the spatiotemporal optical wave packets.

¹Theoretical Quantum Physics Laboratory, RIKEN Cluster for Pioneering Research, Wako-shi, Saitama 351-0198, Japan

²Centre of Excellence ENSEMBLE3 Sp. z o.o., 01-919 Warsaw, Poland

³Donostia International Physics Center (DIPC), Donostia-San Sebastián 20018, Spain

Monochromatic vortex beams carrying an intrinsic OAM along their propagation direction play significant role in modern optics and other areas of wave physics, both classical and quantum [51–54]. Such beams have found numerous applications in optical manipulations, imaging, microscopy, classical and quantum communications, etc. Vortex beams have a circularly-symmetric intensity distribution in the transverse $\left( {x,y} \right)$-plane and a screw dislocation of the phase front (vortex) on the beam $z$-axis. The intrinsic OAM of a vortex beam is ${L_z} = \ell$ per photon in the $\hbar = 1$ units, where $\ell$ is the integer TC of the vortex.

Recently, there was a great theoretical [55–57] and experimental [41, 45, 50, 58, 59] progress in studies of spatiotemporal vortex pulses (STVPs) carrying an intrinsic OAM orthogonal (or, generally, tilted) to their propagation direction. One can anticipate that such pulses will provide new geometries and functionalities to vortex states, extending their applications to the ST domain and ultrafast processes. STVPs have an elliptical intensity distribution in the propagation $\left( {x,z} \right)$-plane and a spatiotemporal vortex formed by the edge (or, generally, mixed edge-screw) phase dislocation in the pulse center, figure 9. Such vortex propagates with the pulse and is actually located in the center of the $\left( {x,z - vt} \right)$-plane, where $v$ is the velocity of the pulse propagation.

Although spatiotemporal vortices are also characterized by the integer TC $\ell$, calculation of the OAM carried by STVPs is rather nontrivial. On the one hand, one can expect the intrinsic OAM ${L_y} = \ell$ per photon in a STVP with a circular intensity profile [41, 55, 56]; generalizing this expression to an elliptical profile yields ${L_y} = \ell \frac{{\gamma + {\gamma ^{ - 1}}}}{2}$ [55, 56], where $\gamma$ is the ratio of the ellipse semiaxes along the $z$ and $x$ axes. On the other hand, recent calculations of the OAM of an optical STVP [57] resulted in ${L_y} = \ell \frac{\gamma }{2}$, which yields half-integer OAM in circular pulses. This result is in sharp contrast to the previous three-decade-long experience with monochromatic vortex beams [51–54] (for non-integer monochromatic vortex beams see [60]).

To understand the origin of this controversy, we consider paraxial linearly-polarized optical pulses, where one can neglect spin and spin-orbit interaction effects. The normalized (per photon) energy, momentum, and OAM of such pulses can be written as

$$\begin{align}W = \frac{1}{{2N}}\mathop \int \nolimits \left( {{{\mathbf{E}}^2} + {{\mathbf{H}}^2}} \right){\text{d}^3}{\mathbf{r}} = \frac{{\mathop \smallint \nolimits |\widetilde {\mathbf{E}}{|^2}{\text{d}^3}{\mathbf{k}}}}{{\mathop \smallint \nolimits {\omega ^{ - 1}}|\widetilde {\mathbf{E}}{|^2}{\text{d}^3}{\mathbf{k}}}} \simeq {\omega _0}\,,\end{align} \tag{ 1 }$$

$$\begin{align}{\mathbf{P}} = \frac{1}{N}\mathop \int \nolimits \left( {{\mathbf{E}} \times {\mathbf{H}}} \right){\text{d}^3}{\mathbf{r}} = \frac{{\mathop \smallint \nolimits {\omega ^{ - 1}}{\mathbf{k}}|\widetilde {\mathbf{E}}{|^2}{\text{d}^3}{\mathbf{k}}}}{{\mathop \smallint \nolimits {\omega ^{ - 1}}|\widetilde {\mathbf{E}}{|^2}{\text{d}^3}{\mathbf{k}}}} \simeq {{\mathbf{k}}_0},\end{align} \tag{ 2 }$$

$$\begin{align}{\mathbf{L}} \simeq \frac{1}{N}\mathop \int \nolimits {\mathbf{r}} \times \left( {{\mathbf{E}} \times {\mathbf{H}}} \right){\text{d}^3}{\mathbf{r}} \simeq \frac{{\mathop \smallint \nolimits {\omega ^{ - 1}}{{\widetilde {\mathbf{E}}}^*}\left( { - i{\mathbf{k}} \times {\nabla _{\mathbf{k}}}} \right)\widetilde {\mathbf{E}}\,{\text{d}^3}{\mathbf{k}}}}{{\mathop \smallint \nolimits {\omega ^{ - 1}}|\widetilde {\mathbf{E}}{|^2}{\text{d}^3}{\mathbf{k}}}}\,.\end{align} \tag{ 3 }$$

Here ${\mathbf{E}}\left( {{\mathbf{r}},t} \right)$ and ${\mathbf{H}}\left( {{\mathbf{r}},t} \right)$ are the real-valued electric and magnetic fields, $\widetilde {\mathbf{E}}\left( {\mathbf{k}} \right){e^{ - i\omega \left( {\mathbf{k}} \right)t}}$ and $\widetilde {\mathbf{H}}\left( {\mathbf{k}} \right){e^{ - i\omega \left( {\mathbf{k}} \right)t}}$ are their complex plane-wave Fourier amplitudes related as $\widetilde {\mathbf{H}} = {\omega ^{ - 1}}{\mathbf{k}} \times \widetilde {\mathbf{E}}$, $\omega \left( {\mathbf{k}} \right) = k$, we use the $c = 1$ units, $N \propto \mathop \smallint \nolimits {\omega ^{ - 1}}|\widetilde {\mathbf{E}}{|^2}{d^3}{\mathbf{k}}$ is the number of photons, and ${\omega _0} = \omega \left( {{{\mathbf{k}}_0}} \right)$ and ${{\mathbf{k}}_0}$ are the central frequency and the wavevector of the pulse. Equations (1)–(3) have the form of the normalized expectation values of the canonical energy, $\omega$, momentum, $\widehat {\mathbf{p}} = {\mathbf{k}}$, and OAM $\widehat {\mathbf{L}} = \widehat {\mathbf{r}} \times \widehat {\mathbf{p}} = - i\left( {{\mathbf{k}} \times {\nabla _{\mathbf{k}}}} \right)$, operators with the 'photon wavefunction' $\widetilde {\mathbf{E}}\left( {\mathbf{k}} \right)/\sqrt {\omega \left( {\mathbf{k}} \right)}$ in the momentum representation. The fact that the photon wavefunction is well-defined only in ${\mathbf{k}}$-space is well known [61, 62] (the photon probability density is ill-defined due to the nonlocality of the ${\omega ^{ - 1}}$ operator in real space), and the corresponding ${\omega ^{ - 1}}$factors in ${\mathbf{k}}$-space integrals in (1)–(3) are crucial for our consideration.

For simplicity, we consider Bessel-type STVPs, with the plane-wave spectrum lying on an ellipse in ${\mathbf{k}}$-space, figure 9(a), and with the linear $y$-polarization of the electric-field bearing a vortex of the TC $\ell$[56]:

$$\begin{align}{k_z} = {k_0} + {{\Delta }}\,{\text{cos}}\phi ,\;\;{k_x} = \gamma {{\Delta }}\,{\text{sin}}\phi ,\;\;{\tilde E_y} \propto {\text{exp}}\left( {i\ell \phi } \right).\end{align} \tag{ 4 }$$

Zoom In Zoom Out Reset image size

Here ${{\Delta }} \ll {k_0}$ and $\gamma {{\Delta }} \ll {k_0}$ are the ellipse semiaxes along the $z$ and $x$ directions, respectively, and $\phi$ is the azimuthal-angle parameter shown in figure 9(a). Substituting the field (4) into equation (3) and using relations

$$\begin{align*} & \frac{\partial }{{\partial {k_x}}} = \frac{{{\text{cos}}\phi }}{{\gamma {{\Delta }}}}\frac{\partial }{{\partial \phi }},\;\;\frac{\partial }{{\partial {k_z}}} = - \frac{{{\text{sin}}\phi }}{{{\Delta }}}\frac{\partial }{{\partial \phi }},\nonumber\\ & {\omega ^{ - 1}} \simeq k_0^{ - 1}\left( {1 - \frac{{{{\Delta }}\,{\text{cos}}\phi }}{{{k_0}}}} \right),\end{align*}$$

we calculate the $y$-component of the OAM of the STVP:

$$\begin{align}{L_y} \simeq \ell \frac{\gamma }{2}\,.\end{align} \tag{ 5 }$$

This unusual value agrees with the result of [57].

However, expression (5) is the total OAM calculated with respect to the chosen coordinate origin, and it does not mean the intrinsic OAM carrying by the STVP. In the case of monochromatic vortex beams, the well-defined intrinsic OAM ${L_z} = \ell$ is invariant with respect to parallel translations of the z-axis [53, 63]. In the case of STVPs, a parallel translation of the $y$-axis along the $x$-direction by a distance $a$ transforms the OAM (5) as ${L_y} \to {L_y} + a{P_z} \simeq {L_y} + {k_0}a$.

Following the prescription of nonrelativistic classical mechanics, the intrinsic and extrinsic parts of the OAM of a complex object can be separated using the position of the mass centroid of this object, ${\mathbf{R}}$, and its total momentum ${\mathbf{P}}$:

$$\begin{align}{{\mathbf{L}}^{{\text{ext}}}} = {\mathbf{R}} \times {\mathbf{P}},\;\;{{\mathbf{L}}^{{\text{int}}}} = {\mathbf{L}} - {{\mathbf{L}}^{{\text{ext}}}}\,.\end{align} \tag{ 6 }$$

Using this definition, the intrinsic OAM is invariant under translations of the origin, ${\mathbf{r}} \to {\mathbf{r}} + {\mathbf{a}}$: ${{\mathbf{L}}^{{\text{int}}}} \to {{\mathbf{L}}^{{\text{int}}}}$, while the extrinsic OAM is transformed according to the parallel-axis theorem: ${{\mathbf{L}}^{{\text{ext}}}} \to {{\mathbf{L}}^{{\text{ext}}}} + {\mathbf{a}} \times {\mathbf{P}}$. The separation (6) works well for monochromatic optical beams [53, 64], because the beam centroid and momentum are well defined quantities.

However, the centroid of a non-monochromatic pulse is a subtle notion, which can be defined in different ways. In particular, despite the photon probability density is ill-defined in real space, the photon probability centroid is well defined as the expectation value of the position operator $\widehat {\mathbf{r}} = i{\nabla _{\mathbf{k}}}$, similarly to equations (1)–(3) [65, 66]:

$$\begin{align}{\mathbf{R}} = \frac{{\mathop \smallint \nolimits {\omega ^{ - 1}}{{\widetilde {\mathbf{E}}}^*}{e^{i\omega t}}\left( {i{\nabla _{\mathbf{k}}}} \right)\widetilde {\mathbf{E}}{e^{ - i\omega t}}{\text{d}^3}{\mathbf{k}}}}{{\mathop \smallint \nolimits {\omega ^{ - 1}}|\widetilde {\mathbf{E}}{|^2}{\text{d}^3}{\mathbf{k}}}}\,.\end{align} \tag{ 7 }$$

(Here the ${e^{ - i\omega t}}$ factors in the Fourier amplitude are responsible for the propagation of the centroid with the group velocity along the $z$-axis.) Substituting here the STVP field (4), we find that the photon centroid experiences a vortex-induced shift along the $x$-axis, figure 9(b):

$$\begin{align}X \simeq \frac{\ell }{{2\gamma {k_0}}}\,.\end{align} \tag{ 8 }$$

From equations (5), (6) and (8), we obtain the intrinsic OAM of the STVP:

$$\begin{align}L_y^{{\text{int}}} \simeq {L_y} + {k_0}X \simeq \ell \frac{{\gamma + {\gamma ^{ - 1}}}}{2}\,,\end{align} \tag{ 9 }$$

which agrees with the OAM obtained in [55, 56].

Alternatively, one can define the intrinsic OAM with respect to the energy centroid ${{\mathbf{R}}_E}$, which is defined similarly to equation (7) but without ${\omega ^{ - 1}}$ factors in the integrands. The energy centroid is not shifted for the STVP under consideration, ${X_E} = 0$, and the corresponding intrinsic OAM coincides with (5): $L_y^{{\text{int}}^{^{\prime}}} \simeq {L_y} + {k_0}{X_E} = {L_y}$.

The shift of the probability centroid (8) originates from the fact that the pulse spectrum (4) consists of plane waves with equal amplitudes but with different densities of photons, $\rho \propto {\omega ^{ - 1}}|\widetilde {\mathbf{E}}{|^2}$ (well-defined for plane waves). Considering a modified STVP with ${\tilde E_y} \propto \sqrt \omega \,{\text{exp}}\left( {i\ell \phi } \right)$, having the same density of photons in each plane wave in the spectrum, figure 9(c), we obtain: $X = 0$, ${L_y} = L_y^{{\text{int}}}$ given by (9), but ${X_E} \simeq - \frac{\ell }{{2\gamma {k_0}}}$ and $L_y^{{\text{int}}^{^{\prime}}} \simeq {L_y} + {k_0}{X_E}$ given by (5).

Thus, the intrinsic OAM is highly sensitive to the definition of the pulse center, whereas position of this center is highly sensitive to the choice of plane-wave amplitudes in the pulse spectrum. This stems from the fact that the centroid of a relativistic extended body is not a uniquely defined notion [67]. It depends on the reference frame and on whether we calculate the centroid of relativistic masses/energy (${{\mathbf{R}}_E}$) or that of the rest masses (${\mathbf{R}}$) [65, 66]. In the nonrelativistic case, e.g. for STVPs in the Schrödinger equation, both definitions coincide yielding the well-defined intrinsic OAM (9). In the general relativistic case, including optical STVPs, it is even impossible to construct a circularly-symmetric object with uniquely defined intrinsic OAM. Indeed, in the above examples of STVPs, the probability and energy centroids are always mutually shifted along the $x$-axis, and, hence, at least one of them is shifted with respect to the phase singularity (vortex center). (Moreover, the singularity in the field ${\mathbf{E}}\left( {{\mathbf{r}},t} \right)$ does not generally coincide with, e.g. singularities in the vector-potential field ${\mathbf{A}}\left( {{\mathbf{r}},t} \right)$, i.e. the singularity is not a uniquely defined single point anymore.) This situation is in sharp contrast to symmetric monochromatic vortex beams, where all these centers coincide. A great advantage of defining the intrinsic OAM with respect to the probability centroid ${\mathbf{R}}$, equations (6) and (7), as suggested in [55, 65, 66], is that it yields equation (9), universal for both spatial and spatiotemporal vortices (i.e. longitudinal and transverse OAM), in both the relativistic and nonrelativistic cases.

The above consideration shows that defining the intrinsic OAM of optical STVPs is a subtle and tricky problem. Perhaps it does not have a unique fundamental solution and is a matter of theoretical convention. This requires further investigation. Notably, acoustic STVPs for sound in fluids/gases have the same peculiarities. This problem can also be considered within relativistic wave equations for massive quantum particles.

It would be very important to find suitable experimental methods for measurements of the OAM of optical or other STVPs. Interaction with atoms and STVP-induced atomic transitions could be one of such possibilities. Even if the experimental realization is challenging, theoretical calculations of light-matter interactions with STVPs could provide valuable data.

We have analyzed the OAM properties of optical STVPs. Our results elucidate the origin of the recent controversy [55–57] and suggests a convenient definition of the intrinsic OAM. This results in a universal expression (9) [55, 56] consistent with the longitudinal OAM of monochromatic vortex beams [51–54] and valid for any types of waves, both relativistic and nonrelativistic.

¹JILA and Department of Physics, University of Colorado and NIST, Boulder, CO 80309, United States of America

²Grupo de Investigación en Aplicaciones del Láser y Fotónica, Departamento de Física Aplicada, Universidad de Salamanca, Salamanca E-37008, Spain

In recent years, research on ultrashort, especially femtosecond and attosecond pulses with complex spatiotemporal shapes has focused on generating such light beams with topological phase structures. Building upon the conventional OAM beams discovered three decades ago, as introduced in prior chapter. Nontrivial phases carrying singularities that are time-varying or are defined in ST can now be mapped onto UFP. Moreover, the light science underlying the extreme nonlinear process of high harmonic generation (HHG) makes it possible to up-convert structured light from the IR into the extreme-ultraviolet (EUV) and soft x-ray (SXR) spectral regions. During the last decade, the relevant up-conversion rules for angular momenta have been established, opening up an exciting array of possibilities for imparting spin angular momentum (SAM) and OAM into short wavelength light, with femtosecond to attosecond pulse durations.

In the simplest case, a linearly polarized IR vortex beam with OAM l₁ is coherently up-converted into high-frequency (short wavelength) harmonic beams with charge l_q = ql₁ , where q is the harmonic order [68]. By harnessing this simple law and SAM conservation, new and powerful schemes are now available for tailoring EUV/SXR light. For example, by driving HHG with a necklace-shaped beam derived from the interference of two pulses with different OAM, HHG combs with tunable spectral spacing can be produced [69]. It is also possible to create attosecond pulse trains with controllable polarization states from pulse to pulse [70], harmonic vector-vortex beams [71], or circularly polarized harmonic vortices where the different polarization states have different divergences [72]. Very recently, a new property of light was uncovered via HHG—manifested as a time-varying OAM, that is changing during the pulse (figure 10) [73].

Zoom In Zoom Out Reset image size

In addition, UFP with controllable spatiotemporal OAM (ST-OAM) have been generated recently, as introduced in prior two chapters. Such transverse, spatiotemporal optical vortices (also known as STOV) present topological structures residing in both space and time. Recent experiments demonstrated that such ST-OAM follows (transverse) OAM conservation rules through nonlinear up-conversions (figure 11) [42]. For instance, through a second-harmonic generation process, the ST TC l_q ^st of the fundamental field is doubled along with the optical frequency. Thus, l_q ^st = ql_q ^st, where q = 2 for second-harmonic, reflecting the nature of ST-OAM conservation and following a general ST-OAM nonlinear scaling rule that resembles that of the (spatial) TC. Indeed, spatiotemporal phase singularities in ST-OAM can be interpreted as ST TCs carrying transverse OAM—the term coined in analogy to conventional (longitudinal) OAM.

Zoom In Zoom Out Reset image size

The generation and application of short wavelength ultrashort vortex pulses with controllable phase, polarization, and spectral properties, or with time-dependent OAM or ST OAM, is still in its infancy. For example, a full understanding of the nonlinear conversion and conservation rules of ST-OAM beams still remains to be achieved. While time-dependent OAM or self-torque has already been generated in the EUV using OAM-driven HHG, up-conversion of ST-OAM beams to high-order harmonics has only been proposed theoretically, as introduced in prior chapter, and thus remains to be experimentally validated. In addition, the exploitation of crystal symmetries through HHG in solids may open up new avenues for the generation of ultrashort pulse and short wavelength spatiotemporal OAM beams.

One of the most prominent experimental challenges in ultrashort pulse structured light generation is their extension to the SXR region. This capability would have several benefits: SXR photon energies span the core absorption edges of many magnetic and chiral materials; SXR light penetrates relatively thick samples; and the broad SXR HHG that are naturally generated can support pulse durations from femtosecond to attoseconds. To achieve this, further developments in the technology of structured mid-IR driving lasers is required, which have been shown to efficiently phase match broadband SXR HHG.

To enable applications, the generation of topologically structured ultrashort pulses at shorter wavelengths must be accompanied by accurate and fast characterization techniques. While complete characterization techniques are being developed in the visible-to-IR spectral regions, see chapters 13 and 14, their extension into the EUV/SXR remains very challenging. To date, wavefront sensor measurements [71], ptychography [74] and spectral interferometry [75] techniques present some of the most promising routes. Finally, we note that we have only discussed scalar beams (linearly and circularly polarized beams). Structured vector beams, which can possess space-varying and/or time-varying electric field directions [71], provide additional control knobs for tuning light-matter interactions. For instance, similar to the ST-OAM beams, whose phase structure resides in space and time, light beams can present electric field structures (or polarization states) residing in space and time, forming beams with non-transverse fields ($\nabla \cdot\overset{{\scriptscriptstyle\rightharpoonup}} {E} \ne 0$). The nonlinear up-conversion, TC conservation, and spin–orbit coupling of such non-paraxial beams are largely elusive.

A grand challenge ahead includes the development of a complete understanding of light-matter interactions involving both time-dependent and spatiotemporal OAM. Our understanding of fundamental excitations when a matter is perturbed by such spatiotemporally structured light is still in its early stage. For instance, recent experimental findings show that chiral and/or topological magnetic materials will respond to spatial OAM due to the so-called OAM dichroism (also termed helicoidal dichroism) in the EUV and x-ray regimes [76]. A better understanding of these interactions is critical for potential applications for sensing and controlling magnetic, topological, and quantum excitations of materials and for manipulating molecules and nanostructures on their natural time and length scales. Gaining more insights into such interactions will also help to develop new applications in metrology, sensing, and imaging that could benefit from tailoring ultrashort visible and IR pulses and their up-converted light into EUV, x-rays, and other pulsed particle beams such as electrons (see chapter 9) in both space and time.

In the visible region of the spectrum, OAM beams have enabled applications in super-resolution imaging, optical communications, quantum optics, and microparticle manipulation. Many of these applications show promise for translation to short wavelength OAM beams. In the case of visible spatiotemporal OAM beams, the demonstration of an optical spatiotemporal differentiator is expected, see chapter 15. Spatiotemporal structured short wavelength OAM beams have promise as extraordinary tools for laser-matter manipulation on attosecond time and nanometer spatial scales.

CREOL, The College of Optics & Photonics, University of Central Florida, Orlando, FL, United States of America

Structuring pulsed optical fields jointly in space and time has been a long-standing goal in optics [77]. Spatiotemporal structuring can yield a host of useful characteristics, such as propagation invariance, whereby the spatiotemporally structured pulsed field propagates without diffraction or dispersion in linear media. This was first recognized by Brittingham in 1983 [78] who introduced the focus-wave mode (FWM), which propagates in free space invariantly at a group velocity $c$. To date the FWM has not been convincingly demonstrated. Subsequently, X-waves were introduced in ultrasonics in 1992 and then realized in optics in 1997 by Saari and Reivelt [79]. The X-wave is another spatiotemporally structured propagation-invariant wave packet whose group velocity however is superluminal. Nevertheless, both the FWMs X-waves require extremely large bandwidths and numerical apertures to deviate in any observable way from a conventional wave packet that is separable with respect to its spatial and temporal DoF [22].

This state of affairs changed about 6 years ago when our group at CREOL investigated a new class of propagation-invariant spatiotemporally structured pulsed beams that we have called 'space-time wave packets' (STWPs) [80, 81]. In contrast to previous examples, STWPs can be produced with narrow spectra and small numerical apertures as a consequence of non-differentiable angular dispersion (AD) underlying the field [82, 83]; they have demonstrated diffraction-free behavior over extended distances; their group velocity can be tuned over an unprecedented span from subluminal to superluminal, and negative values (subluminal group velocity—below the speed of light in the corresponding medium, superluminal—above it); and display a host of intriguing and useful characteristics [22]. This has led to rapid growth in the study of spatio-temporally structured optical fields as evinced in this Roadmap article.

These recent demonstrations of STWPs have a simplified field structure that is localized along one transverse dimension only; i.e. the STWP takes the form of a light-sheet. Interestingly, there are no monochromatic diffraction-free beams in one transverse dimension (except for the trivial cases of plane or cosine waves, which are not localized [22]). Indeed, a Bessel light sheet diffracts in contrast to its two-dimensional counterpart. The one exception of course is the monochromatic Airy beam in one dimension that is diffraction-free but travels along a parabolic trajectory. In contrast, because STWPs are pulsed rather than monochromatic fields, they do not suffer from this restriction [22], and thus represent the only truly propagation-invariant fields in one transverse dimension (diffraction-free and propagate along a straight line). Nevertheless, many applications in nonlinear optics and free-space propagation require field localization along both transverse dimensions—an optical needle rather than a light sheet. It should be noted that through this Section we refer to monochromatic optical fields of finite spatial extend as 'beam'. We then use the term 'wave packet' to refer to a pulsed beam (finite spatial and temporal extend).

The challenge in synthesizing STWPs that are localized in all dimensions is the difficulty or producing non-differentiable AD in two dimensions. AD refers to optical fields in which each frequency $\omega$ travels at a different angle $\varphi \left( \omega \right)$. Diffractive and dispersive devices, such as gratings and prisms, introduce AD into collimated pulses. In general, $\varphi \left( \omega \right)$ is assumed to be differentiable; i.e. $\frac{{{\text{d}}\varphi }}{{{\text{d}}\omega }}$ is defined everywhere. This is a natural assumption that is never stated explicitly, and the conventional theory of AD is based on a Taylor expansion of $\varphi \left( \omega \right)$ that takes this for granted.

This is surprising because AD also undergirds STWPs; each $\omega$ travels at a different angle $\varphi \left( \omega \right)$. Nevertheless, the attributes of STWPs are in direct contradistinction to those predicted for conventional AD. Our recent work has revealed that the AD underlying STWPs is non-differentiable: $\frac{{{\text{d}}\varphi }}{{{\text{d}}\omega }}$ is not defined at some frequency ${\omega _{\text{o}}}$ [82, 83]. Such an AD-profile can be produced by the 'universal AD synthesizer' developed in our previous work [22], but only along one transverse spatial dimension. To produce an STWP that is localized along all dimensions, the radial wave number ${k_r}$ must be related to the frequency $\omega$ in the paraxial regime via the relationship: $\frac{{{\Omega }}}{{{\omega _{\text{o}}}}} = \frac{{k_r^2}}{{2k_o^2\left( {1 - \cot \theta } \right)}}$, where ${{\Omega }} = {{\omega }} - {\omega _{\text{o}}}$, ${\omega _{\text{o}}}$ is the non-differentiable frequency, ${k_{\text{o}}} = {\omega _{\text{o}}}/c$, and $\theta$ is referred to as the spectral tilt angle. A useful visualization tool to understand the structure of STWPs is their spectral support on the light-cone surface $k_x^2 + k_y^2 + k_z^2 = {\left( {\frac{\omega }{c}} \right)^2}$. Restricted to azimuthally symmetric fields structures, the AD profile for an STWP is associated with the spectral support in the form of a conic section at the intersection of the light-cone $k_r^2 + k_z^2 = {\left( {\frac{\omega }{c}} \right)^2}$ with a plane ${{\Omega }} = \left( {{k_z} - {k_{\text{o}}}} \right)\tilde v$, which represents a propagation-invariant pulsed field transported rigidly in free space at a group velocity $\tilde v = c\tan \theta$; see figures 12(a) and (b). The resulting field structure is X-shaped in any plane passing through the meridional propagation axis as shown in figure 12(c).

Zoom In Zoom Out Reset image size

How can such a field structure be synthesized? We have recently demonstrated that the experimental strategy outlined in figure 12(d) can achieve this task [84]: spectral analysis via a volume Bragg grating, 'reshuffling' the spatially resolved wavelengths in a desired order, followed by a log-polar transformation to arrange the wavelengths in an annulus, and finally a Fourier-transforming lens. Using this setup, we have produced the first propagation-invariant STWP that is localized in all dimensions [84].

Although interest in spatiotemporally structured wave packets was initially driven by the desire for producing propagation-invariant wave packets, a host of other useful properties can also be realized. Examples include acceleration of the wave packet along the propagation axis, dispersion-free propagation in dispersive media (whether normal or anomalous), long-distance propagation for free-space communications, anomalous refraction (where the group velocity can increase or remain unchanged after traversing a planar interface to a higher-index medium), isochronous propagation (taking the same group delay to traverse a slab of different thickness by changing the angle of incidence), blind synchronization of remote optical clocks (without knowing the distance of multiple clocks from the same source), among other possibilities [22]. The first tests of these applications have been recently performed with STWPs in the form of light sheets. It is now possible to extend these applications to the typically more desirable form of a beam localized in both transverse dimensions.

Localization of STWPs along both transverse dimensions allows for exploring more complex spatiotemporally structures fields. For example, OAM can now be included in the STWP, which was not possible in the case of light sheets. The setup shown in figure 1(d) is indeed capable of introducing OAM into the STWP, and time-resolved measurements performed via the interferometric configuration depicted in figure 12(e) are plotted in figures 13(a)–(d). Off-axis holography helps reconstruct the amplitude and phase of the field at multiple axial planes, and linear interferometry enables resolving the wave packet in time [84]. Figures 13(a) and (b) corresponds to an STWP endowed with no OAM, whereas figures 13(c) and (d) corresponds to an STWP endowed with OAM of order 1. To the best of our knowledge, this is the first propagation-invariant pulsed OAM wave packet. Furthermore, vector beam structures [85] can be realized as shown in figures 13(e) and (f) by placing a vortex wave plate in the path of the spatiotemporal spectrum as shown in figure 12(e). Here, the polarization structure is distributed over the 3D structure of the STWP, and off-axis holography helps reconstructs this polarization distribution in space and time.

Zoom In Zoom Out Reset image size

A critical avenue to be explored is the development of compact and stable arrangements for synthesizing STWPs. Two possible routes are adapting metasurfaces or volumetric Bragg gratings for this purpose; much more work needs to be done along these lines.

The recent developments in synthesizing STWPs have opened new vistas for the study of spatiotemporally structured optical fields [22]. This has culminated most recently with the synthesis of STWPs that are localized in all dimensions and are endowed with OAM and/or vector beam polarization structure. In many ways, this field of study is still in its very early stages of development, and it is anticipated that this research enterprise will expand dramatically in the near future. In addition to ongoing investigations of their basic physics, the vast array of unique attributes of STWPs provides ample opportunities for applications in free-space optical communications, metrology, nonlinear optics, and biomedical imaging. Advancement in the photonic technologies that reduce the footprint of spatiotemporal synthesis systems in particular will be crucial for the success of these applications.

School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore 639798, Singapore

Free electron-driven light-matter interactions play a central role in modern science and technology. Free electron radiation has attracted much interest by revealing unprecedented physics in processes like Cherenkov radiation, Smith–Purcell radiation, superradiance, inverse Compton scattering, etc [86, 87]. At the same time, free electrons have proven an invaluable tool for device characterization and diagnostics through mechanisms like cathodoluminescence and photon-induced near-field electron microscopy [88, 89]. Free electron radiation plays an indispensable role at extreme wavelengths such as the x-ray and gamma ray regimes of the electromagnetic spectrum [90]. It is noteworthy that practically all x-ray sources today—from commercial x-ray tubes to stadium-sized synchrotrons and kilometers-long free electron lasers—use free electrons as their basic operating principle. Therefore, the wealth of applications that depend on x-rays—including medical diagnostics, manufacturing inspection, security scanning, forensics, biological imaging and crystallography—is testament to the usefulness and importance of free electron science in the today's world.

There are fundamental reasons why free electrons have advantages over bound electrons (e.g. in atoms and materials) and other particles/quasi-particles in certain scenarios. Firstly, free electrons are readily accelerated to relatively high energies with just a direct current (DC) power supply. This is used in x-ray tubes to produce electron kinetic energies of several 10 s of keV or more, needed for collisional inner-shell ionization and the consequent generation of characteristic x-ray peaks. Secondly, the free electron has a continuum of energy states, unlike bound electrons which are restricted to discrete energy states. This allows free electrons to emit radiation at any frequency, not being confined to the differences between discrete energy levels. Thirdly, the free electron effectively functions as a highly nonlinear optical medium: a moving electron readily up-converts or down-converts photon frequencies via the Doppler effect. This is exploited in x-ray synchrotrons and free electron lasers, where relativistic free electrons up-convert the periodic magnetostatic field of undulators into x-ray photons via the relativistic Doppler effect.

The importance of shaped light can be seen through numerous advances in imaging, sensing, manufacturing, communications, and computation. For free electron-based radiation processes, the shaping of free electrons provides extra DoF to shape the emitted light or control light-matter interactions. In this article, we discuss the promises of free electron shaping for creating exotic electromagnetic modes and photon statistics (figure 14) that are typically inaccessible via traditional methods of light shaping, across the spectrum from microwaves to x-rays and gamma rays. It should be noted that the use of free electrons in light generation goes beyond synchrotrons and free electron lasers—and include more compact radiation schemes based on bremsstrahlung, undulator radiation, transition radiation etc. As such, our article is broadly written to encompass these various mechanisms.

Zoom In Zoom Out Reset image size

Spatiotemporal shaping of electromagnetic fields is readily achieved in the microwave to optical regimes through an arsenal of light shaping elements, including programmable SLMs, metasurfaces, waveguides, etc. Complex photonic circuits have also been realized for quantum shaping of optical light, which involves shaping of the photon's wavefunction (equivalently, its probability distribution) and of photon statistics. Although these light-shaping techniques have enjoyed rapid advances, many desired electromagnetic profiles and photon modes remain out of reach.

One example is entangled high-number Fock states, which have been predicted to improve quantum metrology, sensing and imaging. The creation of such states, however, remains elusive as there is no efficient, on-demand means to generate Fock states, especially for high photon numbers. As shown recently, free electrons can provide a versatile platform for customizing photon statistics, for instance, to create deterministic, high-number Fock states [91].

Another example is the challenge of tailoring and focusing extreme wavelength radiation such as x-rays. Due to the near-unity refractive indices of materials in the x-ray regime, large, defect-free surfaces and multilayer structures fabricated with ultra-high precision are often needed to steer and mold an x-ray beam. Furthermore, traditional x-ray optics requires x-rays that are substantially coherent over large areas, but such x-rays are only available from large facilities like synchrotrons. Due to these constraints, the question arises as to how to achieve the versatility and robustness of visible-light optics in the x-ray regime.

As we discuss in the next section, shaping free electrons could be key to resolving this challenge. Since free electrons are already an essential component in practically all x-ray sources, it stands to reason that manipulating the electrons themselves could be an effective means of controlling the emitted x-ray radiation [90]. This is in fact the principle behind free electron lasers, where an electron bunch interacts with its own emitted x-ray radiation, ultimately evolving into electron nano-bunches in a process known as self-amplified spontaneous emission [86]. These nano-bunches function like a nanoscale antenna array by emitting x-rays that constructively interfere, producing highly directional, ultra-bright x-rays. The foregoing, however, involves only the control of electrons as a collection of classical point charges. The fundamental question arises as to whether we can shape light by manipulating the wavefunction of a single electron emitter. In the next section, we discuss how this is indeed the case.

As nicely put by Feynman, an electron behaves ultimately as a point-like particle, regardless of its wavefunction. It is therefore valid to question whether shaping the wavefunction of a single free electron could possibly affect the emitted radiation from said free electron. One might expect that this is not the case, since the electron would be emitting as a point-like particle regardless of its wavefunction. Quantum theory, however, reveals surprisingly that shaping the wavefunction has a substantial impact on the emitted photons for both stimulated and spontaneous emission [92]. In particular, it has been theoretically shown using quantum electrodynamics that the free electron wavefunction can be shaped to make bremsstrahlung more directional and undulator radiation more monochromatic (figure 15) [92]. These theoretical findings remain to be experimentally demonstrated. The wealth of possible free electron waveshapes also opens new areas of exploration where the polarization, OAM and other properties of electron waves can be harnessed to realize unique and/or bespoke output radiation profiles and photon statistics, especially at extreme wavelengths like x-rays and gamma rays [90].

Zoom In Zoom Out Reset image size

It is also intriguing to consider the shaping of multi-electron pulses. Disruptive technologies remain to be discovered just by considering the classical picture of electrons as point charges, exemplified by a recent study where shaped electron bunches were used to suppress microbunching instability endemic in free electron lasers [93]. Even more fascinating is the question of whether and how we can combine the notion of classically shaping a distribution of point charges with that of shaping the quantum wavefunction of each individual electron. Some inroads have been made into this in a recent theoretical study where a pair of entangled input electrons were considered [94]. The study found that different free-electron Bell-states leads to substantially different spectra and spatial patterns of the emitted light, in a manner that cannot be accounted for by a classical mixed state. However, the practical question of how to shape each electron individually in a multi-electron pulse for experimental demonstration remains. A recent study created two-electron pulses from a nanotip to study their non-Poissonian nature using coincidence measurements [95], paving the way to further studies of multi-electron effects in electron pulses.

Finally, the inverse design problem remains to be tackled—to develop frameworks and algorithms for determining the wavefunction of input electron/s in a computationally efficient way based on the spatial, spectral and statistical properties of the desired output.

Free electrons play an indispensable role in today's photonics landscape because (1) free electrons can access extreme wavelengths like x-rays (a multi-billion-dollar industry) and gamma rays, which lie beyond the means of purely bound electron-based light sources, and (2) free electrons can realize radiation with spatial, spectral and statistical properties that are not possible or challenging to achieve by other means. The advent of new methods to shape free electrons provides new degrees of freedom with which to shape the emitted light. The shaping of free electrons can take place in two ways: control over the classical distribution of point charges, and the shaping of the electron's quantum wavefunction on a single-electron level. The nascent field of free electron waveshaping has already presented some remarkable ideas in light shaping, including shaping of bremsstrahlung and undulator radiation, and unprecedented control over the quantum statistics of output photons. Future research includes the study of exotic electron waveshapes, the physics of entangled free electrons and multi-electron pulses in light-matter interactions, and the inverse design problem. An improved understanding of the relationship between free electrons and the photons they emit should pave the way to greater robustness and versatility in next-generation light sources.

¹Department of Applied Physics, Stanford University, Stanford, CA 94305, United States of America

²Ginzton Laboratory and Department of Electrical Engineering, Stanford University, Stanford, CA 94305, United States of America

Light usually diffracts or disperses under propagation. However, there are unusual solutions to linear Maxwell's equations, called ST wave packets or light bullets [96], that are propagation-invariant. These wave packets do not suffer from dispersion or diffraction. They can also propagate with an arbitrary group velocity. Such wave packets have been extensively studied for potential applications including communications, bioimaging, lithography, and quantum key distribution.

The fundamental property of all ST wave packets is the characteristic 'space-time correlation' between their temporal and spatial frequencies: $\omega = {v_g}{k_z} + b$, where $\omega$ is the angular frequency, ${k_z}$ the wavevector component in the propagation direction, ${v_g}$ the group velocity, and $b$ a constant. Generating ST wave packets requires the accurate synthesis of the ST correlation. There is substantial progress to create the desired ST coupling for wide ranges of ${v_g}$ and $b$ [81] with the use of spatial light modulation and free-space optics. However, these setups are quite bulky. For further developments in this field, it is highly desirable to develop techniques to generate ST wave packets with a more compact set up.

In recent years, there are significant advances in nonlocal nanophotonics, which enable the control of light in the wavevector space. Such a control can be achieved by using either static periodic structure or dynamic spatiotemporal modulation. The former includes nonlocal metasurfaces [97] that exhibit wavevector-dependent transfer functions, which are in sharp contrast with conventional local metasurfaces characterized by space-dependent transfer functions. They enable important functionalities including optical differentiation [97, 98], squeezing free space [99], and generating optical pulses with tilted OAM [50]. The latter includes traveling wave modulators that can induce wavevector modulation in guided waves. They enable significant opportunities in integrated nonreciprocal photonic devices [100, 101].

Here we briefly review recent works which reveals a deep connection between the fields of ST wavepacket and nonlocal nanophotonics. The basic idea is that the desired ST correlation can be induced by nonlocal nanophotonics directly in the frequency-wavevector domain. This enables the capability of generating ST wave packets using compact nanophotonic devices.

Generating ST wave packets is a nontrivial task; it requires simultaneous and precise control of frequency and wavevector for a light pulse. Here, we highlight two important challenges:

• 1.  
  The generation of ST wave packets in free space using a compact optical device. The conventional approach to generating ST wave packets uses a technique combining spatial-beam modulation and ultrafast pulse shaping. Such a technique requires bulky components and complicated operations. For practical applications, it would be preferable to generate ST wave packets in free space with a simple and compact device.
• 2.  
  The in-situ generation of ST wave packets in a waveguide. Most existing works focus on ST wave packets propagating in free space or bulk media. Creating ST wave packets in a waveguide may lead to new opportunities in integrated photonics. For example, these wave packets can propagate at arbitrarily low group velocity but still possess broad spectral bandwidth. Previous proposals to generate such wave packets either first synthesized a wavepacket in free space and then coupled it into the waveguide or relied upon strong nonlinearity. For practical applications, it would be preferable to generate ST wave packets in a waveguide in-situ without the need for nonlinearity.

Here we highlight two recent advances in nonlocal nanophotonics that provide new approaches to addressing the challenges mentioned above.

• 1.  
  Generation of ST wave packets in free space using nonlocal metasurfaces. Guo et al [102] shows that an ultrathin nonlocal nanophotonic layer provides a compact and versatile platform to generate controllable 3D light bullets in free space. This approach consists of sending a Gaussian wave packet (figure 16(c)) into a single-layer periodic nanophotonic structure supporting guided resonance (figure 16(a)). By designing the band dispersion of the guided resonance (figure 16(b)), one can generate the required ST correlation to achieve a 3D light bullet as the output(figure 16(d)). This method offers straightforward control of the external DoF of the light bullet including the group velocity and the propagation distance, and the internal structures such as the SAM and OAM. Such controllable light bullets may inspire broader applications.
• 2.  
  Generation of ST wave packets in a waveguide using traveling wave modulation. Guo and Fan [103] shows that in a multimode waveguide, ST wave packets can be generated using indirect photonic transitions. In this approach (figure 17(a)), one first excites an eigenmode of the waveguide (figure 17(b, left)), and then transmit the light through a traveling wave modulator. With a suitable design, the modulator can convert such an excitation into a ST wave packet (figure 17(b, middle) and (b, right)). The group velocity of the wave packet equals the phase velocity of the traveling wave modulation and thus can be arbitrarily tuned. Such an approach can generate broadband propagation-invariant ST wave packets with tailored field profiles. This work reveals a connection between the study of time-varying media and ST optics and may enable integrated photonic applications of guided ST wave packets.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

In summary, nonlocal nanophotonics can be utilized to create nontrivial ST correlations. This provides simple methods to sculpt light pulses in the frequency-wavevector domain with high precision. Such a capability opens significant new opportunities in the emerging field of ST nanophotonics. We anticipate further development of compact nonlocal nanophotonic devices that can significantly advance the practical applications of ST wave packets.

¹Optoelectronics Research Centre & Centre for Photonic Metamaterials, University of Southampton, Southampton SO17 1BJ, United Kingdom

²Centre for Disruptive Photonic Technologies, School of Physical and Mathematical Sciences & The Photonics Institute, Nanyang Technological University, Singapore 637378, Singapore

Toroidal light pulses (TLPs), termed Flying or Focused Doughnuts, are few-cycle pulses of toroidal topology with ST non-separable structure and vector polarization (i.e. azimuthally or radially polarized) that were discovered by Hellwarth and Nouchi in 1996 (figure 18(a1)) [104]. Research interest in TLPs has recently surged owing to the emergence of toroidal electrodynamics [105], but also due to advances in ultrafast lasers and the wide availability of wavefront and pulse shaping devices (DMDs, SLMs, etc.). In fact, TLPs have only recently been generated experimentally for the first time [106]. Following the demonstration of their generation, a broader family of pulses has been introduced, termed supertoroidal pulses (STPs) [107] and non-diffracting supertoroidal pulses (ND-STPs) [108], of which the Flying Doughnut is the fundamental member.

Zoom In Zoom Out Reset image size

Toroidal and supertoroidal light pulses exhibit a unique combination of physical characteristics radically different from conventional light waves:

• 1)  
  Non-transverse electromagnetic field configuration: the magnetic (electric) field traces the body of a torus, while the electric (magnetic) field traces the torus surface, resulting in field components oriented along the direction of pulse propagation (figure 18(a2));
• 2)  
  Few-cycle pulse duration: for instance, fundamental TLPs exhibit single and 1 ½ cycle duration, switching from one to the other upon propagation due to a Gouy phase shift (figure 18(a1));
• 3)  
  Toroidal topology: Toroidal pulses are ideal probes for engaging toroidal and non-radiating anapole excitations in matter [105, 109];
• 4)  
  Skyrmion-like field topology: STPs exhibit topological structure similar to that observed in solid-state (e.g. magnetic skyrmions) (figure 18(a3)) [107];
• 5)  
  Self-similar patterns: STPs exhibit a complex, fractal-like singularity structure consisting of nested singular shells (figures 18(b1) and (b2)) [107];
• 6)  
  ST non-separability: the spatial dependence of STPs cannot be separated from the temporal one, which has significant effects on their propagation dynamics. For instance, in the fundamental TLPs, ST non-separablity leads to isodiffracting propagation [110, 111], whereas in STPs non-separability results in non-diffracting propagation over arbitrarily long distances (figure 18(c1)), reminiscent of vortex street flows in fluids (figures 18(c2) and (c3)) [108].
• 7)  
  ST superoscillations: STP fields can oscillate faster than the highest spatial and temporal frequency components of the pulses (figure 18(b3)).

To date, only the fundamental TLPs have been experimentally generated in the optical and THz part of the spectrum [106], while the other members of the family have only been studied theoretically. Moreover, the propagation dynamics and light-matter interactions of TLPs remain largely unexplored, in particular with respect to their unique ST non-separable and skyrmionic field structures.

The main challenges facing the field involve the experimental generation and characterization of said pulses. In particular, the generation part requires precise spatial control of amplitude, phase and polarization of the electric field over a broad frequency band. On the other hand, the characterization of such pulses also introduces a number of significant challenges. Similarly, to pulse generation, characterization will also require accurate retrieval of the electric field of the generated pulses as a function of space and time. Recent developments on techniques for the full 3D retrieval of the electromagnetic fields of few-cycle vector polarized pulses (e.g. [112, 113] and related techniques shown in chapter 13 and chapter 14 of this roadmap) can be expanded and applied to the more complex supertoroidal and ST superoscillatory pulses.

Key advances required to meet challenges involve the availability of ultrabroadband sources that can generate few- to single-cycle pulse. To date, such sources are routinely available only at lower frequencies, e.g. THz and microwave papers. Moreover, the ST nonseparability of TLPs and STPs raises intriguing questions regarding the presence of ST nonseparable excitations in matter and the relation of the latter to free-space propagating STPs. Toroidal pulses can also be employed as information and energy carriers, where information can be encoded in the complex, robust upon propagation, topological structure, while their non-diffracting nature allows energy transfer across arbitrary distances. Further, to date toroidal pulses have only been studied in the classical regime, while the quantum optics of TLPs remain completely unexplored. Finally, a systematic classification and exploration of the family of toroidal pulses is still under way and is expected to lead to new discoveries of spatiotemporal doughnuts pulses.

To summarize, supertoroidal pulses exhibit vector singularities of electromagnetic fields with intriguing topological properties, such as skyrmion-like magnetic field configurations and fractal-like distribution of singularities. Such topological features of STPs will be of interest to fundamental studies and spectroscopic applications. They could enable spectroscopies of localized ST non-separable excitations in matter, where high spatial and temporal resolutions are required simultaneously. Nonlinear light-matter interactions with toroidal pulses are of particular interest, where one can envisage applications as exotic as particle acceleration [114]. The quantum aspects of the interplay between non-separability and quantum coherence and entanglement constitute an interesting future research direction, while an important technological question is whether the supertoroidal pulses can be directly generated in specially designed laser cavities.

¹National Institute of Standards and Technology, Gaithersburg, MD 20899, United States of America

²Maryland NanoCenter, University of Maryland, College Park, MD 20742, United States of America

Developments in sculpting the spatial phase and amplitude distribution of light has led to numerous advances in areas ranging from imaging and lithography to classical and quantum communication. These advances are largely enabled by development of wavefront shaping tools such as liquid-crystal-based SLMs (LC-SLMs), digital micromirror devices (DMDs) and microfabricated phase-plates [115]. Recently, metasurfaces composed of deep-subwavelength nanostructures have demonstrated exceptional versatility in molding the spatial wavefront of light [116, 117]. The unique advantages offered by metasurfaces including multifunctional control of light at the nanoscale, ultrathin planar form-factor, ready design flexibility, and complementary metal-oxide semiconductor fabrication compatibility, make them an attractive platform for applications requiring portability, robustness, and scalability.

Similarly, controlling the temporal waveform of light on an ultrafast time scale, commonly referred to as pulse shaping, has had tremendous impact in the field of ultrafast optics. For example, manipulating the spectral phase and amplitude of the pulse has enabled applications such as chirped pulse amplification and optical arbitrary waveform generation, whereas controlling the polarization degree of freedom has provided access to ultrafast chiral light-matter interactions. Temporal shaping has traditionally been achieved either through manipulation of the pulse directly in the time domain, using, for example, chirped mirrors or acousto-optic modulators (AOMs), or alternatively, in the frequency domain by employing Fourier synthesis combined with LC-SLMs, DMDs or AOMs [118]. Recently, finely tailored pulse shaping operations were realized using a dielectric-metasurface based pulse shaper, where simultaneous and independent control of the phase and amplitude of the constituent frequency components of a near-IR femtosecond pulse over an ultrawide bandwidth was achieved [119].

ST wave packets where the phase, amplitude, and polarization of light in both space and time can be simultaneously and independently engineered have the potential to unveil new physics and enable new capabilities in ultrafast light-matter interactions. The past few years have witnessed numerous advances in spatiotemporally structuring optical fields in free-space or in waveguides by treating its spatial and temporal DoF either as separable or correlated variables. Complex ST wave packets exhibiting diffraction-free propagation, tunable group velocity, transverse OAM, self-torque, toroidal topology, flying focus, and predesigned three-dimensional trajectory, among others, have been demonstrated [22]. Having the ability to arbitrarily control all the fundamental DoF of an optical wave in space and in time would enable synthesis of novel classes of optical wave packets opening intriguing possibilities in both basic research and potential real-world applications.

Despite rapid progress in synthesizing ST wave packets, arbitrary control over all the spatiotemporal properties of the light field over a large bandwidth has proven challenging. This requirement generally translates to simultaneous and independent manipulation of a large group of coherent frequencies with ultrahigh spectral and spatial resolution. Generally, a Fourier transform pulse shaper, consisting of refractive (e.g. lens) and diffractive (e.g. grating) optics, is used to spatially separate the spectral components of an ultrafast optical pulse train at its Fourier plane, and can be engineered to provide a high spectral resolution. However, typically the comb line spacing, determined by the repetition rate of the laser, is on the order of tens of MHz to a few GHz, which is well below the resolvance of gratings. Moreover, high temporal resolution necessitates a large spectral bandwidth, demanding enormous number of units (composed of one or an array of pixels) on the modulation device, placed at the Fourier plane, to achieve parallel and independent manipulation of each spatially dispersed frequency component. At the same time, each pixel is ideally required to provide multifunctional control by being able to simultaneously manipulate the phase, amplitude, and polarization DoF of the corresponding frequency line. To ensure high modulation efficiency, these operations are expected to be implemented by pixels of subwavelength size and spacing. All of these are nontrivial requirements for spatiotemporal shaping with high spectral resolution, ideally achieving comb-line resolution over a large spectral bandwidth. Considering practical applications, miniaturization of ST wave packet synthesizer is also highly desirable. Conventional gratings, lenses, and modulation devices such as LC-SLMs and DMDs are bulky and post a limit on the footprint of the synthesizer. Also, such wave packet synthesizers, depending on the choice of the modulation device, are also often limited to operate in the visible to near- and mid- IR parts of the electromagnetic spectrum, leaving a gap for technologically important ultraviolet and terahertz wavelengths. Finally, high brightness light sources are cropping up worldwide, with ever-increasing interest in ST wave packet synthesizers that can operate at high input peak powers, a capability not supported by current modulation technologies due to their low damage threshold.

Significant progress in design, synthesis and characterization of ST wave packets exhibiting unique characteristics has been made over the last decade. However, each approach only addresses a subset of these challenges primarily because of limitations in the size, number and functionality at the pixel level of traditional approaches based on LC-SLMs or DMDs. By leveraging the multifunctional response at the nanoscale offered by a dielectric metasurfaces, a versatile approach for controlling the four-dimensional properties (temporal phase, amplitude, polarization, and spatial wavefront) of near-IR femtosecond pulses at will over an ultrawide bandwidth has recently been demonstrated [120]. Here, a single-layer metasurface device, consists of hundreds of modulation units (termed superpixels), each containing a two-dimensional (2D) array of deep-subwavelength birefringent Si nanopillars as modulation pixels. The device is designed to provide simultaneous and independent control over the temporal state of light (i.e. phase, amplitude, and polarization) via engineering the metasurface at the superpixel level, and the spatial wavefront evolution via engineering individual nanopillars within each superpixel (figure 19). Complex ST wave packets including a single femtosecond pulse carrying both temporal spin and OAM, a light-coil exhibiting a helical intensity distribution evolution, and a pulse with coherently multiplexed time-varying OAM orders, are experimentally synthesized. These ST wave packets have up until either been theoretically proposed or require high-harmonic nonlinear processes. The metasurface-based approach can be extended towards synthesizing other forms of complex ST wave packets via metasurface design engineering, or other wavelength regimes by simply choosing a compatible dielectric material with an appropriate bandgap. Furthermore, the dispersive and diffractive optics in the synthesizer built using the metasurface approach can enable realization of a compact ST wave packet synthesizer, and the use of dielectric materials lends the platform a high damage threshold.

Zoom In Zoom Out Reset image size

Ultimate realization of shaping at the resolution of individual comb line, technique referred to as line-by-line shaping, opens the prospect of optical arbitrary waveform generation but requires significantly expanding the spectral resolution of the pulse shaper. By spectrally dispersing the pulse into a 2D array using a cross-dispersion configuration leveraging combination of a virtually imaged phased array and a grating [121], combined with high-spatial resolution offered by metasurfaces offers one potential path towards achieving this goal. Continued developments in the field of metasurfaces, such as by incorporating multiple nanopillars within one pixel or the combined use of propagation phase and geometric phase [117], further provide more advanced multifunctional control over the fundamental properties of light. Moreover, the rapid progress in metasurface design strategies, for example, leveraging inverse design and machine learning, is also expected to provide novel capabilities not possible with traditional nanopillar library-based approaches [122]. Although a static metasurface response is sufficient for several applications, reconfigurability such as those offered by LC-SLMs is still favorable. While much of the development in the area of active metasurfaces using various transduction mechanisms such as electro-optical, nano-electro-mechanical, nonlinear or via phase change materials, are at an early-stage, we expect them to in time reach a level of maturity required to provide a full programmable control over the light properties [123]. Benefiting from the continued interest from both academia and industry, efforts in developing low-cost, large-scale nanofabrication of metasurfaces also show promise for scaling and commercial viability of metasurface-based ST wave packet synthesizers.

Arbitrary ST wave packet synthesis is an emerging research field offering the promise of enabling new applications via customized control over all the fundamental properties of light field. We have outlined some outstanding challenges and offered perspective on the requirements for a universal approach towards synthesis of arbitrary ST wave packets. These capabilities are expected to be further accelerated by continued developments in new theoretical models, metasurface design concepts, nanofabrication technologies, and light-field characterization. In the short term, we expect the versatile optical control offered by metasurfaces to be applied to other spatiotemporal light field concepts and applications discussed in this roadmap. In the long term, development of a large area, lithography friendly, fully reconfigurable modulator offering multidimensional control at the nanoscale is expected to have lasting impacts beyond the field of ultrafast optics.

School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, QLD 4072, Australia

Historically, it has been possible to create reconfigurable optical beams that are functions of space, F(x,y), using components such as SLMs, or reconfigurable functions of time, F(t) using components such as electro-optic modulators or spectral pulse shapers. By using these respective technologies together in series it is possible to create separable spatiotemporal beams of the form F(x,y)F(t), which have the same transverse spatial profile, F(x,y) for every point in time, F(t). A traditional spectral pulse shaper based on a 2D SLM can create 2D beams of the form F(x,t) [41, 124, 125], where one transverse spatial dimension, and one longitudinal time/frequency dimension can be controlled independently. In contrast with the last chapter of localized wave packet synthesis, here our concept of beamshaping means the manipulation of any amplitude, phase and polarization at any point in space and time (figure 20).

Zoom In Zoom Out Reset image size

However the general case where the optical beam is fully 3D of the form, F(x,y,t), and could have any amplitude, phase and polarization at any point in space and time was not experimentally realizable. Hence this class of beams encompassing the majority of spatiotemporal beams, were not experimentally achievable. It would be possible to create such 3D beams directly in the time domain, as is done for lower frequency wave phenomena like microwaves [126]. However attempting to spatially and temporally stitch together such a 3D optical wave in the time domain on femtosecond timescales using tens of thousands of coherently driven electro-optic modulators would be very difficult.

Frequency-domain approaches based on spectral pulse shaping are appropriate for femtosecond pulses, however as mentioned above, a traditional implementation using a 2D SLM can only create 2D spatiotemporal beams of the form, F(x,t) [41, 124, 125]. To generate a fully 3D optical beam of the form F(x,y,t) using a spectral pulse shaper, we employed a special multi-plane light conversion (MPLC) system to implement a 1D-to-2D spatial transformation between a 1D array of spots and a 2D set of Hermite–Gaussian modes [127, 128]. This 1D-to-2D transformation can be thought of as a kind of lookup table, which allows a traditional multi-port spectral pulse shaper to select from a 1D list which selects 2D output spatial components (Hermite–Gaussian modes). In this way a 3D beam can be mapped onto the 2D surface of a traditional SLM and completely arbitrary vector spatiotemporal beams can be controllably created for the first time [27], potentially opening up new applications in deep imaging through complex media [129], as well as nonlinear optics [12] and light-matter interactions [130] as traditionally unrealizable spatiotemporal phenomena could now be observed.

This arbitrary vector spatiotemporal beamshaper [27] enables a new class of optical beam, the most general class, where the beam is an arbitrary function of space, time and polarization (figure 21). We believe the fact that previously unrealizable spatiotemporal beams are now available experimentally could open up applications in both fundamental and applied optics, particularly in areas of nonlinear optics and light-matter interactions. Yet only a single device [27] has been demonstrated at time of writing, and wide adoption would benefit from several advances.

Zoom In Zoom Out Reset image size

Lower Loss: the first device was based on a simplified, single SLM design which was relatively lossy (∼17 dB). The first prototype was not optimized for loss, however given commercial spectral pulse shapers typically have ∼4 dB of loss, and this spatiotemporal beamshaper has multiple additional components, we believe an implementation below 8 dB of loss would be difficult in practice.

Higher power handling: the device is composed mostly of free-space optics which could be suitably designed for very high power operation. The maximum peak and average power such a device could handle would ultimately be constrained by the power handling capabilities of the SLM.

More spatial DoF: the total spatial complexity of the beams, are primarily set by two aspects of the optical system. First, a large number of spatial DoF must be available as independent pixels on the SLM. Second, this single axis of pixels on the SLM must be convertible to 2D spatial modes using a 1D-to-2D spatial transformation that supports the required number of modes. Extending beyond approximately 100 spatial modes would be challenging both from the perspective of a traditional ∼2k resolution SLM, but also for the 1D-to-2D transform.

More spectral/temporal DoF: it is relatively straightforward to adjust the device design to support either shorter temporal features, or longer delays. However the ability to do both simultaneously and control a larger total number of temporal features, is ultimately set by the number of addressable pixels on the SLM along the frequency/time axis.

Different wavelength bands: the first device was the combination of a wavelength selective switch and MPLC mode multiplexer [127] both taken from optical fiber communications, and hence designed for operation at 1550 nm. Implementing devices for other wavelength bands such as 800 nm (for use with Ti:sapphire lasers) and 1 μm (for use with ytterbium based high-power lasers) would open up applications in these areas.

Ultrawide bandwidth: modifying the optical design for operation at different wavelength bands is largely a matter of appropriately scaling dimensions, changing optical coatings, etc. What is more challenging is designing a device for a large fractional bandwidth. For example, for applications such as few cycle pulse manipulation, with octave(s) of bandwidth, the design becomes difficult. In particular, the 1D-to-2D spatial transformation is difficult in practice to scale to very large fractional bandwidths as doing so requires a large number of phase planes.

Higher resolution, physically larger SLMs: SLMs are based on LCoS technology often derived from consumer displays. Although there is a drive for consumers towards larger pixel counts for 4k and 8k video, which benefits spatiotemporal beamshaping, there is also a consumer drive towards physically smaller pixels, which reduces cost by allowing more panels to be fabricated on the same wafer. Smaller pixels mean more crosstalk between adjacent pixels particularly when working with phase-only modulation and longer wavelengths. Physically larger pixels would be desirable not only to minimize crosstalk but also as the associated larger form factor of the panel would mean better heat dissipation properties for high optical power operation.

Spatial/spectral slicing using multiple SLMs: however the number of total pixels on a single SLM panel will always have limits, and at some scale of spatiotemporal complexity the addressable DoF will have to be split amongst multiple SLMs. For distributing the spatial DoF amongst multiple SLMs, one or more panels along the longitudinal direction of the optical path could redistribute light amongst later SLMs each responsible for a subset of the total spatial/polarization modes. This could have the additional advantage of interfacing with a 2D spot array to 2D HG mode sorter [128], which is inherently more efficient than the 1D-to-2D spatial transformation approach. Stitching together the frequency/time axis across multiple SLMs would likely be more difficult and tedious.

Low-loss, high plane count MPLC: to scale the 1D-to-2D spatial transformation to higher mode counts and/or larger optical bandwidths, it will be necessary to increase the number of phase planes, which in practice means more loss and tighter mechanical tolerances. To decrease loss, it is expected that the transformation will have to move towards dielectric rather than metallic reflective coatings and the smoothness of each plane will have to be increased to decrease scattering losses. The transformation would ideally be nanofabricated as a whole, rather than assembled manually from parts to meet the stricter mechanical tolerances required by larger plane counts. It is also possible that some combination of MPLC and waveguide structures may be necessary. Particularly for fanning out 2D spot arrays to 1D spot arrays, as this is relatively easy for waveguides to perform but difficult to perform diffractively or refractively using discrete planes.

Fixed mask: for applications that do not require reconfigurability, and simply require the same spatiotemporal beam every time, the phase mask typically displayed on the SLM need not be implemented on an SLM at all, and could be implemented as a fixed mask. This would have the advantage of supporting far more pixels than an SLM and supporting higher powers.

Although research into spatial and temporal beamshaping are separately mature technologies, arbitrary vector spatiotemporal beamshaping is still in its infancy. As this technology advances, we hope this new ability to completely control all light's DoF simultaneously and independently will open up new practical applications related to the delivery of light spatially and temporally deep into complex media and/or to excite new nonlinear processes previously experimentally out of reach. More exotic spatiotemporal, or indeed spatiospectral beams of theoretical interest but experimentally unrealizable could also be explored for the first time.

¹Service OPERA-Photonique, Université libre de Bruxelles (ULB), Brussels, Belgium

²Laboratory for Laser Energetics, University of Rochester, 250 East River Road, Rochester, NY 14623-1299, United States of America

A crucial aspect of spatiotemporal shaping of ultrashort pulses and the associated probing of laser-matter interactions, complementary to all of the previous sections, is being able to characterize the non-separable ST aberrations. This field has been growing steadily in the past years, with a number of techniques demonstrated, each with their own key advantages and disadvantages [131, 132].

Spatiotemporal measurements are important for at least two reasons. The first is to address the spatiotemporal properties of large and expensive high-power laser systems to maximize the intensity produced on target after tight focusing. Spatiotemporal measurements allow laser scientists to assess how the pulse's properties, e.g. peak intensity, differ from ideal properties calculated using the energy, pulse duration, and focal-spot size. They also allow one to track down optical components that have chromatic aberrations or imperfections, or that are misaligned. The second purpose of spatiotemporal characterization is to confirm and address the results of spatiotemporal shaping, both to understand the shaping process and to understand how exotic spatiotemporal pulses—like those described in the previous sections—propagate and interact with physical systems.

Optical pulses can be fully characterized (without inherent loss of temporal resolution) by a variety of techniques that determine their electric field, either in the time or frequency domain. While temporally characterizing the pulse shape of an isolated ultrashort pulse requires a time-non-stationary element, often synthesized via a nonlinear interaction [133], most spatiotemporal diagnostics use linear optics to characterize the variations in temporal/spectral field properties across the transverse dimensions, x and y, by comparison to a reference field from another source or generated from the test field itself [131, 132]. In these approaches, the spatially dependent spectral phase φ(x,y,ω) is typically reconstructed up to an arbitrary spectral phase that corresponds to the phase of the reference pulse or the phase of the pulse under test at the reference point. For example, in STRIPED FISH (Spatially and Temporally Resolved Intensity and Phase Evaluation Device: Full Information from a Single Hologram) [134], spectrally filtered replicas of the pulse under test and a reference pulse interfere on a camera, allowing for the reconstruction of their spatial phase difference at the filtered frequencies (figure 22(a)). In TERMITES (Total E-field Reconstruction using a Michelson Interferometer's TEmporal Scan) [135], the test pulse interferes with a reference pulse generated from a small portion of the test pulse itself after reflection off a spherical mirror; the spatial interference is recorded as a function of the delay between the two pulses, from which a frequency-dependent interference is generated via Fourier transformation (figure 22(b)). A multispectral camera intrinsically provides spectral resolution thanks to an array of dielectric filters located before the photodetection chip, and it can provide spectrally resolved wavefront measurements when combined with conventional wavefront-sensing techniques (figure 22(c)) [136]. Similar data can be obtained using a conventional camera with preliminary filtering and demultiplexing of the test pulse [137]. Conversely, spectral interference of the field at any point in the transverse plane, sampled by an optical fiber, with a reference field also yields a spatially and spectrally resolved map of the electric field (figure 22(d)) [138]. Implementing spectral resolution, particularly on a single-shot basis to characterize low-repetition-rate laser systems, and generating an adequate reference pulse remain practical challenges.

Zoom In Zoom Out Reset image size

Spatiotemporal or spatiospectral intensity and phase data are in 3 dimensions, but one is usually limited to a 2-dimensional detector. Full spatio-temporal characterization generally requires taking multiple 2D measurements while scanning one parameter, which can result in increased noise due to fluctuations in the source, or at the very least results in longer and less convenient measuring times. Multiplexing images or interferograms on a 2D detector can drastically decrease the required number of acquisitions, but generally results in a similarly drastic reduction in the resolution either in space, frequency/time, or both. This is true for example in (figures 22(a) and (c)) where both the number of frequencies and spatial resolution is decreased for single shot operation. Operation with a small number of frequencies is however sufficient to detect common spatio-temporal couplings such as radial group delay and pulse front tilt (figure 23(a)). Developing single-shot techniques is especially important for extremely high-power lasers that operate at rep-rates below 1 pulse per second, or for trains of pulses that experience significant fluctuations from shot to shot. In some cases, the generation of an adequate reference pulse is a practical challenge. There is an additionally trade-off in the data size and usability—high-resolution ST measurements can be ∼10 s of GB, which can become cumbersome, whereas measurements such as in figure 23(a) produce much less data.

Zoom In Zoom Out Reset image size

Extending spatiotemporal characterization beyond the somewhat standard near-IR pulses corresponding to Ti:Sa laser systems is another significant challenge. Measuring pulses in the UV, extreme ultraviolet (XUV) (e.g. from high-harmonic generation), IR, and THz ranges requires to adapt the existing concepts with adequate optics and detectors. This is true as well for pulses that have an extremely broad spectrum, for example after broadening in a hollow-core fiber or multi-pass cell system, with required operation over a large wavelength range. Additionally, when ultrashort pulses are tightly focused, beyond the paraxial regime, they put additional constraints on spatial resolution and present nuanced challenges related to spatiotemporal propagation, and may introduce vector fields around the focus. Lastly, determining the spatiotemporal properties of ultrashort vector pulses in general, with spatially or temporally-varying polarization, is an even greater challenge that is treated in the following section of this roadmap.

Beyond the already discussed techniques based on measuring spatial data at different frequencies on a 2D detector, developing single-shot or few-shot techniques that make no compromises on spectral or spatial resolution is an ongoing technological challenge. It can be addressed by improved hardware and software (algorithms to reconstruct the spatio-temporal field from the measured experimental trace) to fulfil the need for better-characterized laser systems and highly shaped pulses. Technological improvements in the manufacturing of optical components (e.g. better filters) and sensors (e.g. higher pixel count, smaller pixel size) and their commercial availability will allow for improving the current generation of spatiotemporal diagnostics.

Reducing the number of acquisitions in a multi-shot device can also be achieved by compressed sensing, where choosing random sampling positions can produce similarly resolved spectral images with fewer total measurements. This has been applied to hyperspectral imaging that utilized Fourier-transform interferometry [139], where the number of shots was reduced to considerably sub-Nyquist acquisition rates (down to 15% of the Nyquist limit) by combining bandpass and non-uniform compressive sampling.

Snapshot hyperspectral imaging is a technology in common use for industrial applications (agriculture, cultural heritage, industrial sorting) where frequency-resolved images are produced in a single shot. Inspiration can be drawn from this applied field on how to apply an existing technique to ultrashort pulses, and how to additionally measure the spatiospectral phase. One example of a single-shot method is coded aperture snapshot spectral imaging, where a specifically designed (complicated) aperture, a dispersive element, and an advanced algorithm can produce the spatially-resolved spectrum on a 2D sensor in a single-shot. This has just recently been applied to spatiotemporal pulse measurements [140], showing a path towards high fidelity results (figure 23(b)).

Ultrafast metrology is evolving to answer the experimental need for better-characterized pulses interacting with physical systems. A large range of spatio-temporal diagnostics are now available, with fast progress toward extended operating ranges and more practical implementations. The current and future generation of spatiotemporal diagnostics is paramount to generating high-intensity tightly focused optical pulses but also driving physical phenomena with pulses producing novel interaction conditions via spatio-temporal coupling.

Grupo de Investigación en Aplicaciones del Láser y Fotónica, Departamento de Física Aplicada, Universidad de Salamanca, 37008 Salamanca, Spain

The rising importance of ultrashort light pulse applications increases the need of its characterization. Since the invention of the laser and the provision of ultrashort pulses in the femtosecond time scale, different challenges in their characterization have been gradually fulfilled, e.g. measuring temporal durations, amplitude and phase of linearly polarized pulses or spatiotemporal couplings. The advances in more and more complex pulse structures, currently presenting simultaneous spatial and temporal polarization shaping, here called ultrafast vector beams (see an example in figure 24), have substantially raised the measuring demands. Firstly, when the spatial dependence can be disregarded, in the so-called vector pulses the polarization state is time—and, therefore, frequency—dependent. Their measurement lies in obtaining the amplitude and phase of both polarization components, paying special attention to the accuracy in their relative phase to obtain the full information. Some successful approaches to measure time-dependent polarization pulses are based on dual-channel spectral interferometry [141], where two orthogonal polarization projections are referred to the same reference pulse, and tomographic retrieval [142], where three polarization projections are measured with any temporal diagnostics and two orthogonal polarization projections are connected thanks to the temporal measurement of a third intermediate projection. Other strategies are based on time-resolved ellipsometry or angle-multiplexed spatial-spectral interferometry. More recently, in-line polarization interferometry has been shown to accurately retrieve vector pulses using a birefringent crystal as a bulk common-path interferometer [143]. On the other hand, when the polarization state is constant, i.e. time-independent, different methods can be used to obtain the spatial dependent polarization, e.g. Stokes polarimetry (by measuring Stokes parameters with spatial resolution) or interferometric measurements based on positive operator value tomographic measurements [144].

Zoom In Zoom Out Reset image size

As expressed before, some applications make use of ultrafast vector beams with both space- and time-dependent polarization. Their characterization has recently been accomplished by using two-fold spectral interferometry [145], which is based on the combination of spatially resolved spectral interferometry from a fiber coupler interferometer and in-line polarization interferometry (figure 25). More recently, a combination of Fourier transform spectroscopy with a Mach–Zehnder interferometer using a linearly polarized reference pulse [112] and two-dimensional temporal scanning interferometry with a plane wave reference pulse [146] have been used for the same purpose. Since these techniques are very recent, improvements and new techniques are expected to be introduced in the near future to overcome the different challenges in the measurement of ultrafast vector beams.

Zoom In Zoom Out Reset image size

Although the above-mentioned progresses allow to develop the spatiotemporal and polarization characterization of pulsed beams, further challenges and needs should be addressed because of its increasing rate of application.

Apart from the complexity of the ST polarization structures—which can be reconstructed with the above commented state-of-the-art methods—, a matter to face is that the amplitude and phase of two polarization components of the electric field must be retrieved as a function of three variables: two spatial variables (in the transverse plane to the propagation) and the time (or, equivalently, the frequency). Related to this fact, there is the need to encode all the information in the detection (e.g. through interferometry or spectroscopy) in the different techniques, which is at the end related to having scan measurements. A second relevant challenge in the future is to reach single-shot solutions that can speed up the detection. This upgrade will be beneficial, especially for the measurement of low repetition rate sources (e.g. high-power systems), as well as for unstable vector pulse trains [147].

Expected upgrades will surely be related to the spectral and temporal range of application of the current techniques. In this sense, the extension to mid- and far-IR (MIR/FIR) spectral bands (including THz pulses), as well as to XUV or x-ray pulses in the attosecond scale, is highly desirable since nowadays there are many progresses in complex vector beams generated through high harmonics [148]. Their measurement is partially tackled in space and frequency, but it is not temporally resolved simultaneously, thus other detection methods and strategies will be necessary for their full characterization. In the case of single- or few-cycle pulses with CEP stabilized, the techniques will have to attain the characterization of this CEP, which is key for electric-field dependent interactions.

Most current techniques only work with completely polarized light sources, so new studies on the measurement of partially polarized pulses must be done, e.g. through spatiotemporal resolved Stokes polarimetry.

Also, state-of-the-art methods are usually limited to the transverse electric field, while the longitudinal component is neglected. This challenge must be addressed e.g. for tightly focused beams, being its characterization rather complex, as seen in 1D (temporal) previous diagnostics [149].

Finally, in integrated photonics, miniaturization and simplification of the detection is a challenge for photonic circuits, lab-on-a-chip experiments, or robust industrial detectors, among others.

Naturally, in addition to introducing new strategies, meeting the challenges outlined above will require new developments. Firstly, new schemes and processes to encode the pulse information are necessary (e.g. interferometry, diffraction, nonlinear processes), for example for single-shot detection or partially polarized beams. Secondly, new technology will be needed to accomplish those challenges.

Due to the great interest in measuring the spatial and temporal dependence at once, basic 1D (e.g. a spectrometer) or 2D (e.g. a simple CDD) detectors imply scanning techniques, so sensors resolving simultaneously different dimensions (e.g. hyperspectral detectors, spatially resolved streak cameras, polarization resolving cameras, etc) will play a key role in the development of new techniques, in particular for single-shot schemes.

In an analogous way, the extension to new spectral regions will demand the use of unconventional materials or processes for the pulse manipulation: adapting the way of creating pulse replicas and their relative delay, required nonlinear processes, polarization management, polarimetry analysis, etc. Thus, transparent materials with broadband operation are needed in general, to include new spectral ranges of interest or to retrieve pulsed vector beam at even shorter time scales. In particular, the detection will also play a significant role to work in the MIR, FIR or the XUV. Some technology is nowadays available and is used for beam characterization purposes, but the characterization of, e.g. polarized XUV pulses is partially done separately in time and in space coordinates. Therefore, upgrades are necessary whether to merge existing techniques addressing partial characterization dimensions (e.g. polarization, spatiotemporal, time) or developing new ones including all of them, while keeping the needed resolution.

Regarding the integration of the characterization in optical devices, scientific and, even more, technological developments are required to open the way. Detections using fibers or optical waveguides may be useful for this task. Also, liquid crystals or nonlinear crystals with refractive index tuned by external signals are candidates to ease the integration of the detection. In any case, the fabrication of these miniaturized devices (e.g. through laser processing, lithography..) constitutes a challenge by itself due to the resolution required and the high amount of information to be recorded in a measurement. Recent advances in XUV pulses assisted optoelectronics are promising in this field.

Finally, it is worth mentioning not only the amount of data recorded, but also its processing, which is done by specially designed computational algorithms. Together with fast data acquisition (ideally, single shot), fast algorithms are desirable to monitor the beam information in real time (preferably, at video rates). Undoubtedly, artificial intelligence will provide foremost breakthroughs on this point.

Very recently some techniques have successfully faced the challenge of measuring ultrafast vector beams [112, 145, 146]—i.e. ultrashort pulses with space and time dependent polarization—using different strategies, in all cases operating in the near IR region. It is expected that other new techniques will be developed in the next years, improving the performance of the current state of the art. Some of the main challenges that have been identified are related to the improvement of the capabilities of present techniques (e.g. the resolution or the spectral range of operation) or to increase the pulse information that can be retrieved (for example, cases where the CEP, the longitudinal electric field or pulse train instabilities, among others, are relevant). Also, faster acquisition times (preferably, single-shot detection at high repetition rates) and data processing are targeted in forthcoming developments. Technological challenges have also been highlighted, being truly relevant future advances in multidimensional detection (such as hyperspectral imaging) or micro manufacturing for integrated characterization devices.

Interdisciplinary Center of Quantum Information, State Key Laboratory of Modern Optical Instrumentation, and Zhejiang Province Key Laboratory of Quantum Technology and Device, Department of Physics, Zhejiang University, Hangzhou 310027, People's Republic of China

Over the past decades, optical computing has been of great interests due to high speed and low power consumption in comparison with traditional electronic digital counterparts. With the advantage of optical parallel processing, Fourier optics setups with bulky systems of lenses and filters have been widely explored to enable high-throughput real-time image processing. In order to integrate with optoelectronic devices, deliberate efforts have been made to miniaturize such optical computing elements with on-chip photonic circuits and nanophotonic structures. Differentiation is a fundamental mathematical operation in any field of science or engineering. Unlike phase contrast which mainly works with phase change, spatial differentiation enables edge detection for both amplitude and phase changes, which provides a robust image processing to extract important information about the boundary of objects in an image. In the past several years, there have been substantial efforts on the design of various nanophotonic structures for spatial differential operation [150–154]. For example, based on a simple surface plasmonic structure with a single layer of 50 nm-thick silver, optical spatial differentiation was experimentally demonstrated for image processing of edge detection [151]. In addition, on account of the polarization degree of freedom, the generalized spatial differentiator was proposed with a single planar interface from the spin Hall effect of light, which is an intrinsic optical effect with broad bandwidth [152]. On the other hand, the all-optical computations of time-domain differentiations were realized based on on-chip ring resonators, fiber gratings, and cross-gain modulation, and proposed as all-optical differential equation solvers or pulse reshapers for sensing and control [150].

As a development of optical computation, recently spatiotemporal differentiators have been proposed to couple both spatial and temporal computations. We design such a differentiator with single metasurface nanostructures and show that STOVs [41, 46, 59, 155, 156] can be generated by illuminating a Gaussian-shape optical pulse. Moreover, the spatiotemporal differentiators with the generated STOVs can be used to detect the sharp changes of pulse envelopes, in both spatial and temporal dimensions [155]. To overcome the challenges of fabrication defects of the spatiotemporal differentiator, we clarify the topological protection feature by constructing a synthetic parameter dimension associated with the metasurface geometry of the differentiator [156].

To show general requirements and design principles for spatiotemporal differentiators, without loss of generality, we consider a one-dimensional periodic grating structure by breaking spatial mirror symmetry [155]. Figure 26(a) schematically shows the spatial-symmetry analysis of spatiotemporal differentiators. Suppose that a pulse with both Gaussian envelopes in the spatial and temporal domains impinges the structure at normal incidence. If the structure has mirror symmetry about the plane $x = 0$ (indicated by the dashed line) (figure 26(a)), due to the mirror symmetry, the phase distribution of the transmitted pulse must also be symmetric about the mirror plane. In this case, the output pulse cannot be a first-order Hermite–Gaussian in $x$ direction, and thus the device cannot perform first-order differential operation in spatial dimension. For example, in order to break the mirror symmetry, the spatiotemporal differentiator is designed by the two silicon rods in each period etched on a silicon substrate with different sizes.

Zoom In Zoom Out Reset image size

We design the spatiotemporal differentiator by calculating the transmission spectrum function $T$of the device. Figures 26(b) and (c) show the amplitude and phase distributions of $T$ with respect to ${k_x}$ and $\Omega$, respectively. Here ${k_x}$ is the wavevector component parallel to the structure interface, and $\Omega$ is the sideband angular frequency from the central one ${\omega _0}$, i.e. $\Omega = \omega - {\omega _0}$. Furthermore, the blue circles and orange rhombi in figures 26(d) and (e) correspond to the amplitudes and phases of the transmission spectrum function $T$ along $\Omega = 0$ and ${k_x} = 0$, respectively. They show that $T$ exhibits good linear dependence on ${k_x}$ and $\Omega$ within $- 0.07 \leqslant{{{k_x}} \mathord{\left/ \right. } {{k_0}}} \leqslant0.07$ and $- 0.007 \leqslant{\Omega \mathord{\left/ \right. } {{\omega _0}}} \leqslant0.007$, and the phase shifts with $\pi$ at the minima occurring at ${k_x} = 0$ and $\Omega = 0$, respectively, where ${\omega _0}$ and ${k_0}$ are the frequency and the wavenumber of light in vacuum at the central wavelength. Therefore, around ${k_x} = 0$ and $\Omega = 0$, the transmission spectrum function has the form $T = {C_x}{k_x} + {C_t}\Omega$, where ${C_x}$ and ${C_t}$ are two complex numbers approximately with the $\pi/2$ phase shift. We note that for the input signal with the pulse envelope ${s_{{\text{in}}}}(x,t)$, after propagating through the spatiotemporal differentiator, the output signal is expressed as ${s_{\text{tran}}}(x,t) = - i{C_x}\frac{{\partial {s_{\text{in}}}}}{{\partial x}} + i{C_t}\frac{{\partial {s_{\text{in}}}}}{{\partial t}}{\text{ }}$, where ${s_{{\text{tran}}}}(x,t)$ is the envelope amplitude of the transmitted field. It indicates that the structure enables the first-order differentiation in both the spatial and temporal domains. In addition, by taking the bandwidth into account [155], the asymmetry profile of the transmission spectrum function does not have impact on the performance of the spatiotemporal differentiator. With such transmission spectrum function, the output amplitude distribution of transmitted pulse will be a typical first-order Hermite–Gaussian profile along each direction, and the phase distribution is a vortex carrying a singularity, corresponding to the central zero amplitude schematically shown in figure 26(a). Since the optical vortex exists in $\left( {x,t} \right)$ coordinate, it is a STOV carrying transverse OAM. In addition, there are much more DoF for the parameters to design the spatiotemporal differentiator for the normal incident light, by considering the spatial mirror symmetry breaking.

The application of topological spatiotemporal differentiator and the generation of STOVs is shown in figure 27, which is used to detect sharp changes of pulse envelopes. The incident pulse is with the amplitude modulation as the Zhejiang University logo in the spatiotemporal domain (figure 27(a)). Correspondingly, the phases of the incident pulse envelope are binary with only 0 or $\pi$ (figure 27(b)). Figures 27(c) and (d) correspond to the amplitude and phase distributions of the transmitted pulse envelope, respectively. Remarkably, figure 27(d) shows the generation of large numbers of adjacent STOVs in the spatiotemporal domain. Moreover, as the generated STOVs interfere with each other, the amplitude distribution of figure 27(c) shows that the constructive interference occurs at the sharp changes of the incident pulse envelope in both the spatial and temporal domains, while the destructive one strongly takes place where the amplitudes have slight variations. Therefore, the differentiation in the spatiotemporal domain results in the highlighting-sharpness effect of the incident pulse field.

Zoom In Zoom Out Reset image size

To fabricate such spatiotemporal differentiators, we recently showed that most of the challenges for fabrication defects can be overcome with the topological protection feature of spatiotemporal differentiator [156]. We elucidate that spatial mirror symmetry breaking introduces a synthetic parameter dimension associated with the metasurface geometry. More importantly, the STOV generation through spatiotemporal differentiator can be topologically protected by the configuration of vortices emerging with the synthetic parameter dimension. The strength of the topological protection is evaluated by the distance between the central vortex of the transmission spectrum function $T$ for the device and its nearest vortex in the ${k_x} - \omega$ domain. When the distance is small, two vortices with zero points in $T$ are close to each other, which means that the spatiotemporal differentiator is much more sensitive, because the vortices can be annihilated together in pairs of opposite charges. In contrast, when the distance is large enough, the vortices are stable due to topological protection from the perturbation: Since the continuous structural perturbation only continuously perturbs the contours $\operatorname{Re} \left( T \right) = 0$ and $\operatorname{Im} \left( T \right) = 0$, the intersection of the two contours still gives zero points in the ${k_x} - \omega$ domain. Therefore, we can design the topological spatiotemporal differentiator with topological protection, which has robust immunity to fabrication defects when integrated with other optoelectronic devices for potential applications.

The spatiotemporal differentiator is an optical device coupling both spatial and temporal dimension, with applications to detect the sharp changes in the spatiotemporal domain. Practically, the input spatiotemporal modulated pulse could be generated by ultrafast moving objects, and the differentiator has potential applications in detecting the movements of objects. Furthermore, the transmission spectrum function has an isolated zero amplitude in the frequency and wavevector domain, corresponding to the vortex phase distribution carrying a TC. Therefore, the device can be used to generate STOVs, which has applications in optical tweezers and optical communications. In addition, we propose that the strength of the topological protection is evaluated by the distance between the two vortices associated with a synthetic dimension. We believe that there are high-order vortices emerging in the synthetic parameter dimension by including more geometry parameters, which could be important in the applications of STOVs and spatiotemporal differentiators.

Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, United States of America

Recent developments in wavefront shaping tools such as SLMs and metasurfaces have enabled versatile control over light's phase, amplitude, and polarization, point-by-point in space, creating complex and exotic classes of structured light [1, 117]. While relatively at its infancy, orchestrating structured light in time has also attracted wide attention owing to the massive possibilities it can bring. For instance, structuring light over ultrashort time scales can be widely exploited in probing matter, thus revealing rich physics, as alluded to earlier in this roadmap. The common strategy for synthesizing spatiotemporal modes has traditionally relied on Fourier transform -based pulse synthesizers which mandate pulse spreading and combining as discussed in sections 8, 12 and 13. At the heart of these setups lies a phase mask that can impart an independent spatially varying phase profile onto each spectral component, thereby sculpting the output spatiotemporal waveform at-will. While SLMs have been deployed as phase masks in most of these setups, metasurfaces have also been widely utilized owing to their subwavelength features, ease of integration, and compact footprint, as schematically illustrated in figure 28(a) [119]. Dispersion-engineered metasurfaces are poised to dramatically simplify existing setups even further. For example, instead of spreading the spectral components of the input pulse into different regions of the metasurface (or the phase mask) then combining them with a second grating at the output, a dispersion-engineered metasurface can readily achieve the same task via single transmission. This can be realized by engineering the dispersion behavior of each meta-atom—the subwavelength-spaced scatterers comprising the metasurface; essentially allowing each wavelength of the incident pulse to 'see' a different phase shift and group delay, after a single interaction, as depicted in figure 28(b) [157]. While the two former examples were intended for light control in the temporal domain (1D), wavelength selectivity can similarly be achieved in the spatial domain (in 2D). To illustrate this picture, figure 28(c) depicts a meta-optic which can impart different phase responses onto three discrete wavelengths, i.e. projecting a wavelength-dependent hologram. Multifunctional response of this kind can be realized by spatially multiplexing the nano scatterers, each optimized for a particular wavelength, as shown in figure 28(d) [158]. Spatial interleaving of this kind, however, often leads to low efficiency and undesired diffraction losses that deteriorates the overall efficiency. Dispersion-engineered metasurfaces, on the other hand, can produce this wavelength-selective response on the level of the single meta-atom [160]. One prominent approach relies on guided mode resonances which are highly sensitive to the wavelength of incident light. A meta-atom of this kind is shown in figure 28(e). Using this technique, three completely independent spatial profiles can be encoded on red, green, and blue wavelengths, respectively, as depicted in figure 28(f) without the need for spatial interleaving [159]. In analogy, by sculpting the spatial profile of each 'color' of an incoming pulse, a single meta-optic can in principle produce a spatiotemporal waveform using a very compact and integrated setup. However, given the intrinsic difficulty in decoupling the spatial profiles of spectral lines with fine (sub nanometer) separation for the case of ultrashort pulses, this development is still the subject of ongoing research. If realized, optics of this kind can simply transform an existing pulsed source to a complex spatiotemporal waveform generator, pushing forward the science and applications of this field.

Zoom In Zoom Out Reset image size

Dispersion-engineered metasurfaces stand as a very promising platform for creating wavelength-dependent response as demonstrated in various applications, from multi-color holography to achromatic and wavelength-selective focusing [160]. In contrast to these applications, which typically deploy a CW laser, spatiotemporal pulse shaping inherently mandates the use of a pulsed source. This imposes a limitation on the smallest spectral resolution that can be achieved; that is the maximum number of phase profiles that can imparted by the meta-optic onto the spectral lines, independently. In multi-color holography, for instance, in which the spectral components (R, G, B) are sufficiently separated—a few tens of nano meters apart—it is feasible to decouple the phase profiles assigned to each wavelength. However, this is not the case in spatiotemporal shaping of an incoming pulse with either finely spaced frequency comb lines or a continuous spectrum. Broadband focusing with metasurfaces tackles this by engineering not only the target phase profile at the desired wavelength but also its higher order derivative (group delay and group delay dispersion), thereby tailoring the dispersion relation over a spectral continuum. However, in spatiotemporal light shaping—where the target phase can be more complex than a lens function, or where the dispersion relation is not a continuous or analytic function—it becomes more adequate to impart independent phase profiles onto a discrete set of closely separated wavelengths. Obtaining an independent phase response for two closely separated spectral lines (a few nanometers apart), on the level of each meta-atom, becomes very challenging as it typically mandates the use of resonant structures which are highly sensitive to fabrication intolerances, often altering the target response. Another challenge lies in the need for high aspect ratio meta-atoms; that is using taller metasurface unit cells to achieve the phase redundancy (satisfying the same target phase with multiple meta-atom geometries) required for multi-wavelength response. This often poses a limitation in terms of the fabrication time and robustness of the final device. Multi-wavelength phase coverage can also be achieved using more complex meta-atom topologies based on free-form optimization or multi-layered metasurfaces which often introduce additional challenges in terms of computation, fabrication requirements, and design. Lastly, given their static nature, passive metasurfaces lack the dynamic behavior afforded by SLMs which can be desired in some applications; for e.g. in sensing or free space communications. While active metasurfaces can mitigate this issue, their spatial resolution and multi-wavelength response is still the subject of ongoing research.

Tremendous efforts are still underway to generate spatiotemporal waveforms using compact and integrated setups that directly operate on the input pulse through a single interaction. Local and nonlocal metasurfaces tackle this task in conceptually different ways, by either operating on incident light in real or k-space, respectively. One example of the former is schematically illustrated in figure 29(a); a dispersion engineered meta-optic operates on a pulsed source, transforming it in real space to a time-varying optical vortex—i.e. a beam carrying OAM while changing its OAM state $\ell$, instantaneously. To achieve this, the meta-optic generates a discrete number of propagating modes—for e.g. higher order Bessel beams—equally spaced in the frequency domain such that their temporal beating modulates the envelope's intensity and OAM state in time, akin to a Fourier series (figure 29(b)) [161]. The beating of the copropagating modes can be precisely engineered over an ultrashort time scale by varying the frequency separation and spatial profiles of the copropagating modes. This design principle can be implemented using a frequency comb source as the input pulse whose spectral lines are in turn independently shaped using a dispersion-engineered metasurface, similar to the one depicted in figure 28(e), imparting different Bessel profiles onto different wavelengths (figure 29(c)). Since the comb lines are generally tightly spaced (∼1 nm or less), then a Fabry–Perot etalon is needed to filter out the desired number of coherent spectral lines, ensuring that their frequency spacing is compatible with the meta-optic. Hence, by integrating a single flat optic, a regular frequency comb source can be transformed to a spatiotemporal pulse generator giving rise to complex and dynamic spatial profiles at ultrashort time scales (figure 29(d)). Other variations of this approach that rely on pulse spreading and combining have also been recently demonstrated as discussed in section 3 [23, 25]. Notably, spatiotemporal pulse shaping with metasurfaces has also been exploited in dynamic beam steering without the use of mechanical components, as depicted in figure 29(e) [162]. In this configuration, a (virtual) frequency-gradient metasurface is created by combining a frequency comb source ($\lambda = 720\,$ nm) with a passive metasurface made of silicon, thereby mapping each incoming spectral line into a spatial optical mode with a different wave vector. Since the spectral lines are phase-locked, their corresponding spatial patterns constructively interfere to generate a 4D dynamic pattern that naturally evolves in time, steering ${25^ \circ }$ in $8$ ps. In contrast to these methods which operate on light locally in real space, nonlocal metasurfaces can engineer the dispersion of incoming pulse directly in k-space, akin to photonic crystals, as described more fully in section 10.

Zoom In Zoom Out Reset image size

Ultrafast pulse shaping in space and time, simultaneously, is only a first step towards generating multidimensional structured light. Light's polarization and spatial coherence are two additional degrees-of-freedom that, we believe, will be widely studied in this field moving forward as alluded to in sections 8 and 13. For instance, by considering the full vectorial nature of light under tight focusing, one can create highly nonparaxial spatiotemporal waveforms with intriguing optical force characteristics. While current experimental setups come in different forms, acting on incident light either locally or nonlocally, we believe that hybrid setups that combine SLMs (for their reconfigurability) and metasurfaces (for their dispersion-engineering capabilities and versatile polarization response) will advance this field of research and accelerate its use in future applications. The abundance of these wavefront shaping tools, ongoing advances in nanotechnology, and emerging trends in inverse design will inevitably enable new modalities for manipulating the fastest magnetic, topological, molecular, and quantum excitations at the nanoscale with high ST resolution in 4D.

School of Physics, University of the Witwatersrand, Private Bag 3, Wits 2050, South Africa

Light can be tailored, much like cloth, weaving a pattern using all of its DoFs, in time and space, for so-called structured light [1]. Traditionally this has been done with classical light, for example, amplitude, phase and polarization control in space [163] and time/frequency control [164] to move to higher-dimensional forms of structured light [2]. The quantum control of structured light is far less developed. A good example is the classical and quantum control of OAM (figure 30), the former appearing in the early 1990s and the latter only a decade or more later. Quantum states based on spatial modes as a basis can give rise to (in theory) infinite state spaces, described by a bi-photon state given by

$$\begin{align}|\psi \rangle = \mathop \sum \nolimits {c_{ij}}|{M_i}\rangle|{M_j}\rangle\,,\end{align} \tag{ 10 }$$

Zoom In Zoom Out Reset image size

where the states $|{M_i}\rangle$ and $|{M_j}\rangle$ describe orthogonal spatial modes (the basis) of two entangled photons, with a probability of detection given by $|{c_{ij}}{|^2}$. The dimensionality is determined by the choice of basis (i.e. the modes selected) and the choice of how the modes are created and detected. Here the dimensionality (d) follows the standard definition that there are d terms in equation (10) to describe the state, which implies a Hilbert space of the bi-photon of d × d = d², and a density matrix with ∼d⁴ independent terms. The state coefficients, c_ij are controlled by post selection or by state preparation, e.g. if they are tailored to be equal then the state is maximally entangled [165], while from most quantum sources the coefficients would fall off with increasing mode order, e.g. as is the case from spontaneous parametric down-conversion sources. The benefits of accessing high dimensions (d) include a large encoding alphabet that scales as log₂ d bits per photon, an upper bound to the cloning fidelity that scales as F= ½ + 1/(1 + d), with a maximum of 50% for large d (it is 83% for d = 2 qubits), and thus enhanced security in quantum key distribution. The reader is referred to [166, 167] for more comprehensive reviews.

Despite the many advances with spatial control of quantum states, most studies have only considered two-dimensional analogues to polarization, for qubits. Realizing high-dimensional state spaces with structured photons is very much in its infancy. Nevertheless, up to d = 100 dimensions has been controlled with one DoF [168] and up to ten photons in two DoFs [169]. Optical cycles are much too fast to allow direct temporal light shaping, and so such light shaping is by frequency. For example, to shape classical light temporally, the frequency components are usually path separated by a dispersive element, mapping frequency to space, shaped in amplitude and phase before a return mapping to reconstruct the desired temporal pulse. In the quantum realm, frequency comb technology has been exploited to produce time–frequency entangled qubits (two dimensional states) across two or more photons [169], extended to ten dimensions as bi-photons [170]. In this case the states $|\left. {M_{i}}\right\rangle$ and $|\left. {{M_j}} \right\rangle$ describe pure single-frequency quantum states of the signal and idler photons, both in a single spatial mode (usually a Gaussian). The phase matching condition of non-linear crystals is a mechanism for producing entangled photons (see figure 1), and here frequency entanglement is a natural outcome [171]. Time is often used directly as a means of encoding information into quantum states, particularly for quantum key distribution, with so-called time-binning an easily implemented approach to reach high dimensions [172].

The main challenge in exploiting time binning for quantum information is dispersion, which results in a mixing of states. While dispersion is nominal in free-space, it is rather significant in optical fiber. Further, packing more information into a unit of time means ever more sophisticated detection and control systems. Dispersion compensation fibers do exist, but here the spatial mode associated with the binning is a Gaussian, i.e. a single spatial mode. Multiple spatial modes can be supported in both free-space and optical fiber, but the challenge in both media is modal scattering, which over long distance or after averaging results in mixed states. Compensating this is an ongoing challenge, but with the promise of the quantum equivalent to mode division multiplexing. There exists a tremendous opportunity to merge time with more than one spatial mode for a multi-dimensional state that is high-dimensional in both spatial and temporal DoF. Temporal control is inherent in all quantum experiments by virtue of measuring in coincidence, but this is a control parameter rather than a controllable DoF. Introducing temporal control in spatial mode entanglement is hindered by the typically slow responses of spatial mode detectors, where patterns are typically detected at rates in the order of display technology (10–100 Hz).

A common limitation to both time and space is that the high-dimensional quantum toolkit for state measurement is still very slow and susceptible to noise. For instance, a quantum state tomography takes on the order of d⁴ measurements to construct an unknown state, a process than can take weeks for even modest values of d. This is because quantum measurements give statistical outcomes and so many runs of a single measurement are needed. For instance, at an integration time of 10 s (time per measurement) a d = 20 dimensional state will need ∼19 d of measurement time, excluding auto-alignment during this period and reconstruction of the raw data, which typically takes as long as the experiment. Faster quantitative tools do exist in each DoF, but have yet to be translated across DoFs. Further, dispersion and scattering means that quantum transport of spatiotemporal states is limited in reach, yet the quantum teleportation and repeater tools are not yet developed beyond qubits, very much the focus of cutting edge research the world over.

Despite the challenges, there are many opportunities and exciting prospects to follow to move the state-of-the-art forward. One novel approach is to mix DoFs in such a way that each photon lives in a state space spanned by only one DoF. For instance, it is possible to mix polarization and spatial modes to realize hybrid entangled states, with photon A polarization entangled but in a Gaussian mode, and photon B spatial mode entangled but with one polarization. This has allowed the Gaussian photon (A) to pass through single mode fiber, while exploiting photon B to access an infinite number of spatial mode combinations, for an infinite number of two dimensional states [173]. Such quantum tricks could be extended to time quite easily, allowing an infinite number of d-dimensional states to be realized. Another major opportunity is to create and exploit topologically stable quantum states [174]. Topology and quantum have not traditionally been combined, yet here the promise is entanglement robust to perturbations, for long reach and long-lived states in quantum circuits and network. Finally, the low efficiencies inherent in most quantum experiments could be enhanced by a move from natural materials to artificial atoms that make the building blocks of metamaterials and metasurfaces, for ultra-high non-linearity and unprecedented spatial and frequency control for on-demand entanglement in space and time. Indeed, spatiotemporal states have been suggested to enhance sensitivity in metrology when combined with non-linear optical approaches [175]. The problem of quantum gates and operators for high dimensional states remains highly challenging (what is the equivalent of the beam splitter for d-dimensions?), but with progress of late that promises enhanced functionality in state control [176] and measurement [177]. This is a topic largely untouched yet crucial for practical quantum information processing.

Photonics is an exciting prospect in the context of the emerging second quantum revolution, where entanglement is harnessed for quantum information processing, imaging and sensing. The opportunity is to exploit the weakly interacting nature of photons for robust transport of quantum information, but packing information densely into quantum states remains an open challenge, as does getting the information out and integrated into solid state storage systems. Leveraging all DoFs of a photon may hold the key to unlocking this potential, to move us from polarization qubits to multi- and high-dimensional spatiotemporal states.